For our second assignment, we were asked to select an appropriate method to evaluate a program designed to prevent diabetes in high risk groups, by providing free access to exercise programs to pregnant aboriginal women. Participation in the program would hopefully prevent cases of gestational diabetes mellitus, and type 2 diabetes in pregnant women, therefore hopefully preventing occurrences of type 2 diabetes in the children of these women (Klomp, Dyck, Sheppard, 2003).
I feel that the Outcomes-Based Evaluation method fits with this program, but I realize that it also probably speaks to my own personal philosophies about what makes a program successful. The Outcomes Based Evaluation method, as described on managementhelp.org, focuses on the impact that the program has on the users.
To make my decision, I envisioned the program using a Program Logic Model, as explained on the University of Missouri's website, reflecting on who is involved in the program, and the desired short, medium and long-term outcomes. This lead me to the Outcomes-Based Evaluation method, which is especially appropriate for the non-profit sector (Plantz, 2006). A review of the guiding questions on Managementhelp.org also directed me to this method. Although there are many things that could be assessed, including how the classes were delivered, the qualifications of the instructors, and several other factors, the only thing that truly matters is whether or not the program helps to reduce occurrence of diabetes.
Managementhelp.org provides a brief overview of the steps involved in this type of evaluation, and based on the information provided in the case study, I feel that it would be possible to work through these steps. Beginning with identifying the outcome to be measured (I would consider repeat participation over multiple weeks, as well as instances of GSM), moving on to identifying observable measures (attendance lists and instances of GSM). Information about the target group is clearly available, as described in the case study, and the outcomes could be assessed using a telephone survey. Participant contact information is available, as participants were contacted each week with a reminder to attend, therefore a follow up telephone survey or interview should be easy to obtain. The biggest challenge with using this method would be tracking the long-term outcomes desired. Short term outcomes - for example, participation in the program, could be easily tracked. Medium-term outcomes - the decreased occurrence of GSM, could be tracked with telephone surveys. The decreased occurrence of type 2 diabetes in the children of the participants would take years to track. However, success in short and medium term goals would likely be enough to evaluate the program in the meantime, as studies have proven that these outcomes do impact the occurrence of type 2 diabetes in children. This program evaluation would likely drive further participation, by program participants, and program leaders or employees, as it is the method that most clearly focuses on the good that comes from this initiative.
References
Candidate Outcome Indicators: Health Risk Reduction Program. (n.d.). The Urban Insitute. Retrieved February 6, 2014, from http://www.urban.org/center/met/projects/upload/Health_Risk_Reduc.pdf
Klomp, H., Dyck, R., and Sheppard, S. (2003). Description and evaluation of a prenatal exercise program for urban Aboriginal women. Canadian Journal of Diabetes, 27: 231–238.
McNamara, C. (n.d.). Basic Guide to Program Evaluation (Including Outcomes Evaluation). Basic Guide to Program Evaluation (Including Many Additional Resources). Retrieved February 06, 2014, from http://managementhelp.org/evaluation/program-evaluation-guide.htm#anchor184773
Plantz, M. C., Greenway, M. T., & Hendricks, M. (2006, March 3). Outcome management: Showing results in the nonprofit secotr. Outcome Measurement Resource Network. Retrieved February 6, 2014, from https://www.nationalserviceresources.gov/files/legacy/filemanager/download/ProgramMgmt/Outcome_Measurement_Showing_Results_Nonprofit_Sector.pdf
Program Development & Planning: ProgramLogic Model. (n.d.). Program Development & Planning: ProgramLogic Model. Retrieved February 06, 2014, from http://extension.missouri.edu/staff/programdev/plm/
Karen
ReplyDeleteI agree with your evaluation approach. The use of outcome-based is appropriate in this situation. Access to multiple outcomes would generate a comprehensive look at the effectiveness of the program. Looking at immediate and medium term outcomes is a certainly within reach for this program. Keeping track of long term outcomes is difficult but quite helpful in judging the greater impact of the program.
Jay