Carrying Out Evaluation – Guidance

Now it’s time to follow the evaluation work plan you drafted in Chapter 3 and finalized in the previous step on “Preparing for Evaluation.” Follow the steps listed below to evaluate the effectiveness of your program.

1. Collect the worksheets from chapter 2 entitled, Writing Objectives Outcome Objective Evaluation Worksheet and Writing Objectives Process Objective Evaluation Worksheet. From chapter 3, review the evaluation portion of your nutrition and physical activity plan, which should include a description of your program’s impact, evaluation objectives, an evaluation work plan, and your intended use of evaluation data.
2. Collect the final evaluation work plan.
3. Based on the information in the evaluation worksheets from chapter 2 and your evaluation work plan, make a list of the people who will be involved with your evaluation. Include people who will help collect evaluation data such as co-workers, partners, and evaluation experts. Also list the people who manage the programs that you will be evaluating. For example, if you are collecting program data on the local fitness centers, then the directors of the fitness centers would be on your list. Refer to this list as evaluation partners.
4. Meet with your evaluation partners and let them know how they will be involved with evaluation. Ideally the evaluation partners have been involved at some level before now. If you are meeting someone for the first time who is on your list, they will likely be resistant to helping. Even people who have been involved may resist participating in evaluation. If evaluation partners are not willing to help, the best advice is to be patient, do what you can this year, and try again next year.
5. Based on the information in the evaluation worksheets from chapter 2 and your evaluation work plan, make a list of the currently available measuring tools (databases and surveys) that you will use. For example, to track the prevalence of a health condition or a health behavior among adults, the best survey data is from the existing Behavioral Risk Factor Surveillance System (BRFSS), which your state health department conducts annually. See the “Data Resources” handout in chapter 1 for a list and brief description of available data sources and databases with nutrition, physical activity, and health information.
6. Obtain copies of the existing surveys that you will use, and make sure you know how to access the existing databases and data tools that you need.
7. Based on the information in the evaluation worksheets from chapter 2 and your evaluation work plan, make a list of the forms, surveys, and databases that you will need to create. For example, if you are trying to increase the number of worksites that offer worksite wellness programs, you probably need to create the form that will evaluate this objective. And you might need to create a survey for worksite wellness participants to complete midway through the program.
8. Develop draft versions of the forms, surveys, and databases that you need to create. Always test a survey or form that you created and that other people will complete. Revise the survey or form based on test results. For example, if you create a survey for worksite wellness participants, then you need to ask a few people in the program to complete the draft survey. From this testing you may learn things, like the wording of one question generates off-target responses, for example, or one question is slightly offensive to some people. Revise the survey to address these problems. Evaluation tools can be refined later as you start to evaluate if you find they aren’t generating data that you can use. If you want to collect health behavior data and health status data on the adults and youth in your community or in your program, then use the questions from the Behavioral Risk Factor Surveillance System (BRFSS) and the Youth Risk Behavior Surveillance System (YRBSS). The questions in these surveys have been developed and tested by national evaluation experts.
9. As you go through these steps, modify the evaluation work plan as needed. Although it is not necessary, you may find it useful to create a stand-alone evaluation plan that includes your Evaluation Work Plan, an evaluation schedule or timeline, and evaluation forms.
10. Collect evaluation information and data as scheduled according to the evaluation work plan.
11. As results come in, study and analyze the findings. Be sure to share results with coalition members and evaluation partners as appropriate. Discuss the findings with coalition members.
12. Generate recommendations to improve programs, services, and activities based on evaluation findings.
13. Write reports as appropriate for funders, supervisors, program participants, and your community. Refer to the Intended Use of Evaluation Data section of your nutrition and physical activity plan for ways that you intend to use evaluation data. A self-study guide written by staff at the Centers for Disease Control and Prevention includes a chapter with information and worksheets to help professionals share evaluation findings. The guide is entitled, Introduction to Program Evaluation for Public Health Programs and the chapter is “Step 6: Ensure Use of Evaluation Findings and Share Lessons Learned.” You can access the guide from the CDC Evaluation Working Group’s website at CDC Evaluation Working Group website.