Program Evaluation- Don't Skip It!
It is time to look at program evaluation for 2022 and planning for 2023. Whether you are a non-profit, labor union, independent expenditure organization, issue group, or political campaign, taking the time to evaluate the success of your program is a necessary step to future success. Win or lose, understanding what worked and what did not is important, and it can be the beginning of real future improvements.
We get it, after a loss or even a hard-fought win the last thing you want to do is evaluate. You want to book a vacation, shut down your laptop, and not look back to your win or lost. But even though this takes a little bit of time, it is better to get program evaluation out of the way than letting it sit or frankly saying you will do it and never get there.
For a thoughtful look at program evaluation download and read our program evaluation eBook.
Our program evaluation overview:
Create a timeline for evaluation: Program evaluations will take time. Don’t rush it–set a timeline and stick with it. Have initial and longer-term goals for your evaluation. A good program evaluation will take a few weeks or longer to put together so don’t wait too long to get started.
What does success look like?: Beyond winning or losing, having primary and secondary goals to judge whether you have achieved success is important. Hopefully, you had goals listed in a membership, campaign, or organizational plan. Goals could include increasing diversity, fundraising, increasing engagement, building capacity, list building, increasing engagement in specific areas, and/or data collection.
Do folks really agree?: Just because you have a set of metrics and goals does not mean that you team members and organizational partners have the same goals. Make sure you understand and agree on goals and listen to each other. Make sure you leave room for disagreement and some conflict – without it you likely won’t achieve your shared goals.
Engage coalition partners: Make sure you reach out to coalition members to get their goals, data, and opinions. Understand what they think worked and what their concerns are. Take the time to listen and get real concerns.
Who did you miss?: Sometimes the best insights come from non-traditional allies, folks who should have been at the table but were not asked or other observers, elected officials, business owners, etc. Make sure you go beyond your inner circle to evaluate your program.
Someone needs to be in charge: Make sure someone oversees the program evaluation process, whether that is a staff person in your organization or an outside consultant. Outside help can facilitate a single meeting or help build and write a detailed report. Program evaluation is a service that TCW provides. Reach out if you are interested in learning more.
Organize a meeting: Pull together an initial Zoom meeting or call to discuss results and information you will need to create a report. Be as inclusive as you can and create a clear agenda. Do this to discuss both wins and losses. Make sure you have another meeting set when all this comes in.
Field a survey: Work with your team to ask a series of questions to folks who have been involved in your program. Work to get input on actionable questions or tasks in the survey. Use a tool like SurveyMonkey to collect and analyze survey data (larger programs may want to engage a professional polling company).
Interview partners: Don’t stop with a survey. Look at the survey and ask what are other questions that need to be answered, then create a list of folks to have a detailed interview with. The interviews will get you more detailed answers and follow up.
Interview staff and supporters: Don’t stop at the heads of the organizations. Staff, members, and donors may be a better source of information than leadership.
Interview treatment targets: If your program was focused on members or voters, take time to interview your targets to see how they saw the program.
Collect data and metrics: Data will come in varying forms and timing. Election data will take a while to get applied to voter files. This will usually happen five to eight months after the election. Metrics from ad campaigns, email and social metrics can and should be collected as soon as possible.
For the short term: Program evaluation can give insight to what worked and what did not. What goals were achieved, what were not.
For the long term: A program evaluation is something that should not be done just once. The benefit of a long-term program evaluation process is the more you do it, the more results you will see. Create a process and stick with it.
Share a vision: For the next program, based on what you achieved or did not achieve, create, and have a vision for what the future would be. Understand in what areas you need to be more successful, maybe that is dedicating more resources, a more defined strategy, new tactics, or a more diverse coalition. Learn from your evaluation, create a vision, and try again.
Don’t get distracted: Often folks don’t do full evaluations because it takes time and there are other priorities. The process of evaluation will allow you to set future goals and help your program grow. This can be a powerful exercise.
Define your audience to share findings: Have a defined list to share findings with. Be clear on who the audience for your findings is and who the audience is not. Use this as way to continue to build for the next year and future programs.
Keep planning, testing, and evaluating: Program evaluation is not a one and done kind of thing, learn what worked and set goals for the future to grow your program and test new tactics.
Get help: Pulling together a real program evaluation will take time and planning. Depending on the scale and scope of you program, you will need dedicated staff time and you may need outside help. There are a range of companies (and costs) for program evaluation, including us. Make sure you take time, internally or externally, to get the help you need.
For a comprehensive look at program evaluation download our program evaluation eBook.
Have questions about program evaluation and program planning? Drop us a note we are happy to chat.