Building a Culture of Evaluation in Your Organization
Sarah M. Dunifon
A boy stands in front of a statue of an astronaut and mimics its victory pose
Back in February, we surveyed our newsletter subscribers about what they were most interested in reading in upcoming features. In future Insights editions, we’ll be reflecting on the topics that you indicated would be most beneficial to you.
Building a culture of evaluation emerged as a top interest in the poll. As a consultant who works with organizations to conduct evaluations and build evaluation capacity, this is no surprise! One of the most challenging parts of evaluating programs can be figuring out how to embed the practice into existing structures. This often starts with changing people’s perceptions of what evaluation is and why it is so necessary.
Especially for folks coming from the world of formal education, evaluation, assessment, and research can seem like terms associated with compliance rather than learning and improvement. They can be seen as scary things that exist just to tell you how wrong you’re doing your job. This cannot be further from the truth. In the informal learning world, research and evaluation exist to explore questions and interests, provide learning opportunities, and improve practice. We often employ a strengths-based approach, honing in on what is going right in a program so that we can capture that goodness and amplify it.
Often, evaluation is seen as an add-on to programs. Sometimes conducted by external evaluators, sometimes conducted by the program team, but more often than not, evaluation is seen as a “nice to have” rather than a “need to have.” Part of building a culture of evaluation is developing the understanding that evaluation is a crucial part of the program cycle. Without it, we might be missing learning objectives, not reaching our target audiences, or underperforming on anticipated outcomes, and - we might not even know it! Evaluation gives us the chance to understand what is happening and improve our practice in real-time.
Building a culture of evaluation in your organization may start with addressing misconceptions about the purpose and use of evaluation. It is helpful to discuss folks’ current understandings and concerns, and get everyone on the same page about how it will be used in your context. Questions you may include in the conversation might be:
Who has had experience with evaluation before? What did it look like?
What purpose do you think evaluation will serve in our organization?
What might we gain from incorporating evaluation?
What hurdles might we face in incorporating evaluation?
It’s important to recognize that folks may have real and valid concerns about incorporating evaluation into their programs. Especially now, a lack of resources may mean that program managers, coordinators, or educators are asked to add evaluation tasks on top of their already heavy workload. This can be intimidating (as program evaluation isn’t always included in educational programs like M.Eds., for example) and can feel overwhelming (“Where do I start? How can I manage all of this on top of my other work?”)
So, the next important step in building a culture of evaluation is to figure out how it fits into existing programs. Rather than tacking evaluation on top, consider how it can be embedded into your processes. For example, many people think about conducting a post-program survey to assess learning outcomes. Functionally, this is an extra step at the end of the program for whoever is conducting the evaluation. They’ll need to end the program, introduce the survey, pass it out, answer questions, wait for participants to finish, collect it back, and upload or otherwise store the information. You may also need to administer consent forms ahead of time (if, for example, working with youth participants or sensitive content), conduct an assent procedure before handing out surveys to get youth buy-in, etc. All of that might be okay, but if your staff is already stretched to the max, it is likely to be met with resistance and may not be conducted properly in the end.
What if you instead incorporated an evaluation component that fit more seamlessly into the existing program? For example, during an interactive component, participants could record their ideas on a large whiteboard, requiring the facilitator to just take a photo of the responses for later analysis. The right solution for you will be context-dependent. You will have to consider what questions you’re trying to answer, what data you’d like to collect, who your participants are, and many more factors we’ll address in a future feature on choosing the right evaluation instruments. The bottom line is that there are ways to make evaluation an easier “lift” than you might think.
If you’d like to start building a culture of evaluation, a good starting point would be with evaluation activities that do not strain staff or participants. You want it to be a good experience for everyone involved. A nice follow-up to incorporating some of these activities might be to have periodic “data parties” or share-outs in meetings. Questions you might ask are:
Were there any interesting insights you gathered from the feedback?
Have you noticed any changes to the data over time?
How might we tweak our programs in light of these findings?
Helping your staff know how to interpret and use said data is an important part of building a culture of evaluation.
Whew! Okay, that was a long one. To sum it all up, building a culture of evaluation is a combination of addressing understanding and attitudes towards evaluation and building evaluation systems that are accessible and show how the data collected is useful and important.
Thank you to everyone who provided feedback on our newsletter poll. We always appreciate hearing from you and learning more about what interests you! If you have requests for upcoming features, questions about evaluation, or are interested in working with us, you can always reach out to hello@improvedinsights.com.
We hope you enjoyed this article! If you’d like to see more content like this as it comes out, subscribe to our newsletter Insights & Opportunities. Subscribers get first access to our blog posts, as well as Improved Insights updates and our 60-Second Suggestions. Join us!