All organizations, whether business, government, or nonprofit, should be doing more to measure their impact. Samantha Yamada, a doctoral student in psychology at York University, makes the case for measuring program impact in a TEDx presentation (see below). The example she uses is an innovative program for adolescents and their families struggling with substance abuse and mental health problems. She helped get the program started and led the evaluation effort. She reports that measuring impact has helped improve the program, secure long-term funding, and convince youth that this is a good place for them.
I would add that the process of asking questions about something (which is what we do in evaluation work) brings attention to the program and it causes stakeholders to reflect on the program’s design and value. This inquiry builds engagement with and commitment to the program.
Consultants like to quip, “What gets measured, gets done.” This is not necessarily true. In 2007, banks knew foreclosures were occurring at an alarming rate but that didn’t stop loan officers from making risky loans. Colleges know that their student dropout rates are very high, but that hasn’t resulted in much change in students’ college experience. Many hospitals know that they have a high infection rate post-surgery, but that hasn’t changed the behavior of their doctors and nurses. Measuring something doesn’t guarantee that behavior will change.
It takes more than measurement to change practice; it takes a passion for improvement and success. It’s essential to have the data, but you need to put that data in front of stakeholders and help them think through the implications, discuss what needs to be done to improve the situation, commit to making a change, and follow through on that change, whether it is tighter banking controls, implementing student retention programs, or requiring surgical teams to use a checklist for preventing infection.
Conversely, if you are not going to try to improve a program (that is, you just want to keep doing what you’ve been doing) and outcomes are not important to know (that is, you care mainly about keeping the program going and positive results, if you get them, are just icing on the cake), then don’t evaluate. Many managers in organizations care only about delivering programs, not achieving important results (e.g., employee training programs). Those managers shouldn’t be wasting time, money, and effort on program evaluation.