We use the word impact a lot in the nonprofit sector, and often when we talk about telling our story or demonstrating the difference we make. However, when we’re talking about evaluation, this word has a very specific meaning. Better Evaluation defines impact evaluation as going beyond describing or measuring impacts that have occurred to seeking to understand the role of the intervention in producing these (understanding if the program caused the change/impact or something else).
Today I read an interesting article in the Stanford Social Innovation Review: Ten Reasons Not to Measure Impact—and What to Do Instead. While there are definite benefits to engaging in an impact evaluation, it’s not always the best type of evaluation to meet your needs. The authors break down their top 10 reasons NOT to measure impact into 4 buckets:
- Not the right tool (it can’t answer the questions you have)
- Not now (design or implementation isn’t ready yet, or it’s too late)
- Not feasible (don’t have the resources, or large enough sample, can’t measure well other factors that might influence outcomes, not possible to stick to research protocols)
- Not worth it (not worth the cost, since it’s already been studied, or you won’t learn enough to make it worth the cost)