Creating a non-randomized comparison group may be viable but is often hard to do and quite expensive. Undertaking a program in another context provides powerful policy information about when and where a given intervention will work.
How will lump-sum versus spread-out transfers influence short-run investment choices? If this stratagem ultimately leads to more effective impact evaluations, so much the better. But what if the real world takes over and politics or funding mean you must evaluate now or never? Such short-run operational questions may be amenable to evaluation.
How applicable is the context under which those studies were done, and how similar is the intervention? Thinking through the theory of change is the first step to planning out a monitoring or evaluation strategy.
A lack of data about program implementation could hide flaws that are weakening a program. Maintaining strict experimental protocols could be costly, compromising the quality of the implementation. Credible data are valid, reliable, and appropriately analyzed.
Good impact evaluations—those that answer policy-relevant questions with rigor—have improved development knowledge, policy, and practice.
The evidence of impact was strong: But even if IPA deserves only 10 percent of the credit for the policy changes behind the benefits calculated above, the ratio of benefits to costs is still 7. Too little data can also lead to inefficient programs persisting, and thus money wasted.
What if not all mothers complete the training? Despite the demonstrated value of high-quality impact evaluations, a great deal of money and time has been wasted on poorly designed, poorly implemented, and poorly conceived impact evaluations. Evaluators can either work with program leadership to improve implementation or decide that a certain organization is not a good fit for an impact evaluation.
Thirty years ago, frugal budgets restricted long-distance calls. The style and values of the organization and the programs being evaluated will determine who participates in the process. The same holds true for fighting poverty: For each reason, we also offer alternatives that fans of impact evaluation can adopt instead.
A clear theory of intended policy change is critical. An unreliable scale produces a different weight every time one steps on it; a reliable one does not. But the promise of research that can help us do more good per dollar spent is enticing.
Does one reasonably believe and ideally have some evidence from elsewhere that the indirect effects are significant enough that ignoring them may radically alter the policy implication of the results?
A clear and detailed theory of change supports organizations in pinpointing the key outputs of each program activity so that they can develop credible measures for them.
Will sending a text message to remind someone to do something influence short-run behavior? External demands for impact undervalue information on implementation because such data often remain unconnected to a theory of change showing how programs create impact.
Similarly, programs may assume that demand exists for their services e. Are our constituents satisfied with the service we provide? To answer these questions, data collection and analysis need to focus more precisely on the question being asked.Social impact bonds and pay-for-success programs seek to fund effective initiatives by tying financing to proven results.
recent counterfactual-based impact evaluations of microcredit programs found much lower impact on household income than was previously claimed by microcredit advocates. An impact evaluation should help determine why. Impact Evaluation of Social Programs: A Policy Perspective John Blomquist Revised Draft, September Comments Welcome The findings, interpretations, and conclusions expressed in this paper are entirely those of the.
Realist Evaluation; Social Return on Investment; Success Case Method; Overviews / introductions to impact evaluation. UNICEF Impact Evaluation Methodological Briefs and Videos: Overview briefs customized to focus on UNICEF’s work and the unique circumstances of conducting impact evaluations of programs and policies in.
Process Evaluation vs. Outcome Evaluation. Jun 14 | By TSNE MissionWorks Why?
Because process evaluation allows a nonprofit to look at how it develops itself, its structures, its supporting programs like communications and marketing, Show various contributors that their social investment is working — in today’s parlance. Three practices in particular can often produce Examples of key evaluation questions for impact evaluation Overall impact • Did it work?
Did [the intervention] produce programs or services to achieve outcomes? • What helped or hindered [the intervention] to. Impact Evaluation of Three Social Programs Essay - You are to identify a social issue and research on at least 3 social programs or efforts that try/tried to address this issue.Download