The U.S. government has a long and storied experience with producing and using program evaluation. As evaluators, we often like to believe our purpose is clear and necessary. In reality, we know that is not always the case.
The evaluation movement in the Federal government grew out of the War on Poverty initiatives in the 1960s and related efforts to develop prospective analyses for major decisions at the Department of Defense and eventually the Environmental Protection Agency in the 1970s. We have a great many successes where evaluations, and prospective analyses, vastly improved the decisions made by Washington, DC policy-makers.
There are also plenty of examples over the last 50 years when major policy reforms were announced without careful consideration of the evidence. But one question is important in this critique -- did sufficient, credible evidence even exist in a format useful for decision-makers? Far too often the answer is a resounding "no."
Over the last several years, I have frequently been asked to speak to groups about the role of evaluation in informing different aspects of government decision-making. My punchline is often the same: Evaluation can only inform decision-making if it exists.
No evaluation that was promised -- but not delivered -- successfully influenced a policy decision. The challenge in government is developing the capacity to routinely produce evaluations that meet the needs of decision-makers.
The positive influence that evaluation has had on policy in DC occurred in spite of a largely decentralized and uncoordinated evaluation function in government. Not all Federal Departments have active central evaluation offices and there is great heterogeneity in production and use across agencies.
The lack of coordination in Federal evaluation is starting to slowly change. Several years ago, evaluation offices worked with the Office of Management and Budget (OMB) to recognize government could do better in coordinating the evaluation function. Together, they formed the Interagency Council on Evaluation Policy (ICEP). Then, working through ICEP, a group of agencies funded the National Academy of Sciences to convene a workshop to discuss principles and practices for evaluation in the Federal government. [1] Taken together, both actions are positive signs for the growing interest in institutionalizing an evaluation function in the Federal government -- a task longtime Washington Evaluators member Joe Wholey called for nearly 45 years ago. [2]
Why does coordination and the constant presence of the evaluation function matter? Because when evaluation is institutionalized it is also demanded, it is expected, and it happens. Institutionalization creates and maintains champions -- individuals who offer a constant voice to encourage activities and policies be evaluated, ensuring that the evidence does exist to inform decisions. Champions then produce real examples and success stories of the power evaluation can have on improving programs and services.
Ultimately in the long-term, demand for evaluation drives its supply. Evaluation supply relies on a range of factors, from legal authority to resources and expertise. But, perhaps above all, evaluation needs a motivated leader to set the stage. [3]
Leaders and evaluation champions can assume many forms, within the Executive Branch and the Legislative Branch. And that's why this fall Washington Evaluators is happy to co-sponsor EvalAction this fall for members of Washington Evaluators and the American Evaluation Association. EvalAction will provide evaluators the opportunity to engage with congressional offices about the important role of evaluation in informing changes to Federal policies. With any luck, EvalAction may even help identify some new champions for evaluation in Congress.
Hopefully in the next 50 years when asked whether the evidence even existed to inform decisions, we will be able to more frequently offer a resounding and emphatic "yes!"
NICK HART, PH.D. is the President of Washington Evaluators in 2017 and a member of the American Evaluation Association's Evaluation Policy Task Force. The views presented here are those of the author and do not represent the official position of the U.S. Government, including the Office of Management Budget and the Commission on Evidence-Based Policymaking.
References:
[1] National Academy of Sciences. 2017. Workshop on “Principles & Practices for Federal Program Evaluation.” Washington, DC: Committee on National Statistics.
[2] Wholey, J., et al. 1973. Federal Evaluation Policy. Washington DC: The Urban Institute.
[3] Hart, N. 2016. Evaluation at EPA: Determinants of Evaluation Supply at the Environmental Protection Agency. Diss. Washington, DC: George Washington University.
