This volume provides a space for a productive dialogue that, by identifying areas of agreement but also fundamental differences, will promote a more durable working consensus on the circumstances in which some methods are to be preferred over others. The chapter authors and discussants make clear that there are different types of evidence with which to inform this dialogue, including empirical findings of the impact of method choice on evaluation outcomes, the evidence contained in the wisdom of practice, and the results of critical analyses of the broader social impacts of method choice. The editors build on these contributions to suggest pragmatic policies for federal agencies, promoting both context–appropriate method choice and the importance of managing portfolios of evaluative research that maintain desired distributions of methodologies.
This is the 113th volume of the Jossey–Bass quarterly report series New Directions for Evaluation, a publication of Jossey–Bass and the American Evaluation Association.
George Julnes is associate professor of psychology at Utah State University. He has been a proponent of multiple methods for policy evaluation and is directing a random assignment evaluation of disability policy reform that makes extensive use of participant interviews and focus groups.
Debra J. Rog coauthored this chapter while serving as senior research assoociate with the center for Evaluation and Program Improvement and director of its Washington office. She is currently an associate area director with Westat and a vice president of the Rockville Institute.