Systematic Evaluation

1 Introduction to Evaluation.- Objectives.- What Is Evaluation?.- Methods.- Standards.- History.- The Pre-Tylerian Period.- The Tylerian Age.- The Age of Innocence.- The Age of Realism.- The Age of Professionalism.- Roles in Evaluation Work.- Knowledge Test for Unit 1.- Application Exercise.- Questions Without Answers.- References.- 2 An Analysis of Alternative Approaches to Evaluation.- Objectives.- Alternative Conceptualizations of Evaluation.- Pseudoevaluation.- Covert Investigations.- Public-relations-Inspired Studies.- Quasievaluations.- Objectives-based Studies.- Experimentally Oriented Studies.- True Evaluations.- Decision-oriented Studies.- Client-centered Studies.- Policy Studies.- Consumer-oriented Studies.- Conclusion.- Knowledge Test for Unit 2.- Application Exercises.- Questions Without Answers.- References.- 3 Objectives-oriented Evaluation: The Tylerian Tradition.- Objectives.- The Intention of the Tylerian Approach.- Procedure for Evaluation Design.- Advantages of the Tylerian Approach.- Uses of the Tylerian Model.- Some Limitations of the Tylerian Approach.- Evaluation as a Terminal Process.- Technical Constraints.- Behaviors as the Ultimate Criterion.- Bias and Narrow Focus of Evaluation Components.- Metfessel and Michael: An Extension of the Tylerian Approach.- The Eight-Step Procedure for Evaluation.- The Appendix of Multiple Criterion Measures.- Knowledge Test for Unit 2.- Application Exercises.- Questions Without Answers.- References.- 4 Edward A. Suchman and the Scientific Approach to Evaluation.- Objectives.- Conceptual Aspects of Evaluation.- Purposes and Principles of Evaluation.- Values and the Evaluation Process.- Assumptions for Evaluation Studies.- Types of Evaluation.- Categories of Evaluation.- Methodological Aspects of Evaluation.- Evaluation versus Nonevaluation Research.- Methodological Approaches to Evaluation.- Variations in Evaluation Research Design.- Principles of Evaluation Research Design.- The Measurement of Effects.- Evaluation and Program Administration.- Evaluation and Administrative Science.- Administration of Evaluation Studies: Some Problem Areas.- Compromises to Meet Administrative Constraints.- Knowledge Test for Unit 4.- Questions Without Answers.- References.- 5 Cronbach's Designing Evaluations: A Synopsis.- Objectives.- to the Issues.- Thoughtful Planning for Flexibility.- The Profitable Evaluation.- Conflicting Approaches to Evaluation.- Consensus in Design Elements.- Balance in Evaluation Design: Summary.- Cronbach's Concepts of the Elements in an Evaluation Design: Uto.- Units, Treatments, and Observing Operations.- UTOS: The Domain of Investigation.- Definition of the Elements of uto and UTO.- *UTOS: The Domain of Application.- Identifying the Research Questions.- The Divergent Phase.- The Convergent Phase.- Planning for Communication.- The Utility of Findings.- Direct Communication.- Extrapolating from Findings.- The Promise of Evaluation.- Flexibility of Design.- Evaluations and Politics: Limitations.- Other Considerations.- Knowledge Test for Unit 5.- Application Exercises.- Questions Without Answers.- References.- 6 Stufflebeam's Improvement-oriented Evaluation.- Objectives.- Some Personal History.- Development of the CIPP Model.- The PDK Study Committee's Elaboration of CIPP Evaluation.- CIPP Compared to Other Recent Evaluation Proposals.- CIPP as a Strategy for Improving Systems.- An Overview of CIPP Categories.- Context Evaluation.- Input Evaluation.- Process Evaluation.- Product Evaluation.- Designing Evaluations.- Metaevaluation and Standards.- Conclusion.- Knowledge Test for Unit 6.- Application Exercises.- Questions Without Answers.- References.- 7 Stake's Client-centered Approach to Evaluation.- Objectives.- The Countenance Statement of Evaluation.- Description.- Judgment.- Format for Data Collection.- Responsive Evaluation.- Responsive versus Preordinate Evaluation.- Substantive Structure of Responsive Evaluation.- General Observations.- Knowledge Test for Unit 7.- Application Exercises.- Questions Without Answers.- References.- 8 T.R. Owens, R.L. Wolf: An Adversary Approach to Evaluation.- Objectives.- Intention of the Adversary Model.- Adversary Proceedings in Their Legal Context.- Applicability to Educational Evaluation.- Practical Implementations.- Dimensions of the Adversary Proceedings.- One Form of the Adversary Approach: Wolf's Judicial Model.- Rationale.- The Four Stages of the Judicial Model.- Pros and Cons of Adversary Evaluation.- The Argument For.- The Argument Against.- Knowledge Test for Unit 8.- Application Exercises.- Questions Without Answers.- References.- 9 Illuminative Evaluation: The Holistic Approach.- Objectives.- Traditional Evaluation: Seeds of Doubt.- Emphasis on Learning Processes for Measurement.- The Agricultural-Botany Paradigm.- MacDonald and the Holistic Approach.- Stake's Concept of Evaluation as Portrayal.- Illuminative Evaluation: A Social-Anthropological Paradigm.- The Context of Educational Programs.- Organization and Methods of Illuminative Evaluation.- Salient Features of Illuminative Evaluation.- Reporting and Decision Making.- Who Receives a Report?.- Communication of the Spirit of the Program.- Diversity Among Reports.- Problems Associated with Illuminative Evaluation.- Stenhouse's Reservations.- Experimental Bias.- Large-scale Studies.- Knowledge Test for Unit 9.- Application Exercises.- Questions Without Answers.- References.- 10 Michael Scrlven's Consumer-Oriented Approach to Evaluation.- Objectives.- Evaluation Defined.- Critique of Other Persuasions.- Formative and Summative Evaluation.- Amateur versus Professional Evaluation.- Intrinsic and Payoff Evaluation.- Goal-free Evaluation.- Needs Assessment.- The Key Evaluation Checklist.- Metaevaluation.- Evaluation Ideologies.- The Separatist Ideology.- The Positivist Ideology.- The Managerial Ideology.- The Relativist Ideology.- Professionalization of Evaluation.- Knowledge Test for Unit 10.- Application Exercises.- Questions Without Answers.- References.- Indexes.