Foreword Section 1: Introduction to Evaluation
Chapter 1
Introduction to Evaluation
Key Points
Decisions, Decisions...
A Brief Overview of Evaluation History
Evaluation: Purpose and Definition
Performance Improvement: A Conceptual Framework
Making Evaluation Happen: Ensuring Stakeholders Buy-in
The Evaluator: A job or a role?
The Relationship to Other Investigative Processes: Evaluation, Needs Assessment, and Scientific Research
When Does Evaluation Occur?
General Evaluation Orientations
Challenges Faced By Evaluators
Ensuring Commitment
Benefits of Evaluation
Basic Definitions
Chapter 2 Principles of Performance-Based Evaluation
Key Points
What are we seeking?
Principle 1: Evaluation-good evaluation-is based on asking the right questions
Evaluation Question Strings
Developing Questions that Guide Valid and Useful Evaluation
Principle 2: Evaluation of process (means) is a function of obtained results (ends)
Getting Ends and Means Identified and Linked
Principle 3: Goals and objectives of organizations should be based on valid needs
Principle 4: Derive valid needs using a top-down approach: rolling down from external impact and value added to the results and methods used within the organization
Principle 5: Every organization should aim for best that society can attain. Evaluation standards must be linked to a useful and real strategic vision
Principle 6: The set of evaluation questions drives the evaluation study
Section 2 Models of Evaluation
Chapter 3 Overview of Existing Evaluation Mode
Key Points
Overview of Classic Evaluation Models
Selected Evaluation Models
Which model should you select?
Conceptualizing a Useful Evaluation That Fits Your Situation
Reflection Questions
Chapter 4 Kirkpatrick's Four Levels of Evaluation
Key Points
Overview of the Model and Relevant Research
Description of Kirkpatrick's Levels
Criticisms of the Model
Strengths and limitations
Application Sample
Additional Application
Reflection Questions
Chapter 5 Phillips Return on Investment
Key Points
Overview of the Model
Description of Phillip's ROI Process
Application Example
Additional case applications
Reflection Questions
Chapter 6 Brinkerhoff Success Case Method
Key Points103
Overview of the Model
Description of the SCM Process
Conclusion
Application Example
Reflection Questions
Chapter 7 The Impact Evaluation Process
Key Points
Overview of the Model
A note on the nature of inquiry of the Impact Evaluation Process
The Impact Evaluation Process: Seven Aligned and Reiterative Steps
Conclusion
Application Example
Reflection Questions
Chapter 8 The CIPP Model
Key Points
Overview of the Model
Description of Stufflebeam's Four Types of Evaluation
Articulating Core Values of Programs and Solutions
Methods Used in CIPP Evaluations
Application Example
Additional case applications
Reflection Questions
Chapter 9 Evaluating Evaluations
Key Points
Can we trust Evaluation Findings?
Definition of Metaevaluation
Evaluation Standards
The American Evaluation Association Principles for Evaluators
Application Example
Reflection Questions
Section 3 Tools and Techniques of Evaluation
Chapter 10 Data
Key Points
Data Defined
Characteristics of Data
Defining Required Data from Performance Objectives
Deriving Measurable Indicators
Finding Data Sources
Chapter 11 Data Collection
Key Points
Introduction to Data Collection Methods
Observation Methodology and the Purpose of Measurement
Problems with Classic Experimental Studies in Applied Settings
Time Series Studies
Document-Centered Methods
Reflection Questions
Chapter 12 Analysis of Evaluation Data: Tools and Techniques
Key points
Methods of Qualitative Analysis
Analysis of Models and Patterns
Analysis using Structured Discussion
Imposing Structure on Emerging Issues
Relevancy
Controls on Structured Discussion
Methods of Quantitative Analysis
Statistics
Two types of statistics
Measures of Central Tendency
Measures of Dispersion (Variability)
Graphical Representations of Data27
Measures of Relationsh
Inferential Statistics: Parametric and Non-Parametric
Interpretation
Reflection Questions
Chapter 13 Communicating the Findings
Key Points
Recommendations
Considerations for Implementing Recommendations
Developing the Report
The Evaluator's Role After the Report
Reflection Questions
Section 4 Continual Improvement
Chapter 14 Common Errors in Evaluation
Errors of System Mapping
Errors of Logic
Errors of procedure
Reflection Questions
Chapter 15 Continual Improvement
Key Points
What is Continual Improvement?
Monitoring Performance
Adjusting Performance
The Role of Leadership
Reflection Questions
Chapter 16 Contracting for Evaluation Services
Key Points
The Nature of Contracts
Fixed Price
Cost Plus Fixed Fee
Time and Materials
Consulting Retaine
Contracting Controls
Contract Elements
Other Contract Clauses
Sample Statement of Work (SOW)
General Information
Contract Award Meeting
General Requirements
Specific Mandatory Tasks and Associated Deliverables
Description of Tasks and Associated Deliverables:
Schedule for Deliverables
Changes to Statement of Work
Reporting Requirements
Travel and Site Visits
Sellalot Corporation Responsibilities
Contractor Experience Requirements
Confidentiality and Nondisclosure
Chapter 17 Intelligence Gathering for Decision Making: Performance Measurement Systems
Key Points
Measures, Intelligence and Decisions
Performance Measurements Systems
Issues in Performance Measurement Systems
Conclusion
Reflection Questions
Chapter 18 The Future of Evaluation in Performance Improvement
Key Points
Introduction
Evaluation and Measurement in Performance Improvement Today
What Does the Future Hold?
Conclusion
References and Related Readings
