| Formatted contents note |
1 From Firefighting to Systematic Action<br/>Adding "Surprise" to the Mix 1<br/>Historical Roots: Evaluation, Planning, and System Behavior 2<br/>From Explaining Surprise to Dealing with It 4<br/>Development Path of This Book 6<br/>Guiding Principles 8<br/>How to Read This Book 12<br/>In Sum 14<br/>2 Structure of the Unexpected<br/>Where Does Surprise Come From? 17<br/>Beyond Simple Distinctions 21<br/>In Sum 26<br/>3 Placing Surprise in the Evaluation Landscape<br/>When Is the Probability of Surprise High? 27<br/>When Is Surprise Disruptive to Evaluation? 35<br/>In Sum 44<br/>4 Minimizing Foreseeable Surprise<br/>Theory: Using Explanatory Power and Simplified Relationships 48<br/>Exploiting Past Experience: Capitalizing on What We Already Know 58<br/>Limiting Time Frames to Minimize the Opportunity for Surprise 63<br/>In Sum 67<br/>5 Shifting from Advance Planning to Early Detection<br/>Leading Indicators 72<br/>System-Based Logic Modeling 78<br/>In Sum 81<br/>6 Agile Evaluation<br/>Data 83<br/>Agile Methodology 87<br/>Retooling Program Theory 95<br/>Agility and Stakeholder Needs 98<br/>In Sum 98<br/>7 How Much Is Too Much?: Appreciating Trade-Offs and Managing<br/>the Balance<br/>A Framework for Appreciating Design Trade-Offs 103<br/>Maximizing Choice, Minimizing Risk 107<br/>Evaluation Design 112<br/>In Sum 113<br/>8 Applying the Examples to Categories of Cases: The Life Cycle View<br/>"Unintended Consequences": Unity across Programs<br/>and Their Evaluations 116<br/>Interpreting Cases through a Life Cycle Perspective 117<br/>In Sum 123<br/>9 Applying the Examples to Categories of Cases:<br/>The Social/Organizational View<br/>Navigating through the Cases 125<br/>Placement of Cases on the Social/Organizational Map 131<br/>Categorizations Derived from the Data 147<br/>In Sum 154<br/>10 Lessons from Individual Cases: Tactics for Anticipating Surprise<br/>In Sum 168<br/>11 Lessons from Individual Cases: Responding to Surprise<br/>The Middle 169<br/>Leading Indicators and Agile Evaluation 172<br/>In Sum 176<br/>12 Unanticipated Program Outcomes<br/>Case Descriptions 177<br/>Applying the Cases to Unintended Program Outcomes 181<br/>Comparing the Cases 183<br/>Predicting the Need for Agile Evaluation 187<br/>In Sum 191<br/>13 Concluding Thoughts<br/>Case 1 Grasping at Straws and Discovering a Different<br/>Program Tfieory: An Exercise in Reengineering<br/>Analysis Logic in a Cfiild Care Evaluation Setting<br/>Dennis P. Afjholter<br/>Case 2 Sfiifting Sands in a Training Evaluation Context<br/>James W Altschuld and Phyllis M. Thomas<br/>Case 3 Evaluating Programs Aimed at Promoting Cfiild Well-Being:<br/>Tfie Case of Local Social Welfare Agencies in Jerusalem<br/>Anat Zeira<br/>Case 4 Assessing tfie Impact of Providing Laptop<br/>Computers to Students<br/>J. Dan Strahl, Deborah L. Lowther, and Steven M. Ross<br/>Case 5 Quasi-Experimental Strategies Wfien Randomization Fails:<br/>Propensity Score Matching and Sensitivity Analysis<br/>in Whole-School Reform<br/>Gary L. Bowen, Roderick A. Rose, and Shenyang Quo<br/>Case 6 Unexpected Changes in Program Delivery:<br/>The Perils of Overlooking Process Data When<br/>Evaluating HIV Prevention<br/>Bryce D. Smith, Danny Sprouse, and Kevin L DeWeaver<br/>Case 7 Evaluating Costs and Benefits of Consumer-Operated Services:<br/>Unexpected Resistance, Unanticipated Insights,<br/>and Deja Vu All Over Again<br/>Brian T. Yates<br/>Case 8 Keep Up with the Program!: Adapting the Evaluation Focus<br/>to Align with a College Access Program's Changing Goals<br/>Kristine L. Chadwick and Jennifer Conner Blatz<br/>Case 9 Assumptions about School Staff's Competencies<br/>and Likely Program Impacts<br/>Laura Hassler Lang, Christine E. Johnson, and Shana Goldwyn<br/>Case 10 Mixed Method Evaluation of a Support Project<br/>for Nonprofit Organizations<br/>Riki Savaya and Mark Waysman<br/>Case 11 Evaluating the Health Impacts of Central Heating<br/>Jeremy Walker, Richard Mitchell, Stephen Platt,<br/>and Mark Petticrew<br/>Case 12 Recruiting Target Audience: When All Else Falls, Use the<br/>Indirect Approach for Evaluating Substance Abuse Prevention<br/>Molly Engle<br/>Case 13 Unintended Consequences of Changing Funder Requirements<br/>Midproject on Outcome Evaluation Design and Results<br/>in HIV Outreach Services<br/>Lena Lundgren, Therese Fitzgerald, and Deborah Chassler<br/>Case 14 Generating and Using Evaluation Feedback for Providing<br/>Countywide Family Support Services<br/>Deborah L. Wasserman<br/>Case 15 Trauma and Posttraumatic Stress Disorder among Female<br/>Clients in Methadone Maintenance Treatment in Israel:<br/>From Simple Assessment to Complex Intervention<br/>Miriam Schiff and Shabtay Levit<br/>Case 16 From Unintended to Undesirable Effects of Health Intervention:<br/>The Case of User Fees Abolition in Niger, West Africa<br/>Valery Ridde and Aissa Diarra<br/>Case 17 Unintended Consequences and Adapting Evaluation:<br/>Katrina Aid Today National Case Management Consortium<br/>Amanda Janis and Kelly M. Stiefel<br/>Case 18 Evaluation of the Integrated Services Pilot Program<br/>from Western Australia<br/>Susanne Therese Bahn, Peter Hancock, and Trudi Cooper1 From Firefighting to Systematic Action<br/>Adding "Surprise" to the Mix 1<br/>Historical Roots: Evaluation, Planning, and System Behavior 2<br/>From Explaining Surprise to Dealing with It 4<br/>Development Path of This Book 6<br/>Guiding Principles 8<br/>How to Read This Book 12<br/>In Sum 14<br/>2 Structure of the Unexpected<br/>Where Does Surprise Come From? 17<br/>Beyond Simple Distinctions 21<br/>In Sum 26<br/>3 Placing Surprise in the Evaluation Landscape<br/>When Is the Probability of Surprise High? 27<br/>When Is Surprise Disruptive to Evaluation? 35<br/>In Sum 44<br/>4 Minimizing Foreseeable Surprise<br/>Theory: Using Explanatory Power and Simplified Relationships 48<br/>Exploiting Past Experience: Capitalizing on What We Already Know 58<br/>Limiting Time Frames to Minimize the Opportunity for Surprise 63<br/>In Sum 67<br/>5 Shifting from Advance Planning to Early Detection<br/>Leading Indicators 72<br/>System-Based Logic Modeling 78<br/>In Sum 81<br/>6 Agile Evaluation<br/>Data 83<br/>Agile Methodology 87<br/>Retooling Program Theory 95<br/>Agility and Stakeholder Needs 98<br/>In Sum 98<br/>7 How Much Is Too Much?: Appreciating Trade-Offs and Managing<br/>the Balance<br/>A Framework for Appreciating Design Trade-Offs 103<br/>Maximizing Choice, Minimizing Risk 107<br/>Evaluation Design 112<br/>In Sum 113<br/>8 Applying the Examples to Categories of Cases: The Life Cycle View<br/>"Unintended Consequences": Unity across Programs<br/>and Their Evaluations 116<br/>Interpreting Cases through a Life Cycle Perspective 117<br/>In Sum 123<br/>9 Applying the Examples to Categories of Cases:<br/>The Social/Organizational View<br/>Navigating through the Cases 125<br/>Placement of Cases on the Social/Organizational Map 131<br/>Categorizations Derived from the Data 147<br/>In Sum 154<br/>10 Lessons from Individual Cases: Tactics for Anticipating Surprise<br/>In Sum 168<br/>11 Lessons from Individual Cases: Responding to Surprise<br/>The Middle 169<br/>Leading Indicators and Agile Evaluation 172<br/>In Sum 176<br/>12 Unanticipated Program Outcomes<br/>Case Descriptions 177<br/>Applying the Cases to Unintended Program Outcomes 181<br/>Comparing the Cases 183<br/>Predicting the Need for Agile Evaluation 187<br/>In Sum 191<br/>13 Concluding Thoughts<br/>Case 1 Grasping at Straws and Discovering a Different<br/>Program Tfieory: An Exercise in Reengineering<br/>Analysis Logic in a Cfiild Care Evaluation Setting<br/>Dennis P. Afjholter<br/>Case 2 Sfiifting Sands in a Training Evaluation Context<br/>James W Altschuld and Phyllis M. Thomas<br/>Case 3 Evaluating Programs Aimed at Promoting Cfiild Well-Being:<br/>Tfie Case of Local Social Welfare Agencies in Jerusalem<br/>Anat Zeira<br/>Case 4 Assessing tfie Impact of Providing Laptop<br/>Computers to Students<br/>J. Dan Strahl, Deborah L. Lowther, and Steven M. Ross<br/>Case 5 Quasi-Experimental Strategies Wfien Randomization Fails:<br/>Propensity Score Matching and Sensitivity Analysis<br/>in Whole-School Reform<br/>Gary L. Bowen, Roderick A. Rose, and Shenyang Quo<br/>Case 6 Unexpected Changes in Program Delivery:<br/>The Perils of Overlooking Process Data When<br/>Evaluating HIV Prevention<br/>Bryce D. Smith, Danny Sprouse, and Kevin L DeWeaver<br/>Case 7 Evaluating Costs and Benefits of Consumer-Operated Services:<br/>Unexpected Resistance, Unanticipated Insights,<br/>and Deja Vu All Over Again<br/>Brian T. Yates<br/>Case 8 Keep Up with the Program!: Adapting the Evaluation Focus<br/>to Align with a College Access Program's Changing Goals<br/>Kristine L. Chadwick and Jennifer Conner Blatz<br/>Case 9 Assumptions about School Staff's Competencies<br/>and Likely Program Impacts<br/>Laura Hassler Lang, Christine E. Johnson, and Shana Goldwyn<br/>Case 10 Mixed Method Evaluation of a Support Project<br/>for Nonprofit Organizations<br/>Riki Savaya and Mark Waysman<br/>Case 11 Evaluating the Health Impacts of Central Heating<br/>Jeremy Walker, Richard Mitchell, Stephen Platt,<br/>and Mark Petticrew<br/>Case 12 Recruiting Target Audience: When All Else Falls, Use the<br/>Indirect Approach for Evaluating Substance Abuse Prevention<br/>Molly Engle<br/>Case 13 Unintended Consequences of Changing Funder Requirements<br/>Midproject on Outcome Evaluation Design and Results<br/>in HIV Outreach Services<br/>Lena Lundgren, Therese Fitzgerald, and Deborah Chassler<br/>Case 14 Generating and Using Evaluation Feedback for Providing<br/>Countywide Family Support Services<br/>Deborah L. Wasserman<br/>Case 15 Trauma and Posttraumatic Stress Disorder among Female<br/>Clients in Methadone Maintenance Treatment in Israel:<br/>From Simple Assessment to Complex Intervention<br/>Miriam Schiff and Shabtay Levit<br/>Case 16 From Unintended to Undesirable Effects of Health Intervention:<br/>The Case of User Fees Abolition in Niger, West Africa<br/>Valery Ridde and Aissa Diarra<br/>Case 17 Unintended Consequences and Adapting Evaluation:<br/>Katrina Aid Today National Case Management Consortium<br/>Amanda Janis and Kelly M. Stiefel<br/>Case 18 Evaluation of the Integrated Services Pilot Program<br/>from Western Australia<br/>Susanne Therese Bahn, Peter Hancock, and Trudi Cooper |
