Impact Evaluation in International Development : : Theory, Methods, and Practice.

• Author: Glewwe, Paul.
• TeilnehmendeR: Todd, Petra.
• Place / Publishing House: Washington, D. C. : : World Bank Publications,, 2021.; ©2020.
• Year of Publication: 2021
• Edition: 1st ed.
• Language: English
• Physical Description: 1 online resource (425 pages)

Table of Contents

Front Cover -- Contents -- Preface -- About the Authors and Contributors -- Abbreviations -- Part I The Basics of Impact Evaluation -- Chapter 1 The Purpose of Impact Evaluation -- Introduction -- The difference between monitoring and evaluation -- A definition of impact evaluation -- Brief overview of types of impact evaluations -- Other issues regarding impact evaluation -- Conclusion -- Notes -- References -- Chapter 2 How to Conduct an Impact Evaluation: Getting Started -- Introduction -- Step 1. Defining the program and the outcomes of interest -- Step 2. Forming a theory of change to refine the evaluation questions -- Step 3. Depicting a theory of change in a results chain (logic model) -- Step 4. Formulating specific hypotheses for the impact evaluation -- Step 5. Selecting performance indicators for monitoring and evaluation -- Conclusion -- Notes -- References -- Chapter 3 The Evaluation Problem -- Introduction -- Correlation does not imply causation -- Potential outcomes and the evaluation problem -- Observed outcomes and the gain from treatment -- Parameters of interest -- Conclusion -- References -- Chapter 4 Validity: Internal, External, and Trade-Offs -- Introduction -- Internal validity -- External validity -- Trade-offs and intermediate approaches -- Conclusion -- Notes -- References -- Chapter 5 Overview of Impact Evaluation Methods -- Introduction -- Using randomized controlled trials to evaluate program impacts -- Impact evaluations based on nonrandomized and quasi-experimental data -- Conclusion -- Notes -- References -- Part II Experimental Methods -- Chapter 6 Introduction to Randomized Controlled Trials -- Introduction -- The basic idea of a randomized controlled trial -- How does randomization solve the evaluation problem? -- What if some people assigned to the treatment group choose not to participate?.

Intention-to-treat effects -- Intention-to-treat effects when effects spill over onto nonparticipants -- Encouragement designs -- Conclusion -- Notes -- References -- Chapter 7 Regression Methods for Randomized Controlled Trials -- Introduction -- Estimating average treatment effects when no problems occur -- Estimation when some in the treatment group are not treated -- Complications caused by sample attrition -- Methods for increasing precision of the estimates -- Methods for obtaining correct standard errors -- Other useful advice and recommendations -- Conclusion -- Notes -- References -- Chapter 8 Practical Advice for Implementing Randomized Evaluations -- Introduction -- Potential problems with randomized experiments, and possible solutions -- Practical advice for randomizing into treatment and control groups -- The use of pre-analysis plans in impact evaluations -- Other practical advice -- Increasing external validity -- Conclusion -- Notes -- References -- Chapter 9 Sample Size, Sample Design, and Statistical Power -- Introduction -- Statistical power as a criterion for choosing the sample design -- Power and MDE calculations in more complex settings -- Practical issues regarding power calculations -- Further statistical issues -- Conclusion -- Notes -- References -- Chapter 10 Recommendations for Conducting Ethical Impact Evaluations -- Introduction -- Two frameworks for conducting ethical evaluations and research -- Confidentiality -- Ethics of randomized controlled trials -- Conflicts of interest -- Ethical research in practice -- Conclusion -- Notes -- References -- Part III Nonexperimental Methods -- Chapter 11 Regression Methods for Nonrandomized Data: Cross-Sectional and Before-After Estimators -- Introduction -- Examples: Cross-sectional, before-after, and difference-in-differences estimators -- Parameters of interest.

The cross-sectional estimator and sources of bias -- The before-after estimator -- Conclusion -- Notes -- References -- Chapter 12 Regression Methods for Nonrandomized Data: The Difference-in-Differences Estimator and the Within Estimator -- Introduction -- The difference-in-differences estimator -- Within estimators -- Applications of difference-in-differences and within estimators -- Conclusion -- Notes -- References -- Chapter 13 Matching Methods -- Introduction -- Two simple examples -- Cross-sectional matching -- Implementation of propensity score matching estimators -- Difference-in-differences matching -- Additional topics for matching methods -- Empirical applications of matching estimators -- Conclusion -- Notes -- References -- Chapter 14 Regression Discontinuity Methods -- Introduction -- Intuition for regression discontinuity methods -- Identification of treatment effects under "sharp" and "fuzzy" data -- Checking the validity of a regression discontinuity design -- The Hahn, Todd, and van der Klaauw estimation method -- Examples of regression discontinuity methods -- Conclusion -- Notes -- References -- Chapter 15 Instrumental Variables Estimation and Local Average Treatment Effects -- Introduction -- Two uses of instrumental variables estimation for impact evaluation analysis -- Instrumental variables estimation of ATE and ATT -- Using IV methods to estimate local average treatment effects -- Conclusion -- Notes -- References -- Chapter 16 Control Function Methods -- Introduction -- The basic idea of the control function approach -- Methods for estimating control functions -- Standard error calculations for control function estimation methods -- Comparing control functions to matching methods and instrumental variables -- Adapting the control function approach for estimating ATE(X) and ATE.

An application: The performance of public and private schools in Chile -- Conclusion -- Notes -- References -- Chapter 17 Quantile Treatment Effects -- Introduction -- The basic idea of quantile regression, with an example -- Conditional and unconditional quantile treatment effect estimators -- Conditional quantile treatment effect estimators -- Unconditional quantile treatment effect estimators -- Standard errors -- Examples of applications -- Conclusion -- Notes -- References -- Part IV Data Collection and Project Management -- Chapter 18 Designing Questionnaires and Other Data Collection Instruments -- Introduction -- General principles and recommendations -- General advice on the design of questionnaires -- Household questionnaires -- Service provider questionnaires -- Community (and price) questionnaires -- Other data collection instruments -- Paper questionnaires versus computer-assisted personal interviewing -- Conclusion -- Notes -- References -- Chapter 19 Data Collection and Data Management -- Introduction -- The steps involved in data collection and data management -- Establish procedures for collecting and managing the data -- Collect the data (including monitoring of data quality) -- Further checks of data quality after the fieldwork -- Create data files for analysis and dissemination -- Establish a system to store, revise, and disseminate the data -- Conclusion -- Notes -- References -- Chapter 20 Survey Management -- Introduction -- Budgeting and developing an overall plan of activities -- Human resources (personnel) management -- Logistical coordination -- Community relations -- Lessons from unfortunate experiences -- Conclusion -- Note -- References -- Part V Related Topics -- Chapter 21 Dissemination of Results and Working with Policy Makers -- Introduction -- What products should the impact evaluation deliver?.

Dissemination of the findings -- Working with policy makers -- Conclusion -- Notes -- References -- Chapter 22 Qualitative Approaches, Data, and Analysis in Impact Evaluations -- Joan DeJaeghere -- Introduction -- Contributions and challenges in using qualitative research in impact evaluations -- Different purposes and types of qualitative approaches -- The most common methods for collecting qualitative data -- Exploratory and explanatory qualitative approaches -- Practical suggestions for designing, gathering, and analyzing qualitative data -- Conclusion -- Annex 22A Questions for realist evaluations -- References -- Chapter 23 Cost-Benefit Analysis and Cost-Effectiveness Analysis -- Introduction -- Calculation of costs -- A simple comparison of cost-benefit analysis and cost-effectiveness analysis -- Cost-benefit analysis (valuing the benefits) -- Cost-effectiveness analysis -- Conclusion -- Notes -- References -- Boxes -- Box 1.1 Key organizations and agency departments that focus on impact evaluation in international development -- Box 3.1 Requirements for answering the evaluation problem, "What is the causal impact of the program (or project or policy) on the outcomes of interest?" -- Box 22.1 Case studies and comparative qualitative analysis: Example of Akazi Kanoze Youth Livelihoods Project (Alcid 2014) -- Box 22.2 Longitudinal and theory-based design and analysis: Example of Learn, Earn, and Save Initiative of Youth Livelihoods Programs in Tanzania and Uganda -- Figures -- Figure 2.1 A results chain diagram: Basic layout and components -- Figure 2.2 A more detailed view of what goes into a results chain (logic model) -- Figure 2.3 Examples of a results chain's components: Education and health sectors -- Figure 2.4 Example of a results chain: Mexico's PROGRESA program.

Figure 2.5 Example of a detailed, successive results chain: Regional vaccination program.