Impact Evaluation in International Development : : Theory, Methods, and Practice.

• Author: Glewwe, Paul.
• TeilnehmendeR: Todd, Petra.
• Place / Publishing House: Washington, D. C. : : World Bank Publications,, 2021.; ©2020.
• Year of Publication: 2021
• Edition: 1st ed.
• Language: English
• Physical Description: 1 online resource (425 pages)

Table of Contents

• Front Cover
• Contents
• Preface
• About the Authors and Contributors
• Abbreviations
• Part I The Basics of Impact Evaluation
• Chapter 1 The Purpose of Impact Evaluation
• Introduction
• The difference between monitoring and evaluation
• A definition of impact evaluation
• Brief overview of types of impact evaluations
• Other issues regarding impact evaluation
• Conclusion
• Notes
• References
• Chapter 2 How to Conduct an Impact Evaluation: Getting Started
• Introduction
• Step 1. Defining the program and the outcomes of interest
• Step 2. Forming a theory of change to refine the evaluation questions
• Step 3. Depicting a theory of change in a results chain (logic model)
• Step 4. Formulating specific hypotheses for the impact evaluation
• Step 5. Selecting performance indicators for monitoring and evaluation
• Conclusion
• Notes
• References
• Chapter 3 The Evaluation Problem
• Introduction
• Correlation does not imply causation
• Potential outcomes and the evaluation problem
• Observed outcomes and the gain from treatment
• Parameters of interest
• Conclusion
• References
• Chapter 4 Validity: Internal, External, and Trade-Offs
• Introduction
• Internal validity
• External validity
• Trade-offs and intermediate approaches
• Conclusion
• Notes
• References
• Chapter 5 Overview of Impact Evaluation Methods
• Introduction
• Using randomized controlled trials to evaluate program impacts
• Impact evaluations based on nonrandomized and quasi-experimental data
• Conclusion
• Notes
• References
• Part II Experimental Methods
• Chapter 6 Introduction to Randomized Controlled Trials
• Introduction
• The basic idea of a randomized controlled trial
• How does randomization solve the evaluation problem?
• What if some people assigned to the treatment group choose not to participate?.
• Intention-to-treat effects
• Intention-to-treat effects when effects spill over onto nonparticipants
• Encouragement designs
• Conclusion
• Notes
• References
• Chapter 7 Regression Methods for Randomized Controlled Trials
• Introduction
• Estimating average treatment effects when no problems occur
• Estimation when some in the treatment group are not treated
• Complications caused by sample attrition
• Methods for increasing precision of the estimates
• Methods for obtaining correct standard errors
• Other useful advice and recommendations
• Conclusion
• Notes
• References
• Chapter 8 Practical Advice for Implementing Randomized Evaluations
• Introduction
• Potential problems with randomized experiments, and possible solutions
• Practical advice for randomizing into treatment and control groups
• The use of pre-analysis plans in impact evaluations
• Other practical advice
• Increasing external validity
• Conclusion
• Notes
• References
• Chapter 9 Sample Size, Sample Design, and Statistical Power
• Introduction
• Statistical power as a criterion for choosing the sample design
• Power and MDE calculations in more complex settings
• Practical issues regarding power calculations
• Further statistical issues
• Conclusion
• Notes
• References
• Chapter 10 Recommendations for Conducting Ethical Impact Evaluations
• Introduction
• Two frameworks for conducting ethical evaluations and research
• Confidentiality
• Ethics of randomized controlled trials
• Conflicts of interest
• Ethical research in practice
• Conclusion
• Notes
• References
• Part III Nonexperimental Methods
• Chapter 11 Regression Methods for Nonrandomized Data: Cross-Sectional and Before-After Estimators
• Introduction
• Examples: Cross-sectional, before-after, and difference-in-differences estimators
• Parameters of interest.
• The cross-sectional estimator and sources of bias
• The before-after estimator
• Conclusion
• Notes
• References
• Chapter 12 Regression Methods for Nonrandomized Data: The Difference-in-Differences Estimator and the Within Estimator
• Introduction
• The difference-in-differences estimator
• Within estimators
• Applications of difference-in-differences and within estimators
• Conclusion
• Notes
• References
• Chapter 13 Matching Methods
• Introduction
• Two simple examples
• Cross-sectional matching
• Implementation of propensity score matching estimators
• Difference-in-differences matching
• Additional topics for matching methods
• Empirical applications of matching estimators
• Conclusion
• Notes
• References
• Chapter 14 Regression Discontinuity Methods
• Introduction
• Intuition for regression discontinuity methods
• Identification of treatment effects under "sharp" and "fuzzy" data
• Checking the validity of a regression discontinuity design
• The Hahn, Todd, and van der Klaauw estimation method
• Examples of regression discontinuity methods
• Conclusion
• Notes
• References
• Chapter 15 Instrumental Variables Estimation and Local Average Treatment Effects
• Introduction
• Two uses of instrumental variables estimation for impact evaluation analysis
• Instrumental variables estimation of ATE and ATT
• Using IV methods to estimate local average treatment effects
• Conclusion
• Notes
• References
• Chapter 16 Control Function Methods
• Introduction
• The basic idea of the control function approach
• Methods for estimating control functions
• Standard error calculations for control function estimation methods
• Comparing control functions to matching methods and instrumental variables
• Adapting the control function approach for estimating ATE(X) and ATE.
• An application: The performance of public and private schools in Chile
• Conclusion
• Notes
• References
• Chapter 17 Quantile Treatment Effects
• Introduction
• The basic idea of quantile regression, with an example
• Conditional and unconditional quantile treatment effect estimators
• Conditional quantile treatment effect estimators
• Unconditional quantile treatment effect estimators
• Standard errors
• Examples of applications
• Conclusion
• Notes
• References
• Part IV Data Collection and Project Management
• Chapter 18 Designing Questionnaires and Other Data Collection Instruments
• Introduction
• General principles and recommendations
• General advice on the design of questionnaires
• Household questionnaires
• Service provider questionnaires
• Community (and price) questionnaires
• Other data collection instruments
• Paper questionnaires versus computer-assisted personal interviewing
• Conclusion
• Notes
• References
• Chapter 19 Data Collection and Data Management
• Introduction
• The steps involved in data collection and data management
• Establish procedures for collecting and managing the data
• Collect the data (including monitoring of data quality)
• Further checks of data quality after the fieldwork
• Create data files for analysis and dissemination
• Establish a system to store, revise, and disseminate the data
• Conclusion
• Notes
• References
• Chapter 20 Survey Management
• Introduction
• Budgeting and developing an overall plan of activities
• Human resources (personnel) management
• Logistical coordination
• Community relations
• Lessons from unfortunate experiences
• Conclusion
• Note
• References
• Part V Related Topics
• Chapter 21 Dissemination of Results and Working with Policy Makers
• Introduction
• What products should the impact evaluation deliver?.
• Dissemination of the findings
• Working with policy makers
• Conclusion
• Notes
• References
• Chapter 22 Qualitative Approaches, Data, and Analysis in Impact Evaluations
• Joan DeJaeghere
• Introduction
• Contributions and challenges in using qualitative research in impact evaluations
• Different purposes and types of qualitative approaches
• The most common methods for collecting qualitative data
• Exploratory and explanatory qualitative approaches
• Practical suggestions for designing, gathering, and analyzing qualitative data
• Conclusion
• Annex 22A Questions for realist evaluations
• References
• Chapter 23 Cost-Benefit Analysis and Cost-Effectiveness Analysis
• Introduction
• Calculation of costs
• A simple comparison of cost-benefit analysis and cost-effectiveness analysis
• Cost-benefit analysis (valuing the benefits)
• Cost-effectiveness analysis
• Conclusion
• Notes
• References
• Boxes
• Box 1.1 Key organizations and agency departments that focus on impact evaluation in international development
• Box 3.1 Requirements for answering the evaluation problem, "What is the causal impact of the program (or project or policy) on the outcomes of interest?"
• Box 22.1 Case studies and comparative qualitative analysis: Example of Akazi Kanoze Youth Livelihoods Project (Alcid 2014)
• Box 22.2 Longitudinal and theory-based design and analysis: Example of Learn, Earn, and Save Initiative of Youth Livelihoods Programs in Tanzania and Uganda
• Figures
• Figure 2.1 A results chain diagram: Basic layout and components
• Figure 2.2 A more detailed view of what goes into a results chain (logic model)
• Figure 2.3 Examples of a results chain's components: Education and health sectors
• Figure 2.4 Example of a results chain: Mexico's PROGRESA program.
• Figure 2.5 Example of a detailed, successive results chain: Regional vaccination program.