Policy Impact Evaluation Research Initiative (PIERI)

Experimental research

PEP support program for experimental research

Research supported under the PEP-PIERI program uses experimental approaches to conduct rigorous impact evaluations of social policies/programs on targeted populations in developing countries. 

Results from PEP experimental impact evaluation projects provide scientifically-sound empirical evidence to inform program managers, donors, policymakers and civil society on program effectiveness, budget use and ways to improve the design of future interventions.

This evidence is used to:  

  • Inform decisionmakers, donors and taxpayers on the realization of expected benefits
  • Help improve and assist in program design and implementation
  • Foster accountability of implementation processes
  • Generate political support for continuation or expansion of programs, both within and beyond national boundaries (public good value)

In addition, through the global infrastructure of PEP, the PIERI program also encourages comparative analyses between researchers from around the world, and especially in developing countries of Africa, Asia and Latin America, to compare methods, tools, findings and experience of impact evaluation projects.

Browse through the left-side menu to find the lists of PEP-supported projects, publications and researchers associated with this type of research.

 

Methods: Randomized Controlled Trials and Field Experiments

The key characteristic of both approaches is the experimental framework, conducted in a real-world environment. Central to this is the notion of random assignment between treatment groups (who receive the intervention) and control groups (who do not). Random assignment ensures that the two groups are comparable and thus all differences noted after the intervention can be confidently ascribed to the intervention. This method is considered the gold standard for policy evaluation, because it produces unbiased results.  

The experimental framework requires careful design of the intervention and collection of baseline data on both groups well before the intervention takes place. This framework is typically used to test new interventions in a pilot phase, to explore their impacts and to compare alternative intervention designs (i.e. by distinguishing several different treatment groups). Results can then be used to determine if and how to eventually scale up the new intervention.
 

Randomized controlled trials (RCTs)

RCTs use random assignment to allocate the program or intervention being analyzed. They are generally considered the gold standard for policy evaluation, producing unbiased results. To quantify the impact of the intervention, the outcomes of the units receiving the program are compared against those who did not receive it.

RCTs are generally large-scale endeavors with the aim of evaluating the effectiveness of policy packages. These projects are often expensive, multi-year, and are carried out by governments or multilateral organizations wanting to test whether a program comprised of several interventions achieves the outcome it was set to achieve or not. 

  • For example, Conditional Cash Transfer (CCT) programs targeted at vulnerable populations combine conditionalities, income support and some sort of health assistance with the aim of increasing schooling. An RCT designed to evaluate the effectiveness of CCTs would be implemented in the form of a pilot, where program participation is assigned randomly, and where results refer to the effect of the whole package of interventions on schooling. 
     

Field experiments (FEs) 

FEs also use random assignment to allocate treatment as well as controlled environments that make it possible to clearly identify the effect of a treatment and/or learn about micro-motives for behavior. 

While sharing a common methodology, field experiments are generally smaller scale endeavors than their RCT counterparts. 

FEs may also be set up to evaluate a policy, but they are more frequently devised to learn about the effectiveness of a given mechanism to implement a program or to test economic hypothesis. Thus, they are an important tool to design better policies. Due to their focused nature, FEs can be implemented in shorter periods of time and generally require less funding than RCTs. 

  • In the case of a government trying to make decisions about a Conditional Cash Transfer program, an FE could be carried out to find out whether the income support portion of the program has a higher impact on schooling when paid to mothers or fathers of children, for example. 
     

The PEP-PIERI group encourages researchers to think of creative ways to adapt well-known experimental protocols to answer research questions relevant to local barriers faced by subjects in developing countries.

Both types of projects (RCTs and field experiments) require a good institutional knowledge of the program under study, in terms of:
  • Eligibility rules and the target population
  • Objectives of the intervention and its potential unintended effects
  • Outcomes that may potentially be affected by the program
  • The intervention calendar and the timing of effects (short-term, medium-term, long-term)
     

Scientific Support

Team

The PIERI program and related research/support activities are managed by the following team:

With support from a group of PEP resource persons, who are leading international experts in the field of impact evaluation techniques.
 

Recommended readings and training material

In addition to peer-review, mentoring and advisory during project implementation, PEP provides supported research teams with:

Partners

  •  
  •  

Funded by

  •  
  •  
  •