5 Common Errors to Avoid When Using the PPI >

The Poverty Probability Index (PPI^®) is a simple poverty measurement tool. Even so, using the PPI incorrectly can lead to erroneous conclusions. In this third installment of the PPI Practitioner Guidance Series (read the first installment on setting poverty outreach goals, and the second installment on transitioning to the new 2011 PPP lines), we will talk about common errors that PPI users make, and how you can avoid them.

Error # 1 – Averaging PPI Scores

The poverty rate of a group of households is the share or proportion of households in the group that are poor. Here, a household is defined as poor with reference to a poverty line, if its per day, per person consumption or income, is below that poverty line[1] for that country. Once you have obtained PPI scores for a group of households, you may be inclined to simply average these PPI scores, and report the poverty likelihood corresponding to that PPI score as the group’s poverty rate. This is incorrect.

What should you do instead?

Convert the PPI Score of each household in the group to a poverty likelihood first, and then average these poverty likelihoods to obtain the group’s poverty rate.

Why does averaging PPI scores lead to an incorrect poverty rate?

The relationship between PPI scores and poverty likelihoods is not linear. As a household’s PPI score increases, its poverty likelihood reduces. However, the amount by which the poverty likelihood decreases for every increase of 1 in the PPI score is not the same across all PPI scores. Let us look at the table below to understand this.

Figure 1: 2014 Burkina Faso PPI Look-Up Table (National Poverty Line)

PPI Score Change = +1

Poverty Likelihood Change = (-0.4%)  

PPI Score Change = +1

Poverty Likelihood Change = (-3%)

The table in Figure 1 represents the look-up table for the 2014 Burkina Faso PPI for the National Poverty Line. As seen above, as the PPI Score increases from 4 to 5, the poverty likelihood drops from 96% to 95.6% (i.e. there is a drop of 0.4%). However, when the PPI score increases from 31 to 32, the poverty likelihood changes from 49.9% to 46.9% (i.e. there is a drop of 3%).

The implication of this is that averaging PPI scores will almost never give you the correct poverty rate. Let’s illustrate this with a simple example. Assume that you have two households, household A and household B. You apply the 2014 Burkina Faso PPI to both households. The table below shows their resulting PPI Scores, and the poverty likelihoods corresponding to these scores.

The poverty likelihood corresponding to the average of the two PPI scores is 25.6% - which is much lower than the correct rate of 46.6%.

Error # 2 – Changing PPI Questions and/or Look-up Tables

Why is this an error?

It may be tempting to reduce the time taken to collect PPI data by deleting or changing PPI questions. However, this would lead to incorrect poverty estimates. PPIs are created by a machine learning model that uses established statistical techniques to select ten indicators from a country’s most recent national income and expenditure survey that are found to be strong predictors of poverty within that country. Each PPI score is associated with a unique poverty likelihood value based on the relationship between these indicators and poverty in the national survey. Changing any component of the PPI—questions, response options, points, or poverty likelihood values—will therefore undermine the PPI’s predictive power and invalidate its results.

What should you do instead?

Do not change any component of the PPI—questions, responses, or scores. If you are interested in obtaining a customized PPI based on the specific needs of your project, you can contact us at ppihelpdesk@poverty-action.org. 

Error # 3 – Not Using the Interview Guide to Train Enumerators

Why is this an error?

A country’s PPI comprises 10 questions selected from the national survey which together most accurately predict household poverty in that country. There are many ways to ask about a given subject. However, to make sure that you obtain accurate poverty estimates using the PPI, it is important that your enumerators ask the PPI questions and interpret responses exactly as the national survey enumerators did. To facilitate this, we release a detailed interview guide with each PPI. This guide is extracted from the national survey’s enumerator manual and contains official instructions that were used to train interviewers at that time. Not following these instructions may result in responses that are inconsistent with the corresponding responses in the national survey. Consequently, the relationship between the answers to the ten questions and household poverty could be different from what the PPI model uses to predict poverty, making the poverty estimate less accurate.

What should you do instead?

Train enumerators using only the guidance provided in the PPI interview guide. This guidance is specific to each country and PPI. Do not create additional guidance on your own outside of what you see in this interview guide without first consulting with the PPI team at ppihelpdesk@poverty-action.org.

Error # 4 – Using the PPI to Infer Impact

Why is this an error?

The PPI is intended to provide a snapshot of poverty levels at a point in time. It is a measurement tool, similar to a ruler. A ruler can be used to measure someone’s height at two separate times and show that someone has grown, but the measurements don't indicate what caused the growth. The ruler cannot attribute the increase in height to any particular cause. Likewise, a program aimed at alleviating poverty can measure the change in the poverty levels of its beneficiaries over time using the PPI. However, PPI results by themselves don’t provide any information about causality. Therefore, an organization or program should not be tempted to use the PPI to measure changes in beneficiaries’ poverty rates and then directly attribute any measured reduction in poverty levels to its own efforts, without considering the many other factors that can influence poverty.

What should you do instead?

If you are using poverty estimates provided by the PPI as an outcome indicator for your program, make sure that you are using the tool as part of a well-designed impact evaluation framework. Use data obtained from such a framework to make any claims about impact. See the video here for more on impact evaluations.  

Error # 5 – Reporting PPI Poverty Estimates as Person- or Individual-Level, Instead of Household-Level

Why is this an error?

The PPI is calibrated to household-level poverty likelihoods. Therefore, the PPI gives you PPI scores and poverty likelihoods for households, and not individuals. When you administer the PPI to a group of households, you obtain the household-level poverty rate which is the share or proportion of households in that group that are poor. This is usually not equivalent to the individual-level poverty rate, which is the share or proportion of individuals in that group who are poor.

The below example illustrates why this is so.  

Assume that you apply the PPI to three households and obtain the corresponding household-level poverty likelihoods shown in Column 3. Column 2 shows the number of members in each household.

 Household-level poverty rate = 36.7%

To obtain the poverty rate of this group of households, you would simply average their poverty likelihoods. In this example, the poverty rate of this group of three households is 36.7%.

However, if you want to obtain the poverty rate of the individuals who comprise this group, you would need to take a weighted average of the household-level poverty likelihoods. This is because each household has a different number of household members. Since we are looking at individual-level poverty likelihoods, households with a larger number of members should contribute more towards the average than households with a lower number of household members. Each household’s poverty likelihood must therefore be multiplied by a factor that reflects its relative importance—which is the number of household members in this case.

The individual-level poverty rate of this group is therefore = ((1*40%) + (8*20%) + (2*50%)) / 11 = 27.3%.

The weights here (highlighted in blue), are the number of members in each household.

Note that the household-level poverty rate of this group is 36.7%, while the individual-level poverty rate of this group is 27.3%. When computing the household-level poverty rate of the group, each household contributed equally when averaging the poverty likelihoods. However, when computing the individual-level poverty rate of this group, household 2 with 8 members contributed more towards the average. Household 2 has a lower poverty likelihood than households 1 and 3, and thus lowered the average. The only situation in which household-level and individual-level poverty rates are the same is therefore when all households in the group have the same number of members.

What should you do instead?

When reporting poverty estimates obtained using the PPI, clarify what your unit of measurement is—households or individuals. If you are directly averaging poverty likelihoods obtained using the PPI, ensure that you correctly interpret and report them as household-level poverty estimates. If you would like to report individual-level poverty estimates, compute the weighted average of these household-level poverty likelihoods as explained above.

Additional Resources

We hope that you find these tips useful. Be sure to look at other Learning Materials and Resources on our website. If you have any questions, please feel free to reach out to us at ppihelpdesk@poverty-action.org.