Alternative PPI Data Collection Methodologies: A Review >

The PPI survey is designed to be administered in the client’s home by a trained enumerator for two primary reasons: the enumerator is there to help explain certain key terms and concepts, and the enumerator can visually verify the answers to some questions, such as the materials used in the construction of the roof or walls of the house.

As of today, the PPI has been used by over 200 organizations around the world. Most are microfinance institutions, but a growing number are other types of businesses or organizations. Unlike the loan officers of microfinance institutions who generally visit their clients’ homes at least once per year, staff members of other types of organizations may not have the reason or means to visit their clients’ homes. For example, it may not be economically viable for staff of an agricultural cooperative to visit the tens of thousands of farmers who live in remote, rural areas. As more and more organizations in sectors outside of microfinance begin to use the PPI to assess their social performance, we are often asked if PPI results will be accurate if a different data collection methodology is used.

Alternative Methodologies – Initial Testing

With this in mind, Grameen Foundation, with support from Ford Foundation, set out to test alternatives. For our initial tests we partnered with Juhudi Kilimo in Kenya, using mobile phones to administer the PPI via SMS, and COPEME in Peru, where we administered the PPI at a coffee collection center as members of an agricultural cooperative dropped off their harvest. In both cases the PPI survey was given to a set of clients via the alternative method; the same set of clients were then visited at a later date and given the PPI survey again, this time in the standard way, by a trained enumerator in their home.
The study answered a number of questions, but primarily:

1. How accurate are the alternative methodologies, as compared to the standard method of surveying the client in their home?
2. What are the inherent challenges of each alternative?

Key Findings

Kenya:

• Discrepancy rate: 30% of PPI questions were answered differently between SMS & in-home collection (note: this had little effect on poverty likelihoods because the majority of the sample had a high likelihood of being above the poverty line.)
• Most commonly identified reasons for discrepancy were the absence of an enumerator to define terms in the scorecard and a relatively low response rate.

Peru: 

• Discrepancy rate: 4.2%
• Discrepancy was minimal; the presence of trained enumerators administering the survey at the collection center proved crucial.

While both alternatives exhibited positive and negative characteristics, it was clear that in these cases, the presence of a trained enumerator proved crucial in ensuring respondents understood key terms and concepts, and were able to answer accurately.

For more information on these studies and their findings, see this short report.

Guide for Organizations

In addition to the above, we produced a short guide (PDF, English, log-in required) on how to test an alternative data collection methodology for managers who want to survey clients using the PPI in a location that is not the client’s home or through a channel that does not involve an enumerator. The guide is not meant to provide all the answers regarding the accuracy or effectiveness of the various methodologies, but rather introduce the alternatives, and get organizations thinking about the key questions related to each.

Looking to the Future

There is no doubt that as the PPI expands to more countries and is used by a more diverse set of organizations it will be crucial to make the tool as user-friendly and flexible as possible, while still maintaining a high degree of accuracy.  Regardless of challenges, the alternative methodologies tested so far have shown great potential in terms of moving us toward this goal. To this end, Grameen Foundation plans on conducting larger scale testing of both (potentially as well as others) in the coming year, in conjunction with Ford and Mark Schreiner, the developer of the PPI. Having learned a number of lessons from our initial testing, these further tests will aim to fully interrogate key points, and begin to form a clearer picture of the viability of alternative collection options.

If you have any further questions on this topic, the linked reports above, or our future research, please contact the PPI team at ppihelpdesk@poverty-action.org.