PRB

(September 2010) In April 2010, the Institute for Health Metrics and Evaluation (IMHE), a research center at the University of Washington, released estimates showing unexpected declines in global maternal mortality compared with previous UN estimates. The release paralleled IMHE’s other estimates that showed faster-than-expected declines in under-5 mortality worldwide. Ensuing coverage in The Lancet and in major media focused on debate within the global health community around the timing of the estimates’ release and its potential negative impact on funding and support of health programs in developing countries. However, despite their variation, the differing maternal and under-5 mortality estimates all show that data quality continues to be weak in many developing countries; that global estimates mask varying or stalling progress at the country level; and that overall progress in reducing maternal and under-5 mortality is still insufficient to meet the Millennium Development Goals (MDGs).

New data and estimates are driving global discussion on program and policy responses toward the two interrelated MDGs of reducing maternal and under-5 mortality. Data determine program funding and policy attention, illustrate success or setbacks, and persuade and move people to act. Knowing how reliable data are and understanding how estimates are determined is necessary for policymakers and health practitioners to plan the most effective programs. Global health organizations are focusing more on monitoring and evaluation, and donors are increasingly scrutinizing how aid is used and programs’ effectiveness. In September 2010, the UN MDG summit will address the need for continued funding at the country level despite global success in recent years, calling for greater accountability and tracking impact.

Global Decline in Maternal and Under-5 Mortality

For decades, maternal mortality was estimated at around 500,000 annual deaths worldwide. Stalled progress in significantly lowering that number has concerned the global health community. In 2008, UNICEF estimated maternal mortality at 536,000 deaths a year: “Maternal deaths have remained stubbornly intractable. Limited gains have been made…to reduce the 1990 mortality ratio by three-quarters by 2015.”¹ In 2009, the World Health Organization (WHO) reported that maternal mortality was still stuck at 1990 levels.² However, the new estimates from the IMHE put the annual figure at 343,000 deaths as of 2008, an approximately 30 percent decrease from other global estimates. IMHE stated that “our research suggests that global progress toward these goals (MDG 4 and 5) has been steadier and more significant than previous studies have indicated.”³ However, the actual level of decline is not really so different. In 2005, the UN estimated a 0.5 percent decline in maternal mortality a year whereas IMHE puts it at 1.3 percent, but both are not enough to reach the MDG target, which requires an annual decline of 5.5 percent.⁴ In June 2010, reflecting the still-evolving estimates, the UN kept its maternal mortality estimate vague, citing that “hundreds of thousands of women” die annually as a result from pregnancy or childbirth. But new estimates are expected later this year that will align more closely with IMHE’s numbers.⁵

Under-5 mortality has decreased worldwide since 1990. According to the UN, under-5 mortality dropped from 12.6 million annual deaths in 1990 to 8.8 million in 2008. IMHE has lower estimates—from 11.9 million deaths in 1990 to 7.7 million in 2010, a faster decline than expected. The IMHE estimates show that the drop in neonatal deaths was especially strong at 2.1 percent annually between 1990 and 2008, whereas other estimates had shown them to be at a standstill. According to Robert Black, professor and chair of the Department of International Health and director of the Institute for International Programs at Johns Hopkins University, differing estimates point to the same conclusion—there are too many preventable child deaths: “The alternative methods [of estimates] matter little in the conclusions regarding the excessively high levels of mortality in some world regions and the ‘debate’ is a distraction from the real need to accelerate interventions to reduce child mortality.”

Changes in Methodology and Data Quality Challenges

The difference between IMHE and UN estimates stems from the limited amount of high-quality data in developing countries. IMHE and the UN use the same data; however, the variation derives from using different statistical models and the weight given to certain data sources to draw trends. Generally, estimates are made from a set of data from a narrow range, such as small local studies, and extrapolated out to the national level. In other words, the interpretation of data leads to the estimate, rather than the data themselves. For example, according to Ties Boerma, director of Department of Measurement and Health Information Systems at the WHO, when estimating child mortality in Nigeria, the 1999 Demographic and Health Survey (DHS) was given less weight in models than other sources because the survey model was determined to be weak. The UN typically evaluates surveys on a country-by-country basis, but IMHE gave the 1999 DHS the same weight as other data sources in its estimation model. “The weakness in that is there’s a level of arbitrary judgment and so IMHE feels you shouldn’t do that,” according to Boerma.

In fact, although there are more data available than ever before, birth and death registration data have not improved over the past 30 years except in a handful of countries. Developing countries tend to have weak reporting systems and poor infrastructure, hindering data collection and quality. Unlike in developing countries, more developed countries have vital registration systems that track all births and deaths, including causes of death, leading to fairly accurate data on mortality.

In recent years, there has been a shift from modeling estimates solely based on data from health clinics, which can be can be misleading or incomplete. Estimates now include data from population-based surveys, where households are interviewed about the health status of family members. For example, in measuring under-5 mortality, IHMC used “summary birth surveys” in DHS data from 70 countries along with census data. Mothers were asked how many births they had and how many children had survived. To determine maternal mortality rates, women are asked whether their sisters have died during pregnancy or within six weeks of childbirth.

However, difficulties in measurement remain. Maternal and child deaths can be misclassified or underreported in developing countries, since many deaths happen at home and in rural areas, outside the official health systems. Surveys also leave out marginalized populations such as the homeless. Maternal deaths are especially difficult to measure. Boerma says, “The problem with maternal mortality is that it’s a relatively rare event and the data are collected in household surveys, and they need to be very large…And what you get then is an estimate that is retrospective—it applies to, say the six-year period before the survey with quite a lot of uncertainty around it. What you really want is for every maternal death to be counted when it happens like it is done in high-income countries…and only then would you have accurate data.”

Linking Data to Programs and Policy

Data help inform which health programs are funded and what type of policy response will be put into place. But which data are studied matters: focusing only on the number of annual maternal and under-5 deaths leaves out important metrics of program impact and challenges. Robert Black at Johns Hopkins University says, “I believe that continued monitoring of maternal and child mortality trends is important to provide evidence of health impact of programs, but needs to be combined with other evidence on the coverage and quality of health services that are proven to have desirable health outcomes.” In addition, too much focus on global estimates and success in lowering the number of annual child and maternal deaths covers up national and regional discrepancies. According to the UN, half of global deaths of children under 5 occur in sub-Saharan Africa. Focusing on global estimates can also complicate messages to policymakers. Advocacy messages need to be tailored to each country’s situation for an effective policy response.

Ultimately, estimates can convey the state of maternal and under-5 mortality, but they are also abstractions and cannot be completely accurate. All estimates on public health involve a choice of which data to include and from which source, depending on a judgment on the quality of data and summary indicators that are used. Understanding how estimates are made and which methods are used is important to program managers and policymakers. But just focusing on methodologies and changing estimates overlooks the fundamental issue: Despite differing estimates and modeling, it is clear that progress has been insufficient toward stopping preventable deaths. According to Boerma, “On the one hand, I think it’s really important if there is good news…But at the same time, it needs to be said that it falls way short of the MDG target. So the message should be that progress can be made even in difficult circumstances, but it’s not enough to achieve the goals…If we can do more, mobilize more funds, invest in the most effective interventions, deliver those interventions, then that actually can pay off.”

Eric Zuehlke is an editor at the Population Reference Bureau.

References

1. UNICEF, Progress for Children: A Report Card on Maternal Mortality (New York: UNICEF, 2008); and UNICEF, State of the World’s Children 2009 (New York: UNICEF, 2008).
2. Tiffany O’Callaghan, “Fewer Women Dying During Childbirth,” Time, April 14, 2010.
3. Institute for Health Metrics and Evaluation, Building Momentum: Global Progress Toward Reducing Maternal and Child Mortality (Seattle: IMHE, 2010).
4. Ties Boerma, personal communication, Aug. 26 2010.
5. UN News Service, “Efforts to Boost Maternal and Child Health Falling Short, UN Report Finds,” (June 23, 2010), accessed at www.un.org/apps/news/, on Aug. 27, 2010.