NEWS
Can better health statistics save lives?
Anita Elash
Paris
An international campaign for universal access to treatment for people with HIV/AIDS has spurred a quest for better quality health data, particularly in developing countries.
One of the central themes of this year's International AIDS Conference, held in Toronto, Canada in August, was a renewed call for universal access to treatment for HIV/AIDS. For Dr Peter Ghys this was all the more reason to pursue one of his main assignments at UNAIDS - improving the quality of data that measure the epidemic. Faced with discrepancies in published statistics and demands from donors for better information, Ghys and his counterparts at WHO have been charged with producing more accurate numbers in a more timely way. "All diseases should have accurate figures, and our numbers form the basis of a lot of important actions related to the fight against AIDS," says Ghys, Manager of Epidemic and Impact Monitoring at UNAIDS. "They are useful to try to quantify what universal access would mean, so it becomes very important."
These efforts have produced important results. In the 2006 Report on the global AIDS epidemic, released in New York City in May, UNAIDS said that the epidemic is smaller than previously thought and changed its estimate of the number of people living with HIV infection to 38.6 million, down from the 40.3 million it had estimated six months earlier. Most of the decrease was due to better information, and not an overall decline.
The efforts of UNAIDS to improve figures represent just one of several initiatives designed to gather more accurate data and harmonize health statistics. One such initiative is the Health Metrics Network, which aims to help developing countries improve health information systems and make timely and accurate information available around the world. Another example is the Mortality Coordination Group, made up of WHO, UNICEF, the World Bank and the UN Population Division that aims to harmonize under-five mortality estimates. WHO has also implemented new systems to harmonize data and make results universally accessible.
Discrepancies in public health data have often led to controversy and raised questions about how health policy is formulated and where health dollars should go. The issue was highlighted this spring, when Greenpeace estimated that a much higher number of people were directly affected by the Chernobyl nuclear disaster than had been estimated by the International Atomic Energy Agency (IAEA) and WHO. Researchers have also had difficulty producing accurate figures for many causes of death, from tuberculosis and malaria to injuries and neglected tropical diseases, because of the lack of data, especially from countries where mortality is high.
Dr Kenji Shibuya, Coordinator of the WHO Health Statistics and Evidence unit, says the push to produce better information, and to come up with one number that everyone agrees on, is fuelled partly by the demands of donors, who are increasingly concerned about whether their money is being well-spent. Developing countries have also recognized that they need better information as they struggle with multiple demands for data to monitor progress towards the Millennium Development Goals and the performance of their health systems.
In order to improve the detection and response to public health emergencies around the world, WHO's Member States have committed themselves from June 2007 to report cases of certain diseases under the revised International Health Regulations. Researchers hope that accurate statistics will lead to better outcomes. "It is actually difficult to prove this, but good statistics should contribute to good decision-making based on evidence, which will eventually lead to improved health status in a population," says Shibuya.
The harmonization efforts to improve health statistics already in place are based on two principles. First, there should be an independent peer-review process of the figures generated. And second, all the information should be publicly available as soon as it is produced. For example, the Mortality Coordination Group meets at least once a year to share information, discuss what is known about how the numbers were gathered, and agree on which data to accept. WHO takes a similar approach for diseases in general, and now requires every programme that produces estimates to use peer-reviewed data and have an up-to-date database. It is essential that statistics are published with an explanation of how they were obtained; in many instances they often state a possible error range.
A group of experts from UNAIDS and WHO meets weekly to review the latest data on AIDS and to advise on data collection methods. But the biggest change in AIDS data is in the quality of the information available. Ghys says the data improved after local health officials were trained how to use more comprehensive computer programs to produce and analyse their own information. "In the past, estimates were largely a Geneva-based exercise, with little involvement from the countries," he says. "If analysts produce their own estimates, they take it more seriously than if it 'dropped out' of Geneva."
Many countries have also expanded their surveillance systems. Until recently, HIV and AIDS estimates were based on tests of pregnant women visiting antenatal clinics, mostly located in cities. The tests were useful in tracking trends, but had limitations for measuring the actual size of the epidemic. Since 2000, a group of 20 countries in sub-Saharan Africa has conducted nationwide household surveys with HIV testing, and some have expanded their surveillance to antenatal clinics in rural areas. When they looked at their respective countries as a whole, researchers in several countries discovered that the prevalence of HIV infection was far lower than previously thought.
Some commentators say the programmes still have a long way to go. John Donnelly, who writes about global health issues for the Boston Globe, argues that UNAIDS and WHO have been far too conservative in downgrading their estimates. Donnelly believes that as more information is gathered, HIV/AIDS prevalence figures will continue to drop. He says that instead of downgrading the figures slowly, it would be "more honest" to admit that information from countries that haven't done nationwide health surveys is inaccurate, and to adjust the worldwide estimate based on results from those countries that have done the surveys. "The global estimate is an advocacy tool, and it [downgrading the numbers] would be a huge embarrassment," he says. "But it would help national governments refocus their programmes."
According to Dr Ties Boerma, Director of WHO's Department of Measurement and Health Information Systems, "this may have been valid before the 2006 Report on the global AIDS epidemic - but not any more". Nevertheless, Kenji Shibuya believes that harmonization still has a long way to go. Boerma agrees: "We won't get the truth easily".
Much more investment is needed in developing countries to improve data collection, such as national health surveys, says Boerma. But even for countries that do carry out national health surveys, there is no guarantee of accuracy. In South Africa, for example, response rates to national health surveys are under 60% and even lower than this among the white population, making it hard to assess how representative the surveys are. Countries also rely on health workers to report diseases, but in many developing countries record-keeping and reporting are poor.
Another challenge is the availability of multiple sources of data - surveys, civil registration and health facility and administrative records. These data need to be harmonized to come up with the best possible estimates, Boerma says. Another challenge, he says, is "an inclination to overestimate" for the sake of advocacy.
Boerma admits that it may never be possible to come up with just one number when estimating the burden of a disease or the size of an epidemic, because there will always be a range of uncertainty. Moreover, although there may always be disagreements over the final estimates these often lead to constructive discussion and better information. "The best we can do is to be completely transparent," he says. "And if there is a disagreement to explain very clearly why." 
Anita Elash, Paris