ARTICLE ARTIGO

 

Cost-effectiveness of community health worker versus home-based guardians for directly observed treatment of tuberculosis in Vitória, Espírito Santo State, Brazil

 

Custo-efetividade dos agentes comunitários de saúde versus supervisores domiciliares na realização do tratamento supervisionado da tuberculose, Vitória, Espírito Santo, Brasil

 

 

Thiago Nascimento do PradoI; Nikolas WadaII; Leticia Molino GuidoniI; Jonathan E. GolubII; Reynaldo DietzeI; Ethel Leonor Noia MacielI

IUniversidade Federal do Espírito Santo, Vitória, Brasil
IIJohns Hopkins University, Baltimore, USA

Correspondence

 

 


ABSTRACT

The objective of this study was to compare the costs and outcomes associated with guardian-supervised directly observed treatment relative to the standard of care Directly Observed Therapy, Short Course (DOTS) provided by community health workers (CHW). New cases of culture-positive pulmonary tuberculosis (TB) treated in Vitória, Espírito Santo State, Brazil, between January 2005 and December 2006 were interviewed and chose their preferred treatment strategy. Costs incurred by providers and patients (and patients' families) were estimated, and cost-effectiveness was assessed by comparing costs per successfully treated patient. 130 patients were included in the study; 84 chose CHW-supervised DOTS and 46 chose guardian-supervised DOTS. 45 of 46 (98%) patients treated with guardian-supervised DOTS were cured or completed treatment compared to 70/84 (83%) of the CHW-supervised patients (p = 0.01). Logistic regression showed only the strategy of supervision to be a significant association with treatment outcome, with guardian-supervised care strongly protective. Cost per patient treated with guardian-supervised DOTS was US$398, compared to US$548 for CHW-supervised DOTS. The guardian-supervised DOTS is an attractive option to complement CHW-supervised DOTS.

Tuberculosis; Cost-Benefit Analysis; Effectiveness


RESUMO

Comparar os custos e os resultados associados ao tratamento de tuberculose (TB) supervisionado por domiciliares quanto ao realizado pelos agentes comunitários de saúde (ACS). Participaram do estudo todos os casos de TB pulmonar com cultura positiva tratada na cidade de Vitória, Espírito Santo, Brasil, entre janeiro de 2005 e dezembro de 2006. Os pacientes escolheram a estratégia de tratamento preferencial. Os custos incorridos pelos prestadores e os doentes foram estimados, e relação custo-efetividade foi avaliada comparando os custos por doente tratado com sucesso. Um total de 130 pacientes foi incluído no estudo, 84 escolheram ACS e 46 escolheram tratamento supervisionado por domiciliares. 45 de 46 (98%) dos doentes tratados com supervisionamento por domiciliares foram curados ou tratamento completado em comparação com 70/84 (83%) dos pacientes ACS (p = 0,01). Regressão logística mostrou o tratamento supervisionado por domiciliares significativamente protetor em relação ao abandono do tratamento da TB ao realizado pelo ACS. Custo por paciente tratado com o tratamento supervisionado por domiciliares foi de US$ 398, em comparação com US$ 548 para ACS. Tratamento supervisionado por domiciliares é uma opção mais custo-efetividade do que a supervisão pelo ACS.

Tuberculose; Análise Custo-Benefício; Efetividade


 

 

Introduction

Tuberculosis (TB) is a serious public health problem, especially in developing countries. Brazil occupies the 16th position among the 22 high-burden countries that comprise 80% of the estimated cases in the world 1. The primary TB control policy endorsed by the World Health Organization (WHO) and the International Union against Tuberculosis and Lung Disease is DOTS (Directly Observed Therapy, Short-Course). The DOTS includes five elements. Among them, directly observed treatment is a key component. Current technical manuals define directly observed treatment as direct supervision of medication ingestion by a treatment supporter who is acceptable and accountable to the patient and to the health system 2. The treatment supporter could be a heath professional, community health worker, or a family member 2,3,4,5.

There is evidence to support the effectiveness of the overall DOTS strategy and direct observation of treatment from a number of observation studies 6,7,8,9. However, a systematic review 10 of the evidence for direct observation from randomized controlled trials found that although one trial 11 demonstrated significant improvements in cure and treatment, two other trials found similar outcomes for directly observed and self administered patient treatment groups 12,13. Furthermore, directly observed treatment incurs considerable costs to the patient for direct observation 14 and requires considerable healthcare resources, particularly in countries where the prevalence of TB is high.

Although there is by no means a consensus, some observational studies and clinical trials 3,4,5,15 have shown that treatment completion rates were significantly greater when treatment was supervised by a family member, compared to supervision by health professionals. In a controlled trial in Nepal, treatment supervision by family members was as effective as treatment supervised by health professionals 4. In Brazil, two observational studies evaluated the use of family members as directly observed treatment supervisors suggested that the strategy was effective, feasible, and affordable 5,15,16,17. Other studies worldwide found that community health supervision of TB patients have been piloted in a variety of settings with successful results in cure rate 18,19,20,21,22,23.

Efforts to expand directly observed treatment to more health facilities have been limited by scarce resources and understaffing, making it a high priority to identify new approaches to directly observed treatment that maintain effectiveness and utilize fewer resources. One such approach could be training home supervisors (guardian-supervised) to provide directly observed treatment. However, little is known about the cost-effectiveness of this option in Brazil. Comparing the costs and outcomes of guardian-supervised directly observed treatment to those associated with the standard of care may assist policy makers in decision-making and in the rational allocation of scarce health resources. We conducted this study to compare the costs and outcomes associated with community health worker (CHW)-supervised and guardian-supervised directly observed treatment in Vitória, the capital of the state of Espírito Santo, Brazil.

 

Methods

Study setting

The study was conducted in Vitória, a city of approximately 300,000 people, is characterized by rapid population growth, socioeconomic indicators that are near the median for Brazil, and TB incidence of 54.24 per 100,000 (DATASUS. http://tabnet.datasus.gov.br/cgi/tabcgi.exe?idb2009/d0202.def, accessed on 24/May/2010). Vitória has three health facilities operating TB control programs. This study included Maruipe Health Unit and Cassiano Antônio de Moraes University Hospital (HUCAM); we did not include the third facility because it was in transition during the study period and did not employ the same TB control strategy as the others.

Community health care worker-supervised DOTS vs. guardian-supervised DOTS

We compared two approaches to TB control: the standard of care, which provides DOTS via mobile CHW, and an intervention in which guardians supervise DOTS. In the conventional CHW approach, patients are diagnosed on an ambulatory basis before treatment. Trained health care workers then directly observe TB treatment in the patient's home five times weekly during the first two months, and twice weekly for the final four months of treatment. A nurse reviews the patient's treatment adherence and if it is found to be poor, a registered nurse undertakes the home visits.

By contrast, in the guardian-supervised DOTS intervention, guardians (ideally close relatives who live in the same residence as the patient) supervise treatment in the patient's home. The patients assist in guardian selection.

Prior to treatment initiation, guardians and patients were trained and instructed regarding TB pathology, DOTS protocols, and proper recording of treatment. The patients and guardians collected drugs from the health facilities twice a month during treatment. At these visits, a nurse reviewed treatment adherence by reviewing treatment cards and pill counts. As in the CHW approach, a nurse would take over patient care if poor adherence was observed. All patients visited the clinic once a month for physician appointments.

Study participants

Patients were selected from the Vitória TB treatment registers. All new cases of smear-positive or culture-positive pulmonary TB treated at Maruipe Health Unit and HUCAM in Vitória city between January 2005 and December 2006 were initially eligible for the study (n = 181).

The patients were interviewed and informed about both treatment strategies. Upon consent, patients were allowed to choose their preferred treatment strategy. Patients were excluded from the study if they were under 18 years of age (n = 13), if they had been previously treated for pulmonary TB (n = 13), if they were culture-negative (n = 46), or if they were HIV-positive (n = 19). After exclusion, 130 patients were eligible for the study (some patients met more than one of the exclusion criteria). We excluded the prior history of TB patients because these patients could be multi-drug-resistant TB and the cost of treatment is higher.

Statistical analysis

Univariate analyses were performed for each variable of interest, and those associated with the outcome with a p < 0.10 were included in a multivariate logistic regression.

Costs

Total costs of each component of care and treatment were calculated by multiplying the quantity of resources used by their unit price. Only post-diagnosis costs are considered, since diagnostic procedures are identical in each treatment approach.

Each cost pertains to either the "provider" or the "community". Provider costs are incurred by the healthcare system, and are associated with operating and maintaining healthcare services. These costs include staff, overhead, supplies and equipment. Community costs are those incurred by patients and treatment supervisors. These include direct and indirect costs. Direct costs are non-medical costs related to treatment, such as the costs incurred at the TB clinic (e.g., food and drink). Indirect costs refer to the value of lost time by the patients and their guardians.

Patients were interviewed about their travel costs, time lost, and other costs associated with TB treatment and care. The opportunity cost of time was converted to a monetary value based on the average reported income (from all sources) among interviewed patients. Costs incurred by guardians were assessed with a questionnaire asking about the time and travel costs associated with supervision, drug collection, and training. The final values are provided in US dollars, with an exchange rate of 1.85 Brazilian Reais (R$) per US$.

Cost-effectiveness

The measure of effectiveness used was treatment success, defined as either DOTS cure or DOTS completion, as per WHO classifications. Cured patients were defined as those with a confirmed negative sputum smear and no TB signs and symptoms six months after starting treatment. Final costs were estimated and costs per patient cured were compared for the two strategies. The incremental cost-effectiveness ratio was defined as difference in total costs between intervention and control divided by the difference in the number of patients cured between intervention and control. The numerator of the incremental cost-effectiveness is defined as the difference (per patient treated) in the costs between intervention and control. The denominator of the incremental cost-effectiveness is defined as the difference in number of patients cured between intervention and control, normalized by the number of patients treated in each group. This means that we are taking the difference in the proportion of patients cured in each group. The reason we normalize the numbers in both the numerator and denominator by the number of patients treated is that the groups were not of equal size. Interpretation is the cost per additional patient cured.

Institutional review board approval

This study was approved by the Institutional Review Board of the Federal University of Espírito Santo and patients provided informed consent.

 

Results

Among 130 patients included in the study, 84 patients (65%) chose the standard of care (CHW-supervised DOTS) and 46 (35%) received the intervention (guardian-supervised DOTS). Table 1 shows the demographic and geographic characteristics of the study population. There was no statistically significant difference among the groups regarding sex, age, race and occupation. Illiteracy was higher in the guardian-supervised DOTS group (30%) relative to the CHW-supervised group (15%; p = 0.04). More people in the guardian-supervised group lived at least one kilometer away from a clinic (87% vs. 2%, p = 0.01). There was no statistically significant difference between groups in the proportion of patients with a positive sputum smear (p = 0.31, Table 1).

We observed that 45 out of 46 (98%) patients treated with guardian-supervised DOT were cured or completed treatment. By contrast, only 70 of 84 (83%) of the CHW-supervised patients were cured or completed treatment. This difference was statistically significant (p = 0.01, OR = 0.11; 95%IC: 0.002-0.79). The 15 patients who did not complete treatment were lost to follow-up, and did not initiate secondary treatment at either of the two treatment facilities in the study at the time this study was completed.

We fit a logistic regression model with cure/treatment completion as the dependent variable (Table 2). The explanatory variables were sex, strategy of supervision, education status and distance between home and clinic (> 1km vs. < 1km). In this analysis, only the strategy of supervision showed a significant association with treatment outcome, with guardian-supervised care strongly protective (OR = 9.07; 95%CI: 1,14-70,83).

Costs and cost-effectiveness

The total cost per patient of both strategies is shown in Table 3. The cost per patient treated with CHW-supervised DOT was US$548 compared to US$389 for guardian-supervised DOTS. The difference between the two was largely due to significant differences in provider costs. In the CHW-supervised DOTS strategy, home visits comprised the largest portion of costs, but these costs were entirely avoided by the guardian-supervised DOTS strategy. Guardian-supervised DOTS cost, on average, was US$398 per patient cured. This figure was US$260 (39%) lower than its equivalent for CHW-supervised DOTS (US$657), resulting in savings of US$1,095 per additional patient cured. Because patients and their guardians made more trips to the treatment facility under the intervention, costs incurred by the patients and their guardians were higher in guardian-supervised DOTS (US$85 vs. US$30 per patient cured). This difference translates to an incremental cost-effectiveness ratio (from the perspective of patients and their guardians) of US$401 per additional patient cured (table 4). The numerator of the incremental cost-effectiveness ratio is defined as the difference (per patient treated) in the costs between intervention and control, or (547.82-389.17) = 158.65. The denominator of the incremental cost-effectiveness ratio is [(70/84) - (45/46)] = (0.83 - 0.98) = -0.14. So the incremental cost-effectiveness ratio is then 158.65/-0.14 = -1,094.7. This implies that there is a cost saving, from an overall perspective, of $1,095 per additional patient cured.  

 

Discussion

In this study population, guardian-supervised DOTS was more effective and less costly than CHW-supervised DOTS. Implementation of guardian-supervised DOTS exceeded the quality of patient outcomes from CHW-supervised DOTS, incurred 32% lower costs per patient treated, and incurred 45% lower costs per patient cured. In cost-effectiveness parlance, the results indicate that guardian-supervised DOTS was the dominant strategy.

Although guardian-supervised DOTS was less expensive for society as a whole, it was nearly three times more expensive per patient cured for the community. This is because patients visited the health care facility twice as often under guardian-supervised DOTS, and guardians accompanied them during each visit. The increased burden on families is an important concern, but this could be addressed in theory by direct payments from the health care provider to the families. Even if providers compensated patients and guardians for all their costs under guardian-supervised DOTS, providers would still save US$134 per patient treated relative to CHW-supervised DOTS. Such compensation would have obvious equity benefits, and indeed may be a necessary component to the intervention in future applications.

The main reason for the substantial reduction of costs under guardian-supervised DOTS was the reduction in CHW time dedicated to a single patient. Each CHW is responsible for approximately 750 people in the community, visiting each family once per month. The CHW provides primary education in health, vaccinations, monitoring of risk groups, and other social services for households 24. As such, the time demands on CHWs are high 25. Any reduction in the time burden for CHWs may have positive benefits for other community members that are not captured in this simple cost analysis.

The study has a number of limitations. First, we evaluate a limited number of risk factors among all the possibilities involved in the theoretical model of tuberculosis treatment. Second, the non-randomized design, where patients chose their own type of supervisor, precludes definitive statements since patients were allowed to select their intervention raising the possibility of selection bias. Nevertheless, a non-randomized design was used because it more closely replicates program conditions where newly diagnosed TB patients participate in choosing their treatment supervisor. Patients in the guardian-supervised DOTS group may have been more likely to complete treatment successfully. If this was the case, we may approximate an upper bound for the bias by assuming that costs are intrinsic to the program, but outcomes are intrinsic to the patients. In a counterfactual scenario where the 46 guardian-supervised DOTS patients were instead in the CHW-supervised DOTS program (but had identical outcomes) and vice versa, the cost per patient cured is US$560 in the CHW-supervised DOTS program, and US$467 in the guardian-supervised DOTS program. Although the advantages of guardian-based care are reduced in this conservative scenario, the guardian-supervised program still results in savings of US$93 per patient cured.

It is likely that the patients who chose guardian-supervised care were also the most likely to benefit from it (e.g., they had stronger support networks at home and family resources, or lived farther from the clinic). From a programmatic perspective, this selection bias would be beneficial. Allowing patient choice should be strongly considered if the intervention is put into practice on a wider scale, as this would allow patients to choose a treatment program according to their own needs, and thus might increase the probability of treatment success. Patients without strong social networks or those for whom guardian-based care is otherwise inappropriate would be able to utilize conventional care.

Another limitation is that the results represent a combination of outcomes and costs for smear-negative and smear-positive TB patients. However, the proportion of smear-positive patients did not differ significantly between treatment groups, and therefore was unlikely to affect the primary conclusion of the study.

There is no information on the 15 patients who did not complete treatment. If poor adherence as a consequence of either treatment program led to resistance, and thus to further illness, expensive second-line therapy, and/or death, then it can be argued that the total cost of the treatment program in question should reflect that. Only one patient from guardian-based DOTS did not complete treatment, while 14 patients did not complete CHW-supervised treatment. If the loss to follow-up obscures any program costs, it is likely that the analysis underestimates the relative cost savings from guardian-supervised DOTS.

We calculated neither the secondary costs of TB transmission, nor the indirect costs to the community of mortality. Treatment of a single case of smear-positive TB has been estimated to avert five deaths from subsequent transmission cycles over two decades. By ignoring the costs of secondary transmission, our cost-effectiveness estimates are conservative. However, quantifying the secondary benefits of TB treatment involves complex assumptions beyond the scope of this analysis.

We used average reported income among TB patients as a means of estimating the costs due to lost time, though other approaches have been suggested 26,27,28,29,30.

This method may have over- or under-estimated costs, depending on the distribution of income among patients.

Studies in Malawi and Kenya reported cost reductions from community-supervised DOTS of 50% and 65%, respectively 22,23. These cost reductions were higher than in our study, primarily because our study did not include hospitalized TB patients. In Brazil, TB patients are treated on an ambulatory basis and admitted only when seriously ill.

The situation in Vitória exemplifies the growing challenge of controlling TB in urban areas, but the extent to which these findings may be generalized depends greatly on the household structure of TB patients. Guardian-based DOTS clearly relies heavily on a strong family support network among TB patients.

Our results indicate that guardian-supervised DOTS may be less costly than, and comparably effective to, conventional health facility treatment in Vitoria. However, guardian-supervised DOTS is likely to be more expensive to patients and their families. This increased financial burden may be entirely offset by compensation from providers. Even with full compensation, guardian-supervised DOTS would be less costly to providers. Where successful, guardian-based DOTS frees up resources and health care worker time for other important tasks in TB control. In addition, guardians may more effectively mobilize a network of family support around the patient's treatment than can healthcare workers. The involvement of other household members may be decisive for treatment completion and cure.

 

Contributors

T. N. Prado, R. Dietze and E. L. N. Maciel designed the study and participated in the data collection and analysis and the writing of the article. N. Wada and J. E. Golub contributed to the data analysis and write-up. L. M. Guidoni participated in the data collection and analysis, as well as the write-up.

 

Acknowledgments

To ICOHRTA AIDS/TB (project 2 U2R TW006883-06) and Edital CTSAUDE/Edital MCT/CNPq/CT-Saúde/MS/SCTIE/DECIT nº 67/2009 - REBRATS.

 

References

1. World Health Organization. Global tuberculosis control: surveillance, planning, financing. WHO Report 2006. Geneva: World Health Organization; 2006.         

2. Hopewell PC, Pai M, Maher D, Uplekar M, Raviglione MC. International standards for tuberculosis care. Lancet Infect Dis 2006; 6:710-25.         

3. Okanurak K, Kitayaporn D, Wanarangsikul W, Koompong C. Effectiveness of DOT for tuberculosis treatment outcomes: a prospective cohort study in Bangkok, Thailand. Int J Tuberc Lung Dis 2007; 11:762-8.         

4. Newell JN, Baral SC, Pande SB, Bam DS, Malla P. Family member DOTS and community DOTS for tuberculosis control in Nepal: cluster-randomised controlled trial. Lancet 2006; 367:903-9.         

5. Maciel ELN, Silva AP, Meireles W, Fiorotti K, Hadad DJ, Dietze R. Tratamento Supervisionado em pacientes portadores de tuberculose utilizando supervisores domiciliares em Vitória, ES, Brasil. J Bras Pneumol 2008; 34:506-13.         

6. Wilkinson D. High compliance tuberculosis treatment programme in a rural community. Lancet 1994; 343:647-8.         

7. China Tuberculosis Control Programme. Results of directly observed short course chemotherapy in 112,842 Chinese patients with smear-positive tuberculosis. Lancet 1996; 347:358-62.         

8. Chowdhury AMR, Chowdhury A, Islam MN, Vaughan JP. Control of tuberculosis by community health workers in Bangladesh. Lancet 1997; 350:169-72.         

9. Stop TB Department. Community contribution to TB care: practice and policy. Geneva: World Health Organization; 2003. (WHO/CDS/TB/2003.312.          ).

10. Volmink J, Garner P. Directly observed therapy for treating tuberculosis. Cochrane Database Syst Rev 2007; (4):CD003343.         

11. Kamolratanakul P, Sawert H, Lertmaharit S, Kasetjaroen Y, Akksilp S, Tulaporn C, et al. Randomized controlled trial of directly observed treatment (DOT) for patients with pulmonary tuberculosis in Thailand. Trans R Soc Trop Med Hyg 1999; 93:552-7.         

12. Zwarenstein M, Schoeman J, Vundule C, Lombard C, Tatley M. Randomised controlled trial of self-supervised and directly observed treatment of tuberculosis. Lancet 1998; 352:1340-3.         

13. Walley J, Khan A, Newell J, Khan H. Effectiveness of direct observation component of DOTS for tuberculosis: a randomised controlled trial in Pakistan. Lancet 2001; 357:664-9.         

14. Khan MA, Walley J, Witter SN, Imran A, Safdar N. Costs and cost effectiveness of different DOTS strategies for treatment of tuberculosis: economic results of a randomised controlled trial in Pakistan. Health Policy Plan 2002; 17:178-86.         

15. Maciel EL, Guidoni LM, Brioshi AP, Prado TN, Fregona G, Hadad DJ, et al. Household members and health care workers as supervisors of tuberculosis treatment. Rev Saúde Pública 2010; 44:339-43.         

16. Akkslip S, Rasmithat S, Maher, Sawert H. Direct observation of tuberculosis treatment by supervised family members in Yosothorn Province, Thailand. Int J Tuberc Lung Dis 1999; 3:1061-5.         

17. Wyss K, Kilima P, Lorenz N. Cost of tuberculosis for households and health care providers in Dar Es Salaam. Trop Med Int Health 2001; 6:60-8.         

18. Lwilla F, Schellenberg D, Masanga H. Evaluation of efficacy of community-based vs institutional-based direct observed short-course treatment for the control of tuberculosis in Kilombero district, Tanzania. Trop Med Int Health 2003; 8:204-10.         

19. Wandwalo E, Kapalata N, Egwaga S, Morkve O. Effectiveness of community based directly observed treatment for tuberculosis in an urban setting in Tanzania: a randomised controlled trial. Int J Tuberc Lung Dis 2004; 8:1248-54.         

20. Lwilla F, Schellenberg D, Masanja H, Acosta C, Galindo C, Aponte J, et al. Evaluation of efficacy of community based vs. Institutional based direct observed short course treatment for the control of tuberculosis in Kilombero district, Tanzania. Trop Med Int Health 2003; 8:204-10.         

21. Nedham DM, Godfrey-Faussett P, Foster SD. Barriers to tuberculosis control in urban Zambia: the economic impact and burden on patients prior to diagnosis. Int J Tuberc Lung Dis 1998; 2:811-7.         

22. Floyd K, Skeva J, Nyirenda T, Gausi F, Salaniponi F. Cost and cost-effectiveness of increased community and primary care facility involvelment in tuberculosis care in Lilongwe District, Malawi. Int J Tuberc Lung Dis 2003; 7(9 Suppl 1):S29-37.         

23. Nganda B, Wangombe J, Floyd K, Kangangi J. Cost and cost-effectiveness of increased community and primary care facility involvement in tuberculosis care in Machakos District, Kenya. Int J Tuberc Lung Dis 2003; 7(9 Suppl 1):S14-20.         

24. Ministério da Saúde. Programa Agentes Comunitários de Saúde. Brasília: Ministério da Saúde; 2001.         

25. Ferraz L, Aerts DRGC. O cotidiano de trabalho do agente comunitário de saúde no PSF em Porto Alegre. Ciênc Saúde Coletiva 2005; 10:347-55.         

26. World Health Organization. Guidelines for cost and cost effectiveness analysis of tuberculosis control. Geneva: World Health Organization; 2002.         

27. Muennig P. Designing and conducting cost-effectiveness analysis in medicine and health care. San Francisco: John Wiley and Sons; 2002.         

28. Murray CJM, DeJonghe E, Chum HJ, Nyangulu DS, Salomao A, Styblo K. Cost-effectiveness of chemotherapy for pulmonary TB in three sub-Saharan African countries. Lancet 1991; 338:1305-8.         

29. World Bank. World Development Report 1993. Investing in health. New York: Oxford University Press; 1993.         

30. Johannesson M, Karlsson G. The friction cost method: a comment. J Health Econ 1997; 16:249-55.         

 

 

Correspondence:
T. N. Prado
Laboratório de Epidemiologia
Núcleo de Doenças Infecciosas
Universidade Federal do Espírito Santo
Av. Marechal Campos 1468
Vitória, ES 29040-091, Brasil
tiagufes@yahoo.com.br

Submitted on 11/Jul/2010
Final version resubmitted on 25/Feb/2011
Approved on 28/Feb/2011

Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz Rio de Janeiro - RJ - Brazil
E-mail: cadernos@ensp.fiocruz.br