Contributions from the systematic review of economic evaluations: the case of childhood hepatitis A vaccination in Brazil


Contribuições da revisão sistemática de avaliações econômicas: o exemplo da vacinação infantil contra hepatite A no Brasil



Patrícia Coelho De SoárezI; Ana Marli Christovam SartoriII; Andreia SantosIII; Alexander ItriaI; Hillegonda Maria Dutilh NovaesI; Celina Maria Turchi MartelliIV, V

IFaculdade de Medicina, Universidade de São Paulo, São Paulo, Brasil
IIHospital das Clínicas da Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brasil
IIILondon School of Hygiene and Tropical Medicine, London, U.K.
IVInstituto de Patologia Tropical e Saúde Pública, Universidade Federal de Goiás, Goiânia, Brasil
VPrograma de Pós-graduação em Medicina Tropical, Universidade Federal de Pernambuco, Recife, Brasil





The aim of this study was to present the contributions of the systematic review of economic evaluations to the development of a national study on childhood hepatitis A vaccination. A literature review was performed in EMBASE, MEDLINE, WOPEC, HealthSTAR, SciELO and LILACS from 1995 to 2010. Most of the studies (8 of 10) showed favorable cost-effectiveness results. Sensitivity analysis indicated that the most important parameters for the results were cost of the vaccine, hepatitis A incidence, and medical costs of the disease. Variability was observed in methodological characteristics and estimates of key variables among the 10 studies reviewed. It is not possible to generalize results or transfer epidemiological estimates of resource utilization and costs associated with hepatitis A to the local context. Systematic review of economic evaluation studies of hepatitis A vaccine demonstrated the need for a national analysis and provided input for the development of a new decision-making model for Brazil.

Cost-Effectiveness Evaluation; Cost-Benefit Analysis; Hepatitis A; Immunization Programs


O objetivo deste estudo foi apresentar as contribuições da revisão sistemática de avaliações econômicas para o desenvolvimento de um estudo nacional, o caso da vacinação infantil contra hepatite A. Foi realizada revisão da literatura nas bases de dados EMBASE, MEDLINE, WOPEC, HealthSTAR, SciELO e LILACS, no período de 1995 a 2010. A maioria dos estudos (8 em 10) mostrou resultados favoráveis de custo-efetividade. As análises de sensibilidade indicaram como parâmetros mais importantes para os resultados os custos da vacina, incidência de hepatite A e custos médicos da doença. Foi observada variabilidade nas características metodológicas e estimativas de variáveis-chaves dos 10 estudos revisados. Não é possível generalização dos resultados e transferibilidade de estimativas epidemiológicas, de usos de recursos e custos associados à hepatite A para o contexto local. A revisão sistemática dos estudos de avaliação econômica da vacina contra hepatite A demonstrou a necessidade de uma análise nacional e forneceu elementos para o desenvolvimento de um novo modelo de decisão para o Brasil.

Avaliação de Custo-Efetividade; Análise Custo-Benefício; Hepatite A; Programas de Imunização




In health systems with an ever-increasing demand and limited resources, economic evaluation studies have become an important tool for policy decisions on the incorporation of new health technologies.

Systematic reviews of economic studies have become an essential part of technological evaluation and public policymaking processes. Some national agencies for the appraisal of health technologies [like the National Institute for Health and Clinical Excellence (NICE), Pharmaceutical Benefits Advisory Committee (PBAC), and Canadian Agency for Drugs and Technologies in Health (CADTH)] require systematic review of the relevant economic literature for evaluation and policymaking processes in public health 1.

The methodology for summarizing the results of economic evaluations is not as well-developed as that applied to clinical evidence. The most solidly established methods for summarizing economic evaluations are: narratives summary, the cost-effectiveness plane, and permutation matrices 2.

The summary should begin with descriptive comparisons of the study questions, methods, and results. It can be useful to include tables that summarize the key information on the population, country, study perspective, comparison of interventions, outcomes, summary measures, and incremental cost-effectiveness ratios 3. Theoretically, a meta-analysis of economic evaluations can be performed, but in practice it is not a simple procedure 4. The lack of details in estimates, variability in the outcomes used for effectiveness, and heterogeneity of the economic evaluations identified in the literature hinder a meta-analysis of the results 3. In most cases, statistical "pooling" of the cost-effectiveness estimates (meta-analysis) is neither feasible, nor does it make sense 1. Narrative synthesis has been the most widely used method.

The systematic review of economic studies has been indicated to: (1) contribute to the development of a new decision model; (2) identify one or two more relevant studies to inform a particular decision in a jurisdiction; or (3) identify the principal economic "trade-offs" in a given treatment choice or disease area 1.

In Brazil, the National Immunization Program (PNI) commissioned an economic study to estimate the cost-effectiveness of incorporating the hepatitis A vaccine. The hepatitis A vaccine has been commercially available since the early 1990s 5 and has proven safe and effective. In Israel, universal childhood vaccination has been conducted with great success since 1999 6,7.

Argentina in 2005 8 and the United States in 2006 9 also implemented programs for universal childhood vaccination against hepatitis A, with a major impact on the epidemiology of the disease 10. The pattern of hepatitis A endemicity in Brazil suggests that the introduction of a universal childhood vaccination program would be beneficial.

In this context, a systematic review of previous economic evaluations is necessary in order to verify whether a recent analysis has been performed in Brazil, with similar objectives. Two systematic reviews on the hepatitis A vaccine were published recently, approaching methodological issues related to the use of economic evaluation models 11 and the methodological quality of economic evaluations of various vaccines 12.

A systematic review is thus necessary that includes complete economic evaluation studies analyzing the use of the hepatitis A vaccine in a program for universal vaccination of children up to six years of age. This cutoff was set for the current study considering the most adequate age for vaccination in Brazil, based on the epidemiological pattern of hepatitis A in transition, with regional variation, including areas of intermediate and low endemicity 13,14.

When a new economic evaluation is justified, a review of previous economic studies can provide input for the development of a new decision model. This type of review is useful for comparing and contrasting how different researchers made their choices in relation to the methodology and the estimation of key variables and to clarify how the results of analyses can differ according to these choices 15.

A systematic review of the existing economic studies allows evaluating whether the results can be generalizable or transferable and establishes the need (or lack thereof) for developing local studies to estimate the cost-effectiveness ratio of the new vaccine to be incorporated.

The aim of this study is to present the contributions of a systematic review of economic evaluations to the development of a Brazilian national study, based on the example of childhood hepatitis A vaccination. The article discusses the importance of this review and its applicability to the design of a local economic evaluation and the definition of public policies for immunization in the country.



Literature search

On May 26, 2010, a search was conducted in six databases: EMBASE, MEDLINE, WOPEC, HealthSTAR, SciELO, and LILACS. The strategy ("economic evaluation" OR "cost" OR "cost-benefit" OR "cost-effectiveness" OR "cost-utility" OR "decision analysis") AND ("hepatitis A" OR "hepatitis A vaccines") was used in all the databases and was limited to articles published from 1995 to May 2010. On December 30, 2010, a new search was conducted, and no new publications were identified.

Inclusion/exclusion criteria

The following selection criteria were used:

• Study design: complete economic evaluation, defined as a comparative analysis of costs and consequences of two alternative healthcare interventions; including cost-minimization analysis, cost-effectiveness analysis, cost-utility analysis, and cost-benefit analysis;

• Type of intervention: universal childhood hepatitis A immunization program;

• Population: children up to 6 years of age.

The review excluded incomplete economic evaluations, like cost analyses, and economic evaluations of specific groups such as workers, travelers, prison inmates, university students, adolescents, hepatitis C patients, and others.

The review also excluded studies that analyzed the combined hepatitis A+B vaccine, since this vaccine, like the hepatitis A vaccine, is recommended for use starting at 12 months, and hepatitis B vaccination is recommended soon after birth (1st dose administered in the maternity ward) 16 and is already included as routine in the Brazilian National Immunization Program.

Each study was reviewed in detail to extract and summarize information on:

• Methodological characteristics: type of study, perspective, model, herd protection, time horizon, number of cohorts, currency and year of costs, discount rate, sensitivity analysis, and parameters are varied in the sensitivity analysis;

• Estimates of key variables: epidemiological (incidence, symptomatic outcomes, and case-fatality rate); characteristics of the vaccine (vaccination scheme, coverage, efficacy, adverse events, and annual rate of waning of protection); costs (direct and indirect), and summary measures.

To compare the results of the studies, the summary measures or incremental cost-effectiveness ratios (ICERs) were converted into U.S. dollars (US$) for 2005 using purchasing power parity (PPP), which is the exchange-rate equivalent of an identical basket of goods and services in two countries 17. The monetary values of the results were corrected by the consumer price index for all annual urban consumers of healthcare in 2005 18. Whenever the study did not specify the year of cost, it was assumed that the year of cost was the same as that of the study's publication. This strategy had been adopted previously in the literature 12.

In addition, the research funding sources were identified in order to assess whether they influenced the studies' results.

The methodology used for summarizing economic evaluation studies was narrative synthesis. The methodological characteristics and estimates of key variables are shown in summary tables.



Literature review

The initial searches identified approximately 1,047 articles. After reading the titles and/or abstracts, 20 studies were considered potentially relevant and retrieved in full text. At this stage, only one Brazilian article was selected, from the State of Paraná 19. However, after reading the complete article it was excluded, since it turned out to be a cost analysis, or incomplete economic evaluation. The authors compared the costs of two strategies: not vaccinating and vaccinating. The non-vaccination strategy included direct medical costs with consultations, laboratory tests, serological diagnostic tests (IgM), fulminant hepatitis, transplants, and retransplantation. The vaccination strategy included the costs of vaccine doses and medical consultations due to adverse events. In order for the study to be considered a complete economic evaluation, in addition to costs it would have had to compare the health outcomes, for example hepatitis A cases averted.

After reading the full text of the 20 studies, 10 studies met the inclusion/exclusion criteria and will be discussed in the current review.

Methodological characteristics of the studies

Table 1 shows the methodological characteristics of the 10 studies 20,21,22,23,24,25,26,27,28,29 analyzed in the current review. Half of the studies were conducted in developed countries, the oldest 20 failed to specify the country but used U.S. data in its analysis, three studies were performed in the United States 21,22,23, and one in Israel 24. Four studies were conducted in South America, including two in Chile 25,26 and two in Argentina 27,28, and one study in China 29.

As specified by the authors, four studies performed cost-effectiveness analysis, three did cost-utility analysis, two did joint cost-effectiveness and cost-utility analysis, and only one did cost-benefit analysis.

As for the perspective from which the analysis was performed, the 10 studies adopted society's perspective, which is that recommended for vaccination programs planned to improve public health, because it considers both the direct and indirect costs of the intervention. Direct costs are those directly related to the disease and are divided into medical and non-medical. Direct medical costs are those immediately related to diagnosis, treatment, and rehabilitation. They include tests, medicines, clinical or surgical procedures, etc. Meanwhile, direct non-medical costs include family costs with transportation (of the patient and accompanying person for receiving care), food (changes in the patient's normal diet), etc. Indirect costs are related to loss of income and/or productivity and disability or death of productive persons as a result of the disease. Importantly, lost productivity is not limited to the patient; costs related to the persons (caregivers) who miss work in order to care for the patient should also be taken into account.

In addition to society's perspective, four studies 22,25,26,29 also took the perspectives of the healthcare system and public payer, which are relevant for assisting decision-making in the health sector.

As for the selected model, most of the studies (7 of 10) used a Markov static model 20,21,22,23,25,28,29. Two studies used dynamic models 26,27. Finally, one study was unable to develop a dynamic model (which the author considered the gold standard) and thus developed a "proxy model", not described clearly by the author 24.

The "dynamic" model allows projecting changes in transmission patterns, i.e., reduction in the force of infection resulting from the vaccination program. Universal vaccination programs can result in complex indirect effects known as "herd protection": unvaccinated susceptible individuals are protected from the disease by the decrease in transmission of the pathogen, which can lead to a shift in the age at infection to older age groups. Meanwhile, a "static" model does not allow incorporating change into the force of infection over time. Although theoretically the dynamic models estimate changes in the epidemiological pattern more accurately, they require larger amounts of data, which are not always available. For example, the lack of data on the annual decrease in the force of infection and seroprevalence for hepatitis A in some age groups lead authors to make assumptions that introduce uncertainties into the model. Despite the great methodological sophistication, the final results depend on the availability and quality of the data used to feed the model 26.

The inclusion of herd protection in the economic analysis can make the program more cost-effective, because it increases the number of cases averted. It was included in four studies 21,24,26,27 and partially included in one 25. Two of these studies 21,25 included herd protection in their analysis, despite using static models. The study by Armstrong assumed a fixed proportion in the decline in infections, attributed to herd protection for persons within the vaccinated cohort and for the unvaccinated 21. In Valenzuela et al. 25, it was not clear how this was done. It should be emphasized that static models usually fail to consider indirect effects of vaccination such as herd protection and shift in the age at infection to older age groups. Some studies using static models consider herd protection, assuming that a fixed percentage of the unvaccinated population is protected by the vaccination of the vaccinated cohort 11. However, another characteristic of herd protection, namely an increase in the age group of infection, can lead to an increase in the frequency of symptomatic cases and consequently a change (increase) in the use of health services, not considered in static models. The inclusion of only the positive aspects of herd protection can result in overestimation of the vaccination program's beneficial effects.

The inclusion of herd protection can have a major impact on the vaccination program's predicted outcomes. For example, Ginsberg et al. 24, when modeling the potential economic and health benefits of an immunization program for children 1-2 years of age in Israel, estimated that it would take 15 years for the program to reduce symptomatic cases by 90%. However, in real life, routine immunization in Israel achieved a reduction of greater than 90% in just three years, due partially to the herd protection induced by vaccination 6.

The majority of the studies (7 of 10) used long time horizons, more than 70 years, accompanying the life expectancy of the cohorts 20,21,22,23,26,27,29. They all used standard discount rates, from 3 to 5%, and only one study 27 failed to report the discount of benefits.

Estimates of key variables

• Epidemiological estimates

Table 2 summarizes the epidemiological estimates used in the economic evaluations.

All the studies used local data to build estimates of incidence and force of infection. Four studies 21,22,23,28 presented data on regional incidence and conducted regional and nationwide cost-effectiveness analyses. The study in China 29 conducted only regional analyses, without presenting a nationwide analysis.

The most frequently used outcomes were outpatient cases, inpatient cases, transplant cases, and deaths. To estimate the number of symptomatic cases, eight studies 21,22,23,25,26,27,28,29 used the study by Armstrong & Bell 30 which calculated the probability of developing jaundice during acute hepatitis A according to age group, utilizing data from different primary studies.

As for case-fatality, the majority of the studies used data from national studies or data furnished by the Ministry of Health to construct their local estimate, and only four studies in Argentina 27,28, Chile 26, and China 29 used U.S. case-fatality data.

• Characteristics of the vaccine

Table 3 presents the vaccine's characteristics. In relation to the vaccination scheme, most studies adopted vaccination of children up to 24 months of age. Only two studies evaluated the strategy with a second dose at 54 and 72 months of age 25,28. And only one Argentine study evaluated single-dose vaccination 28. Argentina is the only country to have implemented a single-dose immunization program against hepatitis A 8.

As for vaccination coverage, only three studies used more conservative coverage rates, from 69 to 89% 20,22,29.

All the studies used data on vaccine efficacy (percentage risk reduction in vaccinated individuals as observed in clinical trials). No study presented data on effectiveness (percentage risk reduction in vaccinated individuals as observed in routine use of the vaccine). Efficacy rates ranged from 91 to 98% for the first dose and from 95 to 100% for the second dose.

Data in the literature suggest that protection provided by the hepatitis A vaccine is long-lasting: persistence of antibodies or anamnestic response to a booster dose of the vaccine, or both, was demonstrated up to 12 years after the 2-dose vaccination scheme 31. Even so, the analyses in the 10 studies assumed waning of protection over time after vaccination. Five studies 20,21,22,23,24 used estimates based on models of antibody persistence 32,33. Three of these studies 25,28,29 used estimates constructed by an expert panel in the United States (1.62% per year in the first 10 years and 2.67% thereafter, after one dose of the vaccine, and 0.31% and 0.62% after two doses) 34. Two studies 26,27, despite using the same expert panel cited above as the reference 34 , presented an estimate of 0.58% per year, without specifying at what year after vaccination the waning of immunity was expected to occur.

• Cost estimates

Table 4 summarizes the elements considered in the cost estimates. All the studies included direct medical costs, costs with the vaccine, and indirect costs. In most cases, direct medical costs were related to treatment of hepatitis A in outpatients, inpatients, hepatic failure with or without subsequent transplant, and follow-up of transplant patients. All the studies used local data to estimate direct medical costs, except for Quezada et al. 26, in Chile, who used international studies, and Armstrong et al. 21, who did not specify the sources used to construct the direct cost estimates. The four studies 22,23,25,28 that included costs with follow-up of transplant patients used data from studies by Berge et al. 35 and/or Hauboldt 36, conducted in the United States.

As for costs with the vaccine, nine studies considered the values for the dose and administration, two studies included the cost of adverse events 24,26 , and only one included the cost of wasted vaccines and transportation to receive the vaccine 24.

As for indirect costs, eight studies calculated lost productivity due to the disease for patients, while four studies also calculated lost productivity for caregivers. One study calculated the wage losses of spouses, assuming that they acted as caregivers 24. Four studies 20,22,23,25 only used local data to estimate indirect costs, three 24,28,29 used international articles, and three 21,26,27 failed to specify their sources. In general, the calculation was performed using the country's median wage, adjusted for the unemployment rate. Only one study 29 used per capita GDP to construct this estimate, due to the lack of a reliable national wage estimate.

The number of lost workdays varied from 13.25 days in mild to moderate cases up to a year and a half for persons with non-fatal fulminant hepatic failure with transplant. In hospital cases this figure ranged from 33 to 71 days. Referring to the same source 35, different authors 25,28,29,34 reported different estimates of lost workdays.

Results of the analyses

Table 5 shows the summary measures presented in the results of the analyses. In general, studies funded by industry reported more favorable ICERs than the independent studies. All the studies showed favorable cost-effectiveness results, below US$ 20,000 per quality-adjusted life year (QALY) gained 37, except for two independent studies 21,23 that showed ICERs of US$ 28,000 and US$ 32,000 per QALY gained, respectively. When the latter included herd protection in the analysis, the ICER dropped from US$ 32,000 to US$ 1,000 per QALY gained.

One study 20 found that the hepatitis A vaccine was within the accepted cost-effective range, using US$ 42,000 per year of life saved as the cutoff 38. Jacobs et al. 22 found the value of the hepatitis A vaccination program comparable to that of other immunization programs. Valenzuela et al. 25 and Zhuang et al. 29 used per capita GDP as the cost-effectiveness parameter. Rein et al. 23 found the hepatitis A immunization program cost-effective, since it was comparable to other public health interventions such as diabetes screening in patients with hypertension (US$ 34,000) and HIV screening in the general population (US$ 42,000). The other five authors 21,24,26,27,28 did not specify the cost-effectiveness criterion used when interpreting the results.

Universal childhood hepatitis A vaccination proved to be cost-saving (costs iess and is more effective than the non-vaccination strategy) in areas with higher incidence of the disease in Argentina 28 and the United States 22,23. In Ellis et al. 28, vaccination proved to be cost-saving in the Cuyo region (high endemicity), while in the South region (intermediate endemicity) the ICER was US$ 673 per QALY gained and in the Northeast region (low endemicity) the ICER was US$ 2,772 per QALY gained. Jacobs et al. 22 showed similar findings: the vaccination program proved to be cost-saving in regions with incidence rates > 200% and 100-199% of the national average, while in regions with incidence rates 50-99% of the national average the ICER was US$ 13,800 per QALY gained and with incidence rates < 50% of the national average the ICER was US$ 63,000 per QALY gained. In Rein et al. 23, vaccination proved to be cost-saving in regions 1 and 2 (with twice the mean national incidence), and in region 3 (with the country's mean incidence) the ICER was US$ 133,000 per QALY saved.

Meanwhile, in China 29, in areas with the lowest, low, intermediate, and high prevalence of hepatitis A antibodies, the immunization program proved to be cost-saving, but in areas with extremely high prevalence of hepatitis A antibodies, the ICER was US$ 277.

Countries described their regional incidence/prevalence rates as high, intermediate, or low in relation to the national average. However, the national average differs between countries. While the United States has low endemicity, Argentina, Chile, and China are in transition from high to intermediate or low endemicity 14. The severity of hepatitis A increases with age: jaundice is reported in only 7% of children less than four years of age with hepatitis A 30. Meanwhile, more than 70% of older children and adults are symptomatic, with clinical jaundice in 40 to 70% of cases 30,39. In countries with high endemicity, infection occurs in early childhood, and the vast majority of cases are asymptomatic. The transition from high to low endemicity can lead to an increase in age at infection, with an increase in symptomatic cases and thus greater utilization of health services, which would explain the higher cost-effectiveness ratio in areas with extremely high endemicity.

Sensitivity analyses (Table 1) showed that the parameters with the greatest impact on the results of economic evaluations were cost of the vaccine dose and administration, followed by incidence and medical costs of the disease. Four 26,27,28,29 of the 10 studies presented robust results in favor of vaccination, which did not change with variations in the parameters for sensitivity analyses.



This systematic review of economic studies on childhood hepatitis A vaccination showed that this type of study can contribute to the development of a national decision-making model. The review pointed to some important issues to be considered when using the methodology and estimates of key variables in the published studies, especially in relation to the transferability of data and generalizability of results.

We observed variability among the studies under review in relation to the methodological characteristics and estimates of key variables. Methodological characteristics accounted for some differences in the studies' results, particularly the perspective and type of model used.

The type of perspective adopted by the study is a methodological choice. Many experts recommend society's perspective, because it is more comprehensive 15. In this review, when three studies 22,23,25 took society's perspective, the results were very different from those obtained using the healthcare system's perspective (Table 5). Although all the studies took society's perspective (considering direct and indirect costs), in the Brazilian case it is also recommendable to conduct the study from the healthcare system's perspective, because it provides results that only consider the amounts reimbursed by the Unified National Health System and can assist public managers in the decision making process.

As for the type of model, although most of the studies (7 of 10) used a Markov static model 20,21,22,23,25,28,29, the review suggests the development of a dynamic transmission model as the best option for evaluating the country's universal childhood immunization program. This methodological choice will require a larger amount of local data, for example data on hepatitis A seroprevalence in Brazil. Data from the recent National Hepatitis Survey allow feeding a dynamic model for hepatitis A.

The Brazilian dynamic model should consider both the positive aspects of herd protection such as the increase in averted cases of the disease and the negative aspects like the change in age at infection and proportion of symptomatic cases and use of health services, in order to avoid overestimating the beneficial effects of the vaccination program with more favorable cost-effectiveness results.

Estimates of the key variables also accounted for differences in the results in the studies reviewed here.

Incidence was one of the factors with the greatest impact on the cost-effectiveness of universal hepatitis A vaccination programs. Cost-effectiveness ratios in areas with low incidence of the infection are heavily influenced by the cost of the vaccine and the inclusion of social costs. Cost-effectiveness in high-incidence areas proved more robust to changes in the cost of the vaccine or other costs 12. Data from epidemiological estimates are considered to have low transferability in economic evaluations. Measures of clinical events can be misleading if there are differences in the underlying epidemiological estimates. The "guidelines" recommend the use of specific country data 40. In the absence of local data, it is recommended that data from countries with similar endemicity be used. Despite the existence of studies in Chile 25,26 and Argentina 27,28 with similar epidemiological profiles, the results are not transferable due to the differences in the organization of healthcare and coverage in the respective health systems.

In the Brazilian case, the preference should be for regional analyses based on different patterns of endemicity, in addition to a nationwide analysis. To conduct a nationwide analysis, the most important issue would be the availability of reliable data on incidence of the infection, number of symptomatic cases, and deaths from hepatitis A. The data on probability of symptomatic infection (jaundice) may be transferable from other studies, because in this specific case there is no reason for the natural history of the disease to differ between countries. Studies in Argentina 27,28, Chile 25,26, and China 29 used the proportion of symptomatic individuals among the infected, provided by Armstrong & Bell 30, a study conducted in the United States. The Brazilian national study could repeat this approach. As for case-fatality, although some studies in the review used international data, the Brazilian study should use national data. Guidelines suggest that this estimate should only be transferred from similar locations in the absence of data on case-fatality in the country itself, but that the estimate should be validated by expert panels.

In addition, detailed data are necessary on access to health services, healthcare patterns, and use of resources in outpatient, inpatient, and transplant cases, as well as on post-transplant follow-up. The studies reviewed here do not provide details on the methodology used for the identification and measurement of resources used in treatment of the previously cited cases. Data on use of resources are considered to have intermediate transferability. The guidelines from pharmacoeconomics recommend obtaining data on the use of resources from the location where the evaluation is being performed. Differences in clinical practices, payment systems, and incentives are generally cited as the principal reasons for variation in the use of resources between one location and another. When there are no local data and it becomes necessary to transfer data from elsewhere, an expert panel is recommended for reviewing the data and evaluating whether the treatment patterns are similar, in order to produce more relevant estimates for the local context 40. The Brazilian case requires obtaining the pattern of care practiced in local health services in order to determine the resources (medical consultations, hospitalizations, tests, drugs, etc.) and the amounts used in outpatient, inpatient, and transplant cases and post-transplant follow-up.

Most studies (8 of 10) used local data to estimate direct medical costs. Data from cost estimates are considered to have low transferability. The guidelines are very strict and do not allow transferring these data under any circumstances. Cost unit estimates must be specific to the context being evaluated, due to differences in absolute and relative prices between countries 40. Thus, in the Brazilian case, direct medical costs for outpatient and inpatient treatment of hepatitis A, hepatic failure with or without subsequent liver transplant, and post-transplant follow-up, as well as the costs of the vaccine, must necessarily use local data.

The inclusion of indirect costs related to lost productivity was important in determining the attractiveness of the immunization program (Table 5). Value assessment of lost work time depends on the country's wage and employment levels. Four studies 20,22,23,25 used only local data to estimate indirect costs, indicating greater difficulty in obtaining local data for this estimate. In the Brazilian national study, additional efforts should be made to accurately document the real number of lost workdays related to the disease locally, since indirect costs of the disease can have an important impact on the results of the analysis.

Data on the effect of treatment are considered highly transferable 40. The vaccine's efficacy, obtained from randomized controlled clinical trials conducted in other countries, can be used in the local study. The vaccination scheme should be adjusted to the prevailing national childhood immunization calendar, with two doses administered in the second year of life. Considering resource limitations, single-dose vaccination may be explored, as in the Argentine experience. Vaccination coverage should be estimated, based on the coverage of other vaccines already incorporated into the National Immunization Program and which are administered during the same period (measles-mumps-rubella, administered at 12 months, or the DTP booster, administered at 15 months).


Summary of Recommendations for a Brazilian National Model

Based on the information obtained from the articles in this review, it is possible to summarize some methodological characteristics and estimates of key variables to be used in the Brazilian national model:

• Adopt the society and healthcare system perspectives;

• Develop a dynamic transmission model;

• In addition to the national analysis, conduct regional analyses according to different patterns of endemicity;

• Use national data on incidence, force of infection, and case-fatality;

• Use national data on access to health services, patterns of care, and use of resources in outpatient, inpatient, and transplant cases;

• As a comparative strategy, adopt a vaccination scheme adjusted to the prevailing national childhood immunization calendar;

• Use local data to estimate direct medical costs;

• Use local data on coverage and cost of the vaccine;

• Consider international data on vaccine efficacy if local data on effectiveness are not available;

• Use local data to estimate indirect costs related to lost productivity;

• Discount costs and benefits at a 5% standard rate;

• Conduct sensitivity analysis for the estimates of incidence, cost of dose, and administration of the vaccine and medical costs of the disease.


Final remarks

Economic evaluations can benefit from systematic reviews of the clinical literature. However, prior review of the literature is not sufficient to guarantee a good economic evaluation 41. Some authors have questioned the contributions of systematic reviews and raised concerns about generalizing the results of economic evaluation studies. Unlike systematic reviews of treatment effects in randomized controlled clinical trials, the cost-effectiveness results of new interventions are more context-dependent than the clinical efficacy results. The cost-effectiveness of a particular intervention is heavily influenced by the use of resources and the costs associated with the intervention, which vary between countries, regions, or types of services, as well as over time 42,43,44.

Economic evaluations in health make use of epidemiological data and data on access and use of health services, treatment patterns, and costs associated with the disease, which vary greatly between countries, making it very difficult to generalize and/or transfer the results to different contexts. The systematic review of economic evaluation studies for the hepatitis A vaccine showed that it is not possible to generalize the results to the Brazilian context, and confirmed the need to develop a local study to estimate the cost-effectiveness ratio of the new vaccine to be incorporated.

The discussion of methodological characteristics and estimates of key variables and the summary of recommendations provided elements for the development of a new decision-making model for Brazil.



P. C. De Soárez participated in the design and planning, data analysis and interpretation, elaboration of the draft, and approval of the final version of the article. A. M. C. Sartori participated in the design and planning, elaboration of the draft, critical review of the content, and approval of the final version of the article. A. Santos participated in the design and planning, critical review of the content, and approval of the final version of the article. A. Itria participated in the data analysis and interpretation, critical review of the content, and approval of the final version of the article. H. M. D. Novaes participated in the design and planning, critical review of the content, and approval of the final version of the article. C. M. T. Martelli participated in the design and planning, critical review of the content, and approval of the final version of the article.



The authors are research members from the National Institute of Science and Technology for Health Technology Assessment (IATS). This research was sponsored by CNPq, IATS, and the authors' institutions. C. M. T. Martelli received resarch scholarship (CNPq # 306489/2010-4)



1. Anderson R. Systematic reviews of economic evaluations: utility or futility? Health Econ 2010; 19:350-64.         

2. Nixon J, Khan KS, Kleijnen J. Summarising economic evaluations in systematic reviews: a new approach. BMJ 2001; 322:1596-8.         

3. Centre for Reviews and Disseminations, University of York. Systematic reviews. CRD's guidance for undertaking reviews in health care. York: Centre for Reviews and Disseminations, University of York; 2008.         

4. Shemilt I, Mugford M, Byford S, Drummond M, Eisenstein E, Knapp M, et al. Chapter 15: incorporating economic evidence. In: Higgins J, Green S, editors. Cochrane handbook for systematic reviews of interventions, Version 5.0.0. The Cochrane Collaboration, 2008.         

5. Nothdurft HD. Hepatitis A vaccines. Expert Rev Vaccines 2008; 7:535-45.         

6. Dagan R, Leventhal A, Anis E, Slater P, Ashur Y, Shouval D. Incidence of hepatitis A in Israel following universal immunization of toddlers. JAMA 2005; 294:202-10.         

7. Chodick G, Heymann AD, Ashkenazi S, Kokia E, Shalev V. Long-term trends in hepatitis A incidence following the inclusion of hepatitis A vaccine in the routine nationwide immunization program. J Viral Hepat 2008; 15 Suppl 2:62-5.         

8. Vacchino MN. Incidence of hepatitis A in Argentina after vaccination. J Viral Hepat 2008; 15 Suppl 2:47-50.         

9. Zhou F, Shefer A, Weinbaum C, McCauley M, Kong Y. Impact of hepatitis A vaccination on health care utilization in the United States, 1996-2004. Vaccine 2007; 25:3581-7.         

10. Hendrickx G, Van Herck K, Vorsters A, Wiersma S, Shapiro C, Andrus JK, et al. Has the time come to control hepatitis A globally? Matching prevention to the changing epidemiology. J Viral Hepat 2008; 15 Suppl 2:1-15.         

11. Bos JM, Alphen L, Postma MJ. The use of modeling in the economic evaluation of vaccines. Expert Rev Pharmacoecon Outcomes Res 2002; 2:443-55.         

12. Anonychuk AM, Tricco AC, Bauch CT, Pham B, Gilca V, Duval B, et al. Cost-effectiveness analyses of hepatitis A vaccine: a systematic review to explore the effect of methodological quality on the economic attractiveness of vaccination strategies. Pharmacoeconomics 2008; 26:17-32.         

13. de Alencar Ximenes RA, Martelli CM, Merchán-Hamann E, Montarroyos UR, Braga MC, de Lima ML, et al. Multilevel analysis of hepatitis A infection in children and adolescents: a household survey in the Northeast and Central-west regions of Brazil. Int J Epidemiol 2008; 37:852-61.         

14. Jacobsen K. The global prevalence of hepatitis A virus infection and susceptibility: a systematic review. Geneva: World Health Organization; 2009.         

15. Pignone M, Saha S, Hoerger T, Lohr KN, Teutsch S, Mandelblatt J. Challenges in systematic reviews of economic analyses. Ann Intern Med 2005; 142(12 Pt 2):1073-9.         

16. Hepatitis B vaccines. Wkly Epidemiol Rec 2009; 84:405-19.         

17. Organization for Economic Co-operation and Development. Purchasing power parities (PPP). (accessed on 20/Sep/2011).         

18. US Department of Labor. US labor statistics data. Consumer Price Index.         

19. Zahdi MR, Maluf I, Maluf EM. Hepatitis A: the costs and benefits of the disease prevention by vaccine, Paraná, Brazil. Braz J Infect Dis 2009; 13:257-61.         

20. Das A. An economic analysis of different strategies of immunization against hepatitis A virus in developed countries. Hepatology 1999; 29:548-52.         

21. Armstrong GL, Billah K, Rein DB, Hicks KA, Wirth KE, Bell BP. The economics of routine childhood hepatitis A immunization in the United States: the impact of herd immunity. Pediatrics 2007; 119:e22-9.         

22. Jacobs RJ, Greenberg DP, Koff RS, Saab S, Meyerhoff AS. Regional variation in the cost effectiveness of childhood hepatitis A immunization. Pediatr Infect Dis J 2003; 22:904-14.         

23. Rein DB, Hicks KA, Wirth KE, Billah K, Finelli L, Fiore AE, et al. Cost-effectiveness of routine childhood vaccination for hepatitis A in the United States. Pediatrics 2007; 119:e12-21.         

24. Ginsber GM, Slater PE, Shouval D. Cost-benefit analysis of a nationwide infant immunization programme against hepatitis A in an area of intermediate endemicity. J Hepatol 2001; 34:92-9.         

25. Valenzuela MT, Jacobs RJ, Arteaga O, Navarrete MS, Meyerhoff AS, Innis BL. Cost-effectiveness of universal childhood hepatitis A vaccination in Chile. Vaccine 2005; 23:4110-9.         

26. Quezada A, Baron-Papillon F, Coudeville L, Maggi L. Universal vaccination of children against hepatitis A in Chile: a cost-effectiveness study. Rev Panam Salud Pública 2008; 23:303-12.         

27. Lopez E, Debbag R, Coudeville L, Baron-Papillon F, Armoni J. The cost-effectiveness of universal vaccination of children against hepatitis A in Argentina: results of a dynamic health-economic analysis. J Gastroenterol 2007; 42:152-60.         

28. Ellis A, Rüttimann RW, Jacobs RJ, Meyerhoff AS, Innis BL. Cost-effectiveness of childhood hepatitis A vaccination in Argentina: a second dose is warranted. Rev Panam Salud Pública 2007; 21:345-56.         

29. Zhuang GH, Pan XJ, Wang XL. A cost-effectiveness analysis of universal childhood hepatitis A vaccination in China. Vaccine 2008; 26:4608-16.         

30. Armstrong GL, Bell BP. Hepatitis A virus infections in the United States: model-based estimates and implications for childhood immunization. Pediatrics 2002; 109:839-45.         

31. Van Damme P, Banatvala J, Fay O, Iwarson S, McMahon B, Van Herck K, et al. Hepatitis A booster vaccination: is there a need? Lancet 2003; 362:1065-71.         

32. Van Damme P, Thoelen S, Cramm M, De Groote K, Safary A, Meheus A. Inactivated hepatitis A vaccine: reactogenicity, immunogenicity, and long-term antibody persistence. J Med Virol 1994; 44:446-51.         

33. Wiedermann G, Ambrosch F, André FE, D'Hondt E, Delem A, Safary A. Persistence of vaccine-induced antibody to hepatitis A virus. Vaccine 1992; 10 Suppl 1:S129-31.         

34. Jacobs RJ, Margolis HS, Coleman PJ. The cost-effectiveness of adolescent hepatitis A vaccination in states with the highest disease rates. Arch Pediatr Adolesc Med 2000; 154:763-70.         

35. Berge JJ, Drennan DP, Jacobs RJ, Jakins A, Meyerhoff AS, Stubblefield W, et al. The cost of hepatitis A infections in American adolescents and adults in 1997. Hepatology 2000; 31:469-73.         

36. Hauboldt R. Cost implications of human organ and tissue transplantation, an update. Seattle: Milliman & Robertson; 1999.         

37. Laupacis A, Feeny D, Detsky AS, Tugwell PX. How attractive does a new technology have to be to warrant adoption and utilization? Tentative guidelines for using clinical and economic evaluations. CMAJ 1992; 146:473-81.         

38. Tengs TO, Adams ME, Pliskin JS, Safran DG, Siegel JE, Weinstein MC, et al. Five-hundred life-saving interventions and their cost-effectiveness. Risk Anal 1995; 15:369-90.         

39. Koslap-Petraco MB, Shub M, Judelsohn R. Hepatitis A: disease burden and current childhood vaccination strategies in the United States. J Pediatr Health Care 2008; 22:3-11.         

40. Barbieri M, Drummond M, Rutten F, Cook J, Glick HA, Lis J, et al. What do international pharmacoeconomic guidelines say about economic data transferability? Value Health 2010; 13:1028-37.         

41. Drummond MF, Iglesias CP, Cooper NJ. Systematic reviews and economic evaluations conducted for the National Institute for Health and Clinical Excellence in the United Kingdom: a game of two halves? Int J Technol Assess Health Care 2008; 24:146-50.         

42. Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S, Urdahl H, et al. Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess 2004; 8:iii-iv, 1-192.         

43. Welte R, Feenstra T, Jager H, Leidl R. A decision chart for assessing and improving the transferability of economic evaluation results between countries. Pharmacoeconomics 2004; 22:857-76.         

44. Drummond M, Pang F. Transferability of economic evaluations results. In: Drummond M, McGuire A, editors. Economic evaluation in health care: merging theory with practice. Oxford: Oxford University Press; 2001. p. 256-76.         



P. C. De Soárez
Departamento de Medicina Preventiva
Faculdade de Medicina
Universidade de São Paulo
Av. Dr. Arnaldo 455, 2º andar
São Paulo, SP 01246-903, Brasil

Submitted on 16/May/2011
Final version resubmitted on 01/Nov/2011
Approved on 21/Nov/2011

Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz Rio de Janeiro - RJ - Brazil