Sensitivity analysis for an unmeasured confounder: a review of two independent methods


Análise de sensibilidade para um confundidor não observado: uma revisão de dois métodos independentes



Ronir Raggio LuizI; Maria Deolinda Borges CabralII

IInstituto de Estudos de Saúde Coletiva da Universidade Federal do Rio de Janeiro
IIInstituto Brasileiro de Geografia e Estatística (IBGE)





Um dos principais objetivos dos estudos epidemiológicos é a estimação de efeitos causais. E a inferência causal deve ser discutida tanto por estudos experimentais quanto por estudos observacionais. Uma importante limitação na interpretação causal de estudos observacionais é a possível presença de confundidores não observados (hidden bias). Uma estratégia para avaliar o possível efeito de um confundidor não observado é através de uma análise de sensibilidade, que determina quão forte deveriam ser os efeitos de um confundidor não observado de modo a explicar uma aparente associação. A proposta deste artigo é rever e integrar dois métodos independentes de análise de sensibilidade: um devido a Greenland, sob uma perspectiva epidemiológica; e outro devido a Rosenbaum, sob uma perspectiva estatística. A combinação de questões estatísticas e epidemiológicas torna a análise de sensibilidade mais completa e direta, e deve estimular sua necessária difusão e mais aplicações. Como os estudos observacionais são mais sujeitos a vieses e confundimento, uma análise de sensibilidade considerando aspectos estatísticos e epidemiológicos fortalece o processo de inferência causal.

Palavras-chave: Análise de sensibilidade. Confundidor não-observado. Confundimento. Estudos observacionais.


One of the main purposes of epidemiological studies is to estimate causal effects. Causal inference should be addressed by observational and experimental studies. A strong constraint for the interpretation of observational studies is the possible presence of unobserved confounders (hidden biases). An approach for assessing the possible effects of unobserved confounders may be drawn up through the use of a sensitivity analysis that determines how strong the effects of an unmeasured confounder should be to explain an apparent association, and which should be the characteristics of this confounder to exhibit such an effect. The purpose of this paper is to review and integrate two independent sensitivity analysis methods. The two methods are presented to assess the impact of an unmeasured confounder variable: one developed by Greenland under an epidemiological perspective, and the other developed from a statistical standpoint by Rosenbaum. By combining (or merging) epidemiological and statistical issues, this integration became a more complete and direct sensitivity analysis, encouraging its required diffusion and additional applications. As observational studies are more subject to biases and confounding than experimental settings, the consideration of epidemiological and statistical aspects in sensitivity analysis strengthens the causal inference.

Keywords: Sensitivity analysis. Unmeasured confounder. Confounding. Observational studies.




One of the main purposes of clinical or epidemiological studies is to estimate causal effects. Causal inference should be addressed by experimental and observational studies. However, observational studies are more subject to biases and confounding factors than experimental studies. A strong constraint for the interpretation of observational studies is the potential (possible) presence of unobserved confounders (hidden biases or unmeasured confounding), namely variables that were not observed owing to unavailable data or even lack of knowledge, because such omissions may distort the existence or magnitude of the association of interest.

Confounding is an epidemiological phenomenon that causes a bias in the estimation of the causal association of interest, characterized by the lack of comparability between exposed (or treated) and unexposed (or untreated) subject. This can be detected in terms of the variables that have a causal relationship with the outcome, the association (causal or not) with the exposure (or treatment), and the association between exposure and outcome in which they are not intermediate. The level of confounding is a joint function of the prevalence of the exposure, the confounder, the association between the confounder and outcome, and the association between the confounder and exposure; however, none of these terms are open to consideration individually if the purpose is a quantitative construct1.

In analyses of observational studies, the measured potential confounders (overt bias) are "adjusted" analytically using statistical techniques, such as stratification, modeling, etc. Quantitative techniques are used infrequently to determine the potential influence of unmeasured confounders. Most of the statistical methods used in clinical or epidemiological studies focus on the assessment of random errors and confounders measured during the data generation process, which are often only a fraction of the total error and rarely the only important source of uncertainty when estimating a causal effect measurement2. Consequently, the development of statistical tools that are also appropriate for unmeasured confounder situations that allow such errors to be assessed, has become important. Besides, in nonrandomized intervention studies, quantitative assessment of unobserved confounding should be mandatory3 if the purpose is to discuss causal inference.

An approach for assessing the possible effects of unmeasured confounders may be drawn up through the use of a sensitivity analysis that determines how strong the effects of an unmeasured confounder should be to explain an apparent association, and the characteristics of this confounder, which can produce such an effect. The purpose of this paper is to review and integrate two independent sensitivity analysis methods - one developed by Greenland2 and the other developed from a statistical standpoint by Rosenbaum4. These methods focus on assessing the impact of an unmeasured confounder variable by maintaining the analysis as simple as possible and viable for healthcare workers and researchers. Nevertheless, nowadays, more advanced integration strategies and sophisticated methods are also available5-9.


Rationale of Sensitivity Analysis for an Unmeasured Confounder

The sensitivity analysis of an unmeasured confounder is a statistical technique, which assesses how the inferences on the effects of exposures can be altered through an unmeasured variable, under various simulations. The sensitivity analysis is a quantitative extension of the qualitative considerations that characterize a proper discussion regarding the findings of a study2. The process of addressing biases quantitatively is called bias analysis10, and an analysis that examines how strong the effects of an unmeasured confounder should be to explain an apparent association and what the characteristics of this confounder should be is called target adjustment sensitivity analysis11.

Cornfield et al12 were the first to formally establish a sensitivity analysis for an unmeasured confounder in an observational study examining the association between smoking and lung cancer. In this study, they asked whether the association found between smoking and lung cancer arose from an effect that was actually caused by smoking, or whether this was owing to a bias originating from an unmeasured confounder variable. Can cancer be prevented if there was no exposure to cigarette smoke? Or are the high rates of lung cancer found among smokers owing to some inherent difference between smokers and nonsmokers? In their efforts to analyze this issue, they derived an inequality for the apparent risk of death caused by lung cancer, based on a risk ratio defined as the ratio of the probability of death owing to lung cancer among smokers by the probability of death owing to lung cancer among nonsmokers. More specifically, Cornfield's condition can be stated as follows: "If an agent, A, with no causal effect upon the risk of a disease, nevertheless, because of a positive correlation with some other causal agent, B, shows an apparent risk, r, for those exposed to A, relative to those not so exposed, then the prevalence of B, among those exposed to A, relative to the prevalence among those not so exposed, must be greater than r."12 Thus, if cigarette smokers have nine times the risk of nonsmokers for developing lung cancer, and if this is not because cigarette smoke is a causal agent, but only because cigarette smokers produce hormone X, then the proportion of hormone-X-producers among cigarette smokers must be at least nine times greater than that among nonsmokers. If the relative prevalence of hormone-X-producers is considerably less than nine-fold, then hormone X cannot account for the magnitude of the apparent effect. As shown in the following paragraph, this argument is the basis of the Rosenbaum sensitivity analysis method.

Based on the work by Cornfield et al, sensitivity analysis became an area increasingly explored in the literature, particularly by Bross13, Gail et al14, Rosenbaum15,16, and Lin et al17. The other recent contributions and developments comprise the work carried out by Fewel et al18, McCandless et al19, Arah et al20, and Rothman et al10. More formally, Rosenbaum4 and Greenland2 developed two sensitivity analysis methods applied to dichotomous variables, which allow for the analyses of the changes in the findings obtained in a study.

In fact, this strategy reflects an important conceptual advance in terms of uncertainty in observational studies. The progress consists of complementing a general qualitative assessment that is applied to all observational studies through a quantitative assessment that is specific to what is noted in the study of interest. Instead of affirming that an association between an exposure and a given outcome does not imply causation, or that unmeasured biases might explain the observed associations, it can be stated that to explain the association found in a specific study, a bias of a specific magnitude is required. If the spurious association is strong, then a large magnitude of unmeasured bias is required. This indicates that the magnitude of the required association between an unmeasured confounder and the exposure of interest may be quantified to alter the conclusions reached through a clinical or epidemiological study, suggesting a cause-and-effect relationship.

Although sensitivity analysis is a topic that has been under discussion since the seminal ideas developed by Cornified et al12 and the paper presented by Greenland2, its application to the validation of the findings obtained in observational studies is still very limited. Modan et al21, Margolis et al22, and Cabral and Luiz23 are a few who carried out health-related studies. Groenwold et al24 used a different strategy to quantify the potential role of unmeasured confounders.


The Greenland Method

When noting an association between a certain exposure and a specific outcome, an important question that should be asked is how to quantitatively assess the potential for the association observed owing to the presence of an unmeasured confounder variable. The Greenland method2 strives to answer this question, by speculating about the magnitude of the association between the unmeasured confounder variable with the exposure and the outcome required to deny the association previously found between the exposure and the outcome of interest.

To carry out this, at least two items of information that are not directly observed in the available data are required:

• the confounder prevalence by exposure level specifically for individuals not presenting the outcome, expressing the association between the confounder and exposure variable; and

• the magnitude of the association between the confounder and the outcome.

In fact, these magnitudes are not directly observed in the study, and may be "obtained" only through speculations based on other sources of information (surveys, literature, systematic reviews, etc.). By considering the various plausible values for these magnitudes, the methods estimate the strength of the association between the exposure and the outcome, adjusted by the confounder variable specified for each combination taken into account. Thus, the method allows an assessment of the variations in the association of interest observed, considering a range of plausible values of the association between the unmeasured confounder and the exposure variable.

On observing the notations found in the paper by Greenland2, it is assumed that an observational study is conducted to assess the association of a certain exposure variable (E) and an outcome (D) of interest in the presence of a potential unmeasured confounder (Z), with these three variables (E, D, and Z) being dichotomous. The data available from the study are assumed to have been arrayed as presented in Table 1. It is also believed that the data do not present any confounding for the covariates measured in the study, or have already been adjusted for the measured confounders. With no loss of generality, it is presumed that the measurement of interest is the Odds Ratio (OR).



For sensitivity analysis, the data given in Table 1 are virtually stratified by Z, the dichotomous unmeasured confounder variable, as shown in Table 2.

By assuming a common OR for associating the exposure to the disease, or by considering that there is no modification of the effect of E on the outcome owing to the unmeasured confounder variable Z, it appears from Table 2 that the OR between the exposure and the outcome, adjusted by this variable, can be given by:

It is assumed that the prevalence of Z for individuals not presenting the outcome - among the exposed and non exposed populations to exposure level E - is known or assumed to be PZ1 and PZ0, respectively, by considering individuals who do not present the outcome as representing the reference population. Thus, B11 = PZ1B1+ and B01 = PZ0B0+ . Now, one can assume that the OR associating the confounder to the outcome is the same for each exposure level E, and is given by ORDZ. As the values of B11 and B01 have already been obtained, the next step is to determine the values for A11 and A01. To carry out this, the following pair of equations should be solved:

with the following solutions2

Having obtained the values for A11, A01, B11, and B01, Table 2 can be completed by directly calculating the OR for the Exposure/Outcome ORDE for each of the Z strata. The estimated ORDE is said to be "adjusted indirectly by Z," because it is the estimate that would be obtained if data on Z were observed, and which presented the characteristics assumed for the calculations. The term suggested by Greenland2 for the ORDE estimate is "adjusted externally" because it uses the ORDZ obtained externally from the study data.

To offer a numerical illustration of the use of this method, the data for a hypothetical observational study are presented in Table 3, covering an association between exposure to a factor E and a given outcome of interest D, in the presence of an unmeasured confounder variable Z. Moreover, it is also assumed that these data do not need adjustment for the measured variables. The data summarized in Table 3 present an unadjusted ORDE of 2.57.



The findings for the sensitivity analysis (Table 4) were obtained by applying the Rosenbaum method to the hypothetical data presented in Table 3, by taking 5, 10, 15, or 50 as the "plausible" values for the OR between the confounder and the outcome, which already represent strong associations (probably not reproducing the most authentic clinical situations) between the confounder and the outcome, and by considering a broad-ranging set for the confounder prevalences varying between 0.10 and 0.90, with a set of values for the prevalences between the non exposed defined in a manner whereby the OR of the confounder with the exposure reaches the values of 1.5, 2, or 3. These results can be obtained by formulas given in the work by Greenland2 or Greenland et al10, and easily implemented in a spreadsheet25.

The sensitivity analysis applied in this hypothetical study shows that the OR of interest, ORDE, "adjusted" by the unmeasured confounder and based on the various specifications considered "plausible" for this confounder, varies between 1.17 and 2.48 (Table 4). The association observed appears to be more sensitive to an unmeasured confounder, indicating that its value "adjusted" by the confounder moves further away from the unadjusted value when the confounder has an OR of at least 3 for the exposure variable and an OR of at least 15 for the outcome of interest. Depending on the study of interest, a reasonable conclusion would be that it is very likely that a confounder of this magnitude is not taken into consideration, suggesting that the hypothetical association observed proved to be "insensitive" to an unmeasured confounder.


The Rosenbaum Method

The Rosenbaum method is based on the following: many statistical procedures are derived as the consequence of randomization in clinical trials. In a simple randomized trial, all individuals are equally likely to receive the treatment, because they receive the treatment on a random basis. In contrast, in observational studies, some individuals are more likely to receive the treatment than others, because the allocation of the treatment depends on individual characteristics, some of which could not have been measured. Let us assume an observational study in which each unit is associated with a covariate vector, X, that contains the measured covariates of interest for that study. If the study is free of unmeasured confounding, then the probability of an individual being exposed, although unknown, depends only on this covariate vector. Thus, two units with the same observed covariate X present equal probabilities of exposure. According to Rosenbaum4, in an observational study free from unmeasured confounding, a stratification of X leads to a distribution of allocation to exposure, similar to that obtained in a uniform randomized experimental trial, indicating that a study with a probability of allocation to exposure given by the stratification is known and constant throughout all possible findings, although with unknown parameters. This process was called "covariates vector randomization" by Rubin26.

When considering an observational study with the presence of an unmeasured confounding, two units with the same observed covariate X may present different odds for exposure. This is because the odds of a unit being exposed do not depend only on the observed covariates vector, but also on an unknown parameter associated with an unmeasured confounder variable. The main idea behind the Rosenbaum method is to establish a quantitative measurement of the difference between randomized trials and observational studies. This is carried out using the parameter Gamma (Γ).

The unknown Γ parameter measures the extent to which a study differs from a randomized trial, that is, the number of units that may differ in their odds of receiving the treatment. This is a measurement of the extent to which the odds of receiving the treatment may vary from person to person. In randomized trials, all individuals have the same odds of receiving the treatment Γ = 1, and hence, they do not differ. A study with Γ = 2 is very different from a randomized trial, because one person may be twice more likely to receive treatment than the other, owing to unmeasured characteristics. An observational study is said to be insensitive to an unmeasured confounding when the conclusions of the study are essentially the same, even when Γ is not close to 1, that is, even when people differ in their odds of receiving the treatment.

According to Rosenbaum4, when using stratification on X in an observational study with hidden bias (unmeasured confounding), there is at least one Γ > 1 value, whereby:

for units with the same X covariates vector.

In contrast to the Greenland method that considers the classic confounding scheme, the Rosenbaum method works only with the association between confounder and exposure. It determines the magnitude of the association between the unmeasured confounder variable and the exposure, sufficient for this confounder variable to be responsible for the association found between the exposure and the outcome of interest, implicitly assuming that the confounder is a "quasi-perfect" outcome predictor, showing that the magnitude of the association between the confounder and the outcome is sufficient for the confounding to depend only on the association between this confounder and the exposure variable.

A technique that is widely used for analyzing the findings of observational studies is stratification. When considering stratification at a data-analysis stage, Rosenbaum proposed to apply a sensitivity analysis technique in which the measurement of the association to be used is the OR given by the Mantel-Haenszel statistic T, based on the number of individuals exposed and presenting the outcome. This procedure is used frequently in analyses considering a third variable that may "mask" the association found between the exposure and the outcome of interest27. The procedure for conducting the sensitivity analysis suggested by Rosenbaum is applied to each stratum, considering the expected T value, and establishing all margins observed for those exposed and those presenting the outcome; under a null hypothesis (H0), that exposure would have no effect on the outcome of interest. The calculation of the T expectation and variance values is carried out under H0, by combining the expectation and variance values obtained in each stratum through a normal approximation of the distribution of statistics, whose formulas consider the unknown λ parameter through Γ, as developed by Stevens28.

Under the null hypothesis stating that there are no effects of exposure on the outcome of interest, the Rosenbaum method considers many possible values for Γ and determines the lowest Γ value that makes the association between E and D statistically nil.

To illustrate the application of the method, the data presented in Table 3 are used again. By establishing Γ = 1.0; 1.5; 1.8; 1.9; 2.0 and 3.0, the p-values of the upper and lower limits obtained through the Rosenbaum method are given in Table 5. These calculations are not easy to obtain manually, but they can be promptly executed using an electronic spreadsheet, as can be found in the study carried out by Cabral and Luiz25.



The lowest value for Γ to remove all statistical significance from Mantel-Haenszel is Γ = 1.9, at a 95% confidence level. Thus, according to the Rosenbaum method, the association between exposure and outcome proves "sensitive" to an unmeasured confounder that increases the odds of exposure of the units by 90% or more.


Proposed Integration of the Greenland and Rosenbaum Methods

As may be noted in the presentation of the two methods, Rosenbaum developed his analysis on the basis of criteria that are strictly statistical, while Greenland concentrated more on the standpoint of the epidemiologist in the proposed analysis. Rosenbaum did not discuss whether the unmeasured variable causes confounding or interaction in the findings, because he was interested in determining whether the value obtained is statistically significant or not. On the other hand, Greenland was concerned about the epidemiological construct of the findings, and hence, proposed an analysis that takes the existence of confounding into consideration, and does not apply the hypothesis of interaction. As the considerations of both models are important when assessing the potential effect of an unmeasured covariate in an observational study, it has been proposed to merge them to perform a sensitivity analysis that integrates these two approaches.

As a way of integrating these two methods, it is suggested that the findings of the application of the Rosenbaum model should be taken as the starting point for the use of the Greenland method. Rosenbaum presented a minimum Γ value that makes the association between exposure and outcome statistically insignificant, but did not estimate the magnitude of this association. However, by using the Greenland approach and determining the values for the association between the unmeasured confounder variable and the outcome that are equal to or greater than the lowest Γ value of the Rosenbaum method (OREZ -> Γ values), the magnitude of the "quasi-perfect" association between the unmeasured confounder (Z) and the outcome of interest (D) can be estimated (ORDZ) through speculations on this association. In addition, the variations in the "adjusted" estimates of the association of interest (ORDE) arising from the presence of confounders defined by the speculated values can also be estimated.

This indicates that the Rosenbaum method can be used to reduce and guide the speculations considered for the association of the confounder with the exposure variable (OREZ), by stipulating values for this association that are equal to or greater than the Γ value found (OREZ -> Γ values). In turn, the Greenland method may be used to calculate the variations in the ORDE estimates arising from the consideration of the various possible distributions of the confounder, which also determine the magnitude of the "quasi-perfect" association between the unmeasured confounder and the outcome of interest, which is implicit but not quantified through the Rosenbaum approach. Hence, this approach can be useful to reduce the methodological efforts to establish causal relations by considering unmeasured confounders.

In parallel, from a conceptual view to estimate the causal effects, but using a different method, MacLehose et al.29 developed similar ideas to complement standard sensitivity analyses.


Final Remarks

In the analysis of the findings of the study showing an association consistent with a causal hypothesis, the researcher should always discuss all possible explanations for these findings (including different hypotheses and biases in the findings), particularly in observational studies, where the association of interest may never possibly be measured with full accuracy, because in most cases, it would be unethical to conduct a study for this purpose. Even when it is not possible to lessen this uncertainty, it is possible to be careful in arguing its magnitude; that is, even when it is not possible to remove the bias through adjustments or avoid biases through careful designs, a sensitivity analysis may provide a quantitative expression of the level of uncertainty about the presence of a potential bias. Some descriptions of the possible unobserved predictive variables are generally desirable, and essential if they represent confounders.

If it is necessary to stratify for a measured confounder using the Greenland method, the calculations presented in the description of the method should be repeated for each stratum, and the findings should then be merged through this integrated approach. In this paper, we have assumed that only one (a single) unmeasured confounder is present, and all the variables are dichotomous. Moreover, multiple unmeasured confounders may certainly coexist and interact, altering the association of interest. These analyses are more complex, because they must consider the odds of the associations between each unmeasured confounder and both the predictor and the outcome, requiring more complex calculations that have not been covered in this paper. Propensity score calibration has been argued as a method of adjusting for multiple unmeasured confounders30. Thus, the important limitations of both the methods of sensitivity analysis for epidemiological purposes are that they consider only the presence of one unmeasured confounding, absence of effect modification, and the considerations of dichotomous exposures and outcomes.

A limitation of the Rosenbaum method is that the p-values are not of primary interest in observational studies to infer causality. However, they are very common in clinical literature and frequently misinterpreted. Hence, although possible scenarios for the Greenland method are based on current knowledge, common sense, and/or plausibility, a combination of clinical or epidemiological quantitative guesses and statistical significance in a sensitivity analysis can be useful and attractive.



1. Walker AM. Observation and inference: an introduction to the methods of epidemiology. Newton Lower Falls: Epidemiology Resources Inc.; 1991.         

2. Greenland S. Basic Methods for sensitivity analysis of biases. Int J Epidemiol 1996; 25: 1107-16.         

3. Groenwold RH, Hak E, Hoes AW. Quantitative assessment of unobserved confounding is mandatory in nonrandomized intervention studies. J Clin Epidemiol 2009; 62: 22-8        

4. Rosenbaum PR. Observational Studies. New York: Spring-Verlag; 1995.         

5. Greenland S. The impact of prior distributions for uncontrolled confounding and response bias: a case study of the relation of wire codes and magnetic fields to childhood leukemia. J Am Stat Assoc 2003; 98: 47-54.         

6. Brumback BA, Hernán MA, Haneuse SJ, Robins JM. Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Stat Med 2004; 23: 749-67.         

7. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf 2006; 15: 291-303.         

8. Stürmer T, Glynn RJ, Rothman KJ, Avorn J, Schneeweiss S. Adjustments for unmeasured confounders in pharmacoepidemiologic database studies using external information. Med Care 2007; 45(10S): S158-65.         

9. McCandless LC, Gustafson P, Levy AR. A sensitivity analysis using information about measured confounders yielded improved uncertainty assessments for unmeasured confounding. J Clin Epidemiol 2008; 61: 247-55.         

10. Rothman KJ, Greenland S, Lash TL. Modern Epidemiology. 3th ed. Philadelphia: Wolters Kluwer/Lippincott Williams & Wilkins; 2008.         

11. Phillips CV. Quantifying and reporting uncertainty from systematic errors. Epidemiology 2003; 14: 459-66.         

12. Cornfield J, Haenszel W, Hammond EC, Lilienfeld AM, Shimkin MB, Wynder EL. Smoking and lung cancer: Recent evidence and a discussion of some questions. J Nat Cancer Inst 1959; 22: 173-203.         

13. Bross IDJ. Pertinency of an extraneous variable. J Chronic Dis 1969; 20: 487-95.         

14. Gail MH, Wacholder S, Lubin JH. Indirect corrections for confounding under multiplicative and additive risk models. Am J Ind Med 1988; 13: 119-30.         

15. Rosenbaum PR. Discussing hidden bias in observational studies. Ann Intern Med 1991; 115: 901-5.         

16. Rosenbaum PR. Design sensitivity in observational studies. Biometrika 2004; 91: 153-64.         

17. Lin DY, Psaty BM, Kronmal RA. Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics 1998; 54: 948-63.         

18. Fewell Z, Davey Smith G, Sterne JA. The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study. Am J Epidemiol 2007; 166: 646-55.         

19. McCandless LC, Gustafson P, Levy A. Bayesian sensitivity analysis for unmeasured confounding in observational studies. Stat Med 2007; 26: 2331-47.         

20. Arah OA, Chiba Y, Greenland S. Bias formulas for external adjustment and sensitivity analysis of unmeasured confounders. Ann Epidemiol 2008; 18: 637-46.         

21. Modan B, Ron E, Lerner-Geva L, Blumstein T, Menczer J, Rabinovici, J et al. Cancer incidence in a cohort of infertile women. Am J Epidemiol 1998; 147: 1038-42.         

22. Margolis DJ, Berlin JA, Strom BL. A comparison of sensitivity analyses of the effect of wound duration on wound healing. J Clin Epidemiol 1999; 52: 123-8.         

23. Cabral MDB, Luiz RR. Use of sensitivity analysis to assess the effects on anti-hepatitis A virus antibodies of access to household water supply. Epidemiol Infect 2008; 136: 334-40.         

24. Groenwold RH, Hoes AW, Nichol KL, Hak E. Quantifying the potential role of unmeasured confounders: the example of influenza vaccination. Int J Epidemiol 2008b; 37: 1422-9.         

25. Cabral MDB, Luiz RR. Sensitivity analysis for unmeasured confounders using an electronic spreadsheet. Rev Saude Publica 2007; 41: 446-52.         

26. Rubin DB. Assignment to treatment group on the basis of a covariate. J Educ Statist 1977; 6: 34-58.         

27. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New York: Wiley; 1981.         

28. Stevens WL. Mean and variance of an entry in a contingence table. Biometrika 1951; 38: 468-70.         

29. MacLehose RF, Kaufman S, Kaufman JS, Poole C. Bounding causal effects under uncontrolled confounding using counterfactuals. Epidemiol 2005; 16: 548-55.         

30. Stürmer T, Schneeweiss S, Avorn J, Glynn RJ. Adjusting effect estimates for unmeasured confounding with validation data using propensity score calibration. Am J Epidemiol 2005; 162: 279-89.         



Ronir Raggio Luiz
Instituto de Estudos de Saúde Pública
Universidade do Rio de Janeiro
Praça Jorge Machado Moreira, 100, Cidade Universitária
Rio de Janeiro, RJ - Brasil, CEP 21941-598

Recebido em: 14/07/09
Versão final reapresentada em: 14/04/10
Aprovado em: 29/04/10

Associação Brasileira de Pós -Graduação em Saúde Coletiva São Paulo - SP - Brazil