Validity and reproducibility of a food frequency questionnaire for adults of São Paulo, Brazil

Soraya Sant'Ana de Castro Selem Aline Martins de Carvalho Eliseu Verly-Junior Jackeline Venâncio Carlos Juliana Araujo Teixeira Dirce Maria Lobo Marchioni Regina Mara Fisberg About the authors

Abstracts

OBJECTIVES:

To assess the validity and reproducibility of a food frequency questionnaire developed for estimating the food consumption of adults in São Paulo, Brazil, based population study.

METHODS:

A sample of individuals aged above 20 years, of both genders, living in São Paulo, was used for the validation study (n = 77) and reproducibility study (n = 74) of the food frequency questionnaire. To verify the validity and reproducibility of energy and 19 nutrients were applied two food frequency questionnaires (60 items) and three 24-hour dietary recalls (24HR - reference method). The validity was verified by Spearman correlation coefficient (crude and de-attenuated) and weighted Kappa, and reproducibility by intraclass correlation coefficients and weighted kappa.

RESULTS:

In analyzes of validity de-attenuated correlation coefficients ranged from 0.21 (carbohydrate) to 0.74 (energy), and weighted kappa exceeded 0.40 for 30% of the nutrients. Polyunsaturated fat and folate did not show significant correlation and weighted kappa. In reproducibility correlation coefficients ranged from 0.36 (polyunsaturated fat) to 0.69 (calcium), and weighted kappa exceeded 0.40 for 80% of the nutrients.

CONCLUSION:

The food frequency questionnaire analyzed has good validity and reproducibility for estimating the food consumption of adults in São Paulo compared to the reference method, so it is an appropriate instrument to be used in epidemiological studies on similar populations. Estimates of polyunsaturated fat and folate should be interpreted with caution.

Diet surveys; Food-frequency questionnaire; Food consumption; Validation studies; Adult; São Paulo


INTRODUCTION

In recent decades, the food frequency questionnaire (FFQ) has been the most used tool to assess food and nutrient intake in epidemiological studies, especially those whose goal is to relate dietary factors with health outcomes. This occurs due to the fact that this method is considered a survey that assesses usual diet presenting a low cost, easy application and processing, and generating information that allows sorting and categorizing of individuals according to exposure to certain dietary components1. Willett WC. Nutritional epidemiology, 2nd edition. New York: Oxford University Press; 1998. , 2. Slater B, Philippi ST, Marchioni DML, Fisberg RM. Validação de Questionários de Freqüência Alimentar - QFA: considerações metodológicas. Rev Bras Epidemiol 2003; 6(3): 200-8..

The development of an FFQ requires prior knowledge of food and nutrient intake in the population on which it will be applied. Usually, 24-hour dietary recalls (24HR) are applied in pilot studies, generating a database for establishment of food items that will make up the instrument, as well as the portion sizes3. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2001; 5(4): 567-87.. Like any other instrument, the FFQ is not free of measurement errors. In this sense, validation and reproducibility studies are strategies to verify the accuracy and precision of their measurements4. Teixeira JA, Baggio ML, Giuliano AR, Fisberg RM, Marchioni DML. Performance of the Quantitative Food Frequency Questionnaire Used in the Brazilian Center of the Prospective Study Natural History of Human Papillomavirus Infection in Men: The HIM Study. J Am Diet Assoc 2011; 111(7): 1045-51..

The FFQ developed by Fisberg et al.5. Fisberg RM, Colucci ACA, Morimoto JM, Marchioni DML. Questionário de freqüência alimentar para adultos com base em estudo populacional. Rev Saúde Pública 2008; 42(3): 550-4. to estimate usual dietary intake of adults is the only one developed from a population-based study with a probability sample of the city of São Paulo, the most populous and multicultural city in Brazil. However, it is necessary to evaluate the performance of this instrument for its application in this population. The objective of this study was to verify the accuracy and precision of the instrument.

METHODS

THE INSTRUMENT

The FFQ, designed to assess the habitual consumption of adults of both sexes during the year preceding its application, had as basis for its development dietary information obtained through a 24HR applied in a random sample of individuals aged 20 years or more (n = 1,477) residing in the city of São Paulo. This information was obtained in the Health Survey (ISA-Capital) study conducted in 2003. Foods that contributed up to 90% for the consumption of energy and nutrients were included and grouped, as well as foods that are sources of nutrients and that could discriminate the study subjects according to levels of consumption. The FFQ consists of 60 food items with frequencies of 0 - 10 times, and a unit of time: day, week, month and year. The portions were defined as small, medium, large and extra large, which correspond to the 25, 50, 75 and 95th percentiles of the consumption in grams of each food item. The median portion is the reference serving size, and is presented in household measures and in grams. Food items are organized into: soups and pasta, meat and fish, dairy products, legumes and eggs, rice and tubers, vegetables, sauces and spices, fruits, beverages, breads and biscuits, sweets and desserts. The FFQ also presents questions about recent changes in eating habits, consumption of dietary supplements, other important foods consumed that the instrument did not include, as well as consumption of visible fat from meat and skin from poultry5. Fisberg RM, Colucci ACA, Morimoto JM, Marchioni DML. Questionário de freqüência alimentar para adultos com base em estudo populacional. Rev Saúde Pública 2008; 42(3): 550-4..

STUDY POPULATION AND DATA COLLECTION

Between 2007 and 2008, a study was conducted in a representative subsample of the participants of ISA-Capital 2003 of both sexes, aged 20 or more, to estimate the prevalence of inadequate nutrient intake. The FFQ used was developed based on data from the 2003 ISA-Capital, and its accuracy and precision were verified in the present study. Two home visits, with an interval of one year between them, were performed for data collection, during which one FFQ and one 24HR - the reference method for validation - were conducted in each of the visits. The one-year interval between data collection was determined because the FFQ was developed to assess habitual intake during the year preceding its application3. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2001; 5(4): 567-87. , 5. Fisberg RM, Colucci ACA, Morimoto JM, Marchioni DML. Questionário de freqüência alimentar para adultos com base em estudo populacional. Rev Saúde Pública 2008; 42(3): 550-4.. In addition, in the period between visits, another 24HR was collected by phone, and the average interval between 24HR collections of was six months (Figure 1). The size of the subsample studied was reduced due to loss of follow-up, whose main reason was the failure to locate the individual. In the first home visit, 607 FFQs were collected, and 84 FFQs in the second visit. Also, 20 individuals who presented under- and overestimation of food consumption were excluded from the analysis, according to the method proposed by Nielsen6. Nielsen SJ, Adair L. An alternative to dietary data exclusions. J Am Diet Assoc 2007; 107: 792-99.. At the end, 77 individuals (34 men and 43 women) responded to the three 24HR and to the second FFQ, and 74 (32 men and 42 women) responded to the two FFQ.

Figure 1
Study diagram.

For the application of the 24HR, a standard form and the Multiple Pass Method, developed by the United States Department of Agriculture (USDA) to assist in the collection of food consumption data, were used, minimizing errors in the measurement of diet7. Raper N, Perloff B, Ingwersen L, Steinfeldt L, Anand J. An overview of USDA's Dietary Intake Data System. J Food Compost Anal 2004; 17(3-4): 545-55.. The 24HR were applied so that all days of the week and seasons were represented. The standardization and quantification of foods and preparations were performed according to national recommendations, which allowed the conversion of household measurements in units of weight or volume8. Fisberg RM, Villar BS. Manual de receitas e Medidas caseiras para Cálculo de Inquéritos Alimentares: manual elaborado para auxiliar o processamento de inquéritos alimentares. São Paulo: Signus; 2002. , 9. Pinheiro ABV, Lacerda EM de A, Benzecry EH, Gomes MC da S, Costa VM da. Tabela para Avaliação de Consumo Alimentar em Medidas Caseiras. 5. ed. São Paulo: Atheneu; 2008..

To quantify the energy and nutrients consumed, obtained by the FFQ and 24HR, the Nutrition Data System for Research software (version 2007, 2007, University of Minnesota, Minneapolis) was used.

DATA ANALYSIS

The following were selected for analysis: energy, protein, carbohydrates, fiber, total fat, saturated fat, monounsaturated fat, polyunsaturated fat, trans fats, cholesterol, vitamin A, beta-carotene, folate, vitamin C, vitamin E, calcium, phosphorus, iron, zinc and sodium. In all analyzes regarding intake of nutrients (validation and reproducibility), intakes of nutrients values were used, adjusted for energy intake by the residual method1. Willett WC. Nutritional epidemiology, 2nd edition. New York: Oxford University Press; 1998..

Validity was determined by comparing the values of energy intake and each nutrient by the second FFQ (FFQ2, applied the second home visit) and the average of the three 24HR. The Spearman's correlation coefficient and weighted Kappa were used to verify compliance according to consumption quartiles.

To minimize the effect of intra-subject variability of intake measured by the 24HR, each correlation coefficient was multiplied by a deattenuation coefficient. This was calculated using the following formula:

where λ is the ratio between intrapersonal variance and interpersonal intake variance; r o is the observed correlation, n is the number of days of 24HR and r v is the deattenuated correlation. The intra- and interpersonal variances were calculated for each nutrient using the method proposed by Iowa State University (ISU)1010 . Nusser SM, Carriquiry AL, Dodd K, Fuller WA. A semiparametric transformation approach to estimating usual daily intake distributions. J Am Stata Assoc 1996; 91(436): 1440-9..

Reproducibility was checked by comparing the values of energy intake and each nutrient obtained by the first FFQ (FFQ1, applied on the first home visit) with those obtained by FFQ2, using the intraclass correlation and weighted Kappa to verify compliance according to consumption quartiles.

All statistical analyzes were performed using Stata software (version 10, 2007, StataCorp LP, College Station, Texas). A significance level of 5% was considered.

This study was approved by the Research Ethics Committee of the School of Public Health, University of São Paulo, and the participation was conditioned upon signing of a free and informed consent form. There are no conflicts of interest.

RESULTS

The average consumption of energy and 10 nutrients estimated by FFQ2 was similar to that estimated by 24HR. In comparison with FFQ1, FFQ2 underestimated the consumption of 15 nutrients (Table 1).

Table 1
Estimation of daily intake, correlation coefficients and concordance analysis to validation and reproducibility analyses.

In validity analyzes, the correlation coefficients ranged from 0.25 (carbohydrates) to 0.63 (energy). After deattenuation, there was a higher correlation for 12 nutrients, and the coefficients ranged from 0.21 (carbohydrates) to 0.74 (energy). The weighted Kappa exceeded 0.40 for 30% of the nutrients analyzed, and 45% of the nutrients showed values between 0.30 and 0.40. Polyunsaturated fat and folate showed no significant correlation and weighted Kappa compared to the reference method (Table 1).

In reproducibility, the correlation coefficients ranged from 0.36 (polyunsaturated fat) to 0.69 (calcium), and the weighted Kappa exceeded 0.40 for 80% of the nutrients (Table 1).

DISCUSSION

This study suggests that the FFQ developed to estimate usual dietary intake of adults in the city of São Paulo has good validity and reproducibility for most nutrients analyzed.

The underestimation of the second FFQ compared to the first has been observed in other studies1111 . Barrat E, Aubineau N, Maillot M, Derbord E, Barthes P, Lescuyer J, et al. Repeatability and relative validity of a quantitative food-frequency questionnaire among French adults. Food &amp; Nutrition Research 2012;56. Disponível em: <http://www.foodandnutritionresearch.net/index.php/fnr/article/view/18472/24636>. (Acessado em 12 de dezembro de 2012).
http://www.foodandnutritionresearch.net/...
, 1212 . Turconi G, Bazzano R, Roggi C, Cena H. Reliability and relative validity of a quantitative food-frequency questionnaire for use among adults in Italian population. Int J Food Sci Nutr 2010; 61(8): 846-62.. This may occur because the participants learn to respond to the questionnaire more accurately. And it is possible to assume that the overestimation of consumption of four nutrients (carbohydrates, fiber, beta-carotene and vitamin C) by FFQ2 in comparison with 24HR happens by overestimating the consumption of sources of these nutrients, such as fruits, a common bias among individuals seeking for social approval of their diet1313 . Monteiro CA, Moura EC, Jaime PC, Claro RM. Validade de indicadores do consumo de alimentos e bebidas obtidos por inquérito telefônico. Rev Saúde Pública 2008; 42(4): 582-9.. It is noteworthy that the overestimation of nutrients can hide the poor consumption of the population1111 . Barrat E, Aubineau N, Maillot M, Derbord E, Barthes P, Lescuyer J, et al. Repeatability and relative validity of a quantitative food-frequency questionnaire among French adults. Food &amp; Nutrition Research 2012;56. Disponível em: <http://www.foodandnutritionresearch.net/index.php/fnr/article/view/18472/24636>. (Acessado em 12 de dezembro de 2012).
http://www.foodandnutritionresearch.net/...
.

The FFQ did not show good validity in estimating polyunsaturated fat and folate, a result observed in other studies4. Teixeira JA, Baggio ML, Giuliano AR, Fisberg RM, Marchioni DML. Performance of the Quantitative Food Frequency Questionnaire Used in the Brazilian Center of the Prospective Study Natural History of Human Papillomavirus Infection in Men: The HIM Study. J Am Diet Assoc 2011; 111(7): 1045-51. , 1414 . Henn RL, Fuchs SC, Moreira LB, Fuchs FD. Development and validation of a food frequency questionnaire (FFQ-Porto Alegre) for adolescent, adult and elderly populations from Southern Brazil. Cad. Saúde Pública 2010; 26(11) 2068-79. , 1515 . Na YJ, Lee SH. Development and validation of a quantitative food frequency questionnaire to assess nutritional status in Korean adults. Nutr Res Pract 2012; 6(5): 444-50.. Thus, it is considered that the estimates of polyunsaturated fat and folate should be interpreted with caution. Nevertheless, the performance of the analyzed FFQ can be considered satisfactory, and the results obtained are similar to other published studies, considering the diversity of these in relation to the study population, the reference method, the time elapsed between the surveys and other characteristics of the instrument1414 . Henn RL, Fuchs SC, Moreira LB, Fuchs FD. Development and validation of a food frequency questionnaire (FFQ-Porto Alegre) for adolescent, adult and elderly populations from Southern Brazil. Cad. Saúde Pública 2010; 26(11) 2068-79.

15 . Na YJ, Lee SH. Development and validation of a quantitative food frequency questionnaire to assess nutritional status in Korean adults. Nutr Res Pract 2012; 6(5): 444-50.
- 1616 . Salvo VLMA, Gimeno SGA. Reprodutibilidade e validade do questionário de freqüência de consumo de alimentos. Rev Saúde Pública 2002; 36(4): 505-12..

The study has some strengths, such as the fact that the FFQ was developed based on a probability sample of São Paulo, the largest city in Brazil, with immigrants from all over the country and a diet that is influenced by different cultures. In addition, there was an effort to seek the participants of the original study to evaluate the performance of the FFQ. The method for calculation of intra- and interpersonal variances used in the deattenuation is considered one of the best1010 . Nusser SM, Carriquiry AL, Dodd K, Fuller WA. A semiparametric transformation approach to estimating usual daily intake distributions. J Am Stata Assoc 1996; 91(436): 1440-9.. Finally, accuracy and precision were evaluated for energy and 19 nutrients, higher than in most published studies, which often refer only to the number of macronutrients.

There are also limitations to consider. Although statistical procedures aimed at correcting the correlation coefficients were applied, providing a better estimate, the reference method used has sources of errors in common with the FFQ, because it depends on the respondent's memory and ability to recall and report the foods and portions consumed. The use of recovery biomarkers as the reference method in validation analysis could provide more reliable results for the study, but these tests are expensive, often invasive, besides being available for few nutrients. Thus, the use of 24HR is still a viable and widely used alternative3. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2001; 5(4): 567-87.. The sample size may compromise the statistical power of the analysis, but reduced sample sizes are common in this type of study due to loss of follow-up1111 . Barrat E, Aubineau N, Maillot M, Derbord E, Barthes P, Lescuyer J, et al. Repeatability and relative validity of a quantitative food-frequency questionnaire among French adults. Food &amp; Nutrition Research 2012;56. Disponível em: <http://www.foodandnutritionresearch.net/index.php/fnr/article/view/18472/24636>. (Acessado em 12 de dezembro de 2012).
http://www.foodandnutritionresearch.net/...
, 1212 . Turconi G, Bazzano R, Roggi C, Cena H. Reliability and relative validity of a quantitative food-frequency questionnaire for use among adults in Italian population. Int J Food Sci Nutr 2010; 61(8): 846-62. , 1717 . Mendes LL, Campos SF, Malta DC, Bernal RTI, Sá NNB, Velásquez-Meléndez G. Validade e reprodutibilidade de marcadores do consumo de alimentos e bebidas de um inquérito telefônico realizado na cidade de Belo Horizonte (MG), Brasil. Rev Bras Epidemiol 2011; 14(1 Supl): 80-9. , 1818 . Lima FEL, Slater B, Latorre MRDO, Fisberg RM. Validade de um questionário quantitativo de frequência alimentar desenvolvido para população feminina no nordeste do Brasil. Rev Bras Epidemiol 2007; 10(4): 483-90..

CONCLUSION

The FFQ considered has good validity and reproducibility to estimate the intake of most nutrients in adults in the city of São Paulo compared to the reference method, so it is an appropriate instrument for epidemiological studies with similar populations. Estimates of polyunsaturated fat and folate, however, should be interpreted with caution.

REFERENCES

  • 1
    Willett WC. Nutritional epidemiology, 2nd edition. New York: Oxford University Press; 1998.
  • 2
    Slater B, Philippi ST, Marchioni DML, Fisberg RM. Validação de Questionários de Freqüência Alimentar - QFA: considerações metodológicas. Rev Bras Epidemiol 2003; 6(3): 200-8.
  • 3
    Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2001; 5(4): 567-87.
  • 4
    Teixeira JA, Baggio ML, Giuliano AR, Fisberg RM, Marchioni DML. Performance of the Quantitative Food Frequency Questionnaire Used in the Brazilian Center of the Prospective Study Natural History of Human Papillomavirus Infection in Men: The HIM Study. J Am Diet Assoc 2011; 111(7): 1045-51.
  • 5
    Fisberg RM, Colucci ACA, Morimoto JM, Marchioni DML. Questionário de freqüência alimentar para adultos com base em estudo populacional. Rev Saúde Pública 2008; 42(3): 550-4.
  • 6
    Nielsen SJ, Adair L. An alternative to dietary data exclusions. J Am Diet Assoc 2007; 107: 792-99.
  • 7
    Raper N, Perloff B, Ingwersen L, Steinfeldt L, Anand J. An overview of USDA's Dietary Intake Data System. J Food Compost Anal 2004; 17(3-4): 545-55.
  • 8
    Fisberg RM, Villar BS. Manual de receitas e Medidas caseiras para Cálculo de Inquéritos Alimentares: manual elaborado para auxiliar o processamento de inquéritos alimentares. São Paulo: Signus; 2002.
  • 9
    Pinheiro ABV, Lacerda EM de A, Benzecry EH, Gomes MC da S, Costa VM da. Tabela para Avaliação de Consumo Alimentar em Medidas Caseiras. 5. ed. São Paulo: Atheneu; 2008.
  • 10
    Nusser SM, Carriquiry AL, Dodd K, Fuller WA. A semiparametric transformation approach to estimating usual daily intake distributions. J Am Stata Assoc 1996; 91(436): 1440-9.
  • 11
    Barrat E, Aubineau N, Maillot M, Derbord E, Barthes P, Lescuyer J, et al. Repeatability and relative validity of a quantitative food-frequency questionnaire among French adults. Food &amp; Nutrition Research 2012;56. Disponível em: <http://www.foodandnutritionresearch.net/index.php/fnr/article/view/18472/24636>. (Acessado em 12 de dezembro de 2012).
    » http://www.foodandnutritionresearch.net/index.php/fnr/article/view/18472/24636>
  • 12
    Turconi G, Bazzano R, Roggi C, Cena H. Reliability and relative validity of a quantitative food-frequency questionnaire for use among adults in Italian population. Int J Food Sci Nutr 2010; 61(8): 846-62.
  • 13
    Monteiro CA, Moura EC, Jaime PC, Claro RM. Validade de indicadores do consumo de alimentos e bebidas obtidos por inquérito telefônico. Rev Saúde Pública 2008; 42(4): 582-9.
  • 14
    Henn RL, Fuchs SC, Moreira LB, Fuchs FD. Development and validation of a food frequency questionnaire (FFQ-Porto Alegre) for adolescent, adult and elderly populations from Southern Brazil. Cad. Saúde Pública 2010; 26(11) 2068-79.
  • 15
    Na YJ, Lee SH. Development and validation of a quantitative food frequency questionnaire to assess nutritional status in Korean adults. Nutr Res Pract 2012; 6(5): 444-50.
  • 16
    Salvo VLMA, Gimeno SGA. Reprodutibilidade e validade do questionário de freqüência de consumo de alimentos. Rev Saúde Pública 2002; 36(4): 505-12.
  • 17
    Mendes LL, Campos SF, Malta DC, Bernal RTI, Sá NNB, Velásquez-Meléndez G. Validade e reprodutibilidade de marcadores do consumo de alimentos e bebidas de um inquérito telefônico realizado na cidade de Belo Horizonte (MG), Brasil. Rev Bras Epidemiol 2011; 14(1 Supl): 80-9.
  • 18
    Lima FEL, Slater B, Latorre MRDO, Fisberg RM. Validade de um questionário quantitativo de frequência alimentar desenvolvido para população feminina no nordeste do Brasil. Rev Bras Epidemiol 2007; 10(4): 483-90.

  • Financial support: National Counsel of Technological and Scientific Development (CNPq) no. 503128/2010-4; Foundation for Research Support of the State of São Paulo (FAPESP) no. 2009/15831-0.

Publication Dates

  • Publication in this collection
    Dec 2014

History

  • Received
    09 Jan 2014
  • Reviewed
    19 Feb 2014
  • Accepted
    29 Apr 2014
Associação Brasileira de Pós -Graduação em Saúde Coletiva São Paulo - SP - Brazil
E-mail: revbrepi@usp.br