ABSTRACT

Objective

To determine the reliability of a non-laboratorial questionnaire, the Encuesta de Identificación de Sujetos Metabólicamente Comprometidos en Fase-I (ESF-I) for identifying Metabolic Syndrome among a population in central Mexico.

Methods

Clinical and biochemical parameters were collected for 232 participants from 1 June 2012 – 31 August 2013. Three definitions of Metabolic Syndrome (Harmonizing, National Cholesterol Education Program Expert Panel and Adult Treatment Panel III [ATPIII], and International Diabetes Federation [IDF]) were used to allocate subjects to either the normal or Metabolic Syndrome positive (MetS+) group. The predictability of the questionnaire was determined by the Area-Under-the-Receiver-Operating Characteristic curve (AUC). Youden's index was calculated and the highest score was considered the optimal cutoff value. Cohen´s kappa (κ) was calculated to determine the level of agreement between the ESF-I questionnaire (max score: 15 based on 15 items) and Metabolic Syndrome.

Results

From 53.8% – 60.7% of the participants were determined to be MetS+. The average questionnaire score was significantly higher in the MetS+ group for each definition (4.0 vs. 8.0, P < 0.05). The ESF-I questionnaire was predictive for the Harmonizing definition (AUC = 0.841, 95%CI: 0.790 – 0.892), the ATPIII definition (AUC = 0.827, 95%CI: 0.774 – 0.880), and the IDF definition (AUC = 0.836, 95%CI: 0.785 – 0.887). A cutoff value of 7 was determined for each definition; therefore, the cohort was re-categorized based on questionnaire results. There was a strong agreement between the ESF-I questionnaire and MetS (Harmonizing: accuracy = 77.6%, κ = 0.554; ATPIII: accuracy = 74.1%, κ = 0.489; IDF: accuracy = 74.6%, κ = 0.495, P < 0.001).

Conclusion

The ESF-I questionnaire can identify MetS+ patients, and therefore, lead to earlier diagnoses, reduced number of consultations, and lower costs with easier application.

Keywords
Metabolic syndrome, diagnosis; surveys and questionnaires; noncommunicable diseases; Mexico

RESUMEN

Objetivo

Determinar la fiabilidad de un cuestionario en el que no se recurre al diagnóstico de laboratorio, la Encuesta de Identificación de Sujetos Metabólicamente Comprometidos en Fase-I (ESF-I), para detectar el síndrome metabólico en una población de la región central de México.

Métodos

Se recogieron parámetros clínicos y bioquímicos de 232 participantes desde el 1 de junio del 2012 al 31 de agosto del 2013. Se usaron tres definiciones de síndrome metabólico (la unificadora, la del Grupo de Expertos en el Tratamiento de Adultos [ATPIII] del Programa Nacional de Educación sobre el Colesterol y la de la Federación Internacional de la Diabetes [FID]) para asignar los participantes al grupo normal o al grupo de síndrome metabólico positivo (SMet+). Se determinó la previsibilidad del cuestionario con el área bajo la curva de eficacia diagnóstica (curva ROC). Se calculó el índice de Youden y la puntuación más alta se consideró el valor de corte óptimo. El coeficiente (k) de Cohen se calculó para determinar el grade de acuerdo entre el cuestionario ESF-I (máxima puntuación: 15 sobre 15 ítems) y el síndrome metabólico.

Resultados

Del 53,8 % al 60,7 % de los participantes se asignaron al grupo SMet+. La puntuación promedio del cuestionario fue significativamente mayor en el grupo de SMet+ para cada definición (4,0 vs. 8.0, P < 0.05). El cuestionario ESF-I fue predictivo para la definición unificadora (AUC = 0,841, 95 % CI: 0,790 – 0,892), la definición ATPIII (AUC = 0,827, 95 % CI: 0,774 – 0,880) y la definición de la FID (AUC = 0,836, 95 % CI: 0,785 – 0,887). Se determinó un el valor de corte óptimo de 7 para cada definición; por lo tanto, se reclasificó la cohorte según los resultados del cuestionario. Hubo una gran coincidencia entre el cuestionario ESF-I y SMet (unificadora: exactitud = 77,6 %, κ = 0,554; ATPIII: exactitud = 74,1 %, κ = 0,489; FID: exactitud = 74,6%, κ = 0,495, P < 0,001).

Conclusiones

El cuestionario ESF-I puede detectar pacientes con SMet+ y, por lo tanto, conducir a diagnósticos más tempranos, reducir la cantidad de consultas y reducir los costos con una aplicación más fácil.

Palabras clave
Síndrome metabólico, diagnóstico; encuestas y cuestionarios; enfermedades no transmisibles; México

RESUMO

Objetivo

Determinar a confiabilidade de um instrumento de coleta de dados não laboratoriais, Encuesta de Identificación de Sujetos Metabólicamente Comprometidos en Fase-I (pesquisa de identificação de indivíduos com comprometimento metabólico em fase I, ESF-I) para identificar síndrome metabólica em uma população da região central do México.

Métodos

Foram coletados parâmetros clínicos e bioquímicos de 232 participantes da pesquisa entre 1° de junho de 2012 e 31 de agosto de 2013. O estudo se baseou em três definições de síndrome metabólica (definição harmonizada do Painel de Especialistas do Programa Nacional de Educação em Colesterol dos Estados Unidos [NCEP]; Painel para Tratamento de Adultos III [ATPIII]; e Federação Internacional de Diabetes [FIL]) para alocar os participantes em um grupo normal ou em um grupo com síndrome metabólica (SM). A previsibilidade do instrumento foi determinada pela área sob a curva ROC (característica de operação do receptor) (AUC). O índice de Youden foi calculado e o escore mais alto foi considerado o valor ideal de corte. O coeficiente kappa de Cohen (κ) foi calculado para determinar o grau de concordância entre o questionário ESF-1 (pontuação máxima de 15 em 15 itens) e a síndrome metabólica.

Resultados

Foi determinado que 53,8% a 60,7% dos participantes apresentavam SM. A pontuação média no instrumento foi significativamente maior no grupo SM com cada uma das definições usadas (4,0 vs. 8,0, P<0,05). O questionário ESF-I foi preditivo com a definição harmonizada (AUC 0,841; IC 95% 0,790–0,892), com a definição do ATPIII (AUC 0,827; IC 95% 0,774–0,880) e com a definição da FIL (AUC 0,836; IC 95% 0,785–0,887). Um valor de corte de 7 foi determinado para cada definição e a coorte foi recategorizada de acordo com os resultados do instrumento. Foi observada uma forte concordância entre o questionário ESF-I e o grupo SM (harmonizada: precisão = 77,6%, κ = 0,554; ATPIII: precisão = 74,1%, κ = 0,489; FIL: precisão = 74,6%, κ = 0,495, P<0,001).

Conclusão

O questionário ESF-I é capaz de identificar pacientes com síndrome metabólica, possibilitando o diagnóstico precoce, um número menor de consultas e um custo menor com uma aplicação mais simples.

Palavras-chave
Síndrome metabólica, diagnóstico; inquéritos e questionários; doenças não transmissíveis; México

Around the world, more people are living sedentary lifestyles and consuming high-energy diets, augmenting the rate of Metabolic Syndrome (MetS) and its associated complications (¹1. Pérez-Martínez P, Mikhailidis DP, Athyros VG, Bullo M, Couture P, Covas MI, et al. Lifestyle recommendations for the prevention and management of metabolic syndrome: an international panel recommendation. Nutr Rev. 2017;75(5):307–26. doi: 10.1093/nutrit/nux014
https://doi.org/10.1093/nutrit/nux014... ). With an estimated 20% – 25% of the world's adult population positive for MetS, it is now one of the most prevalent chronic diseases (²2. Salas R, Bibiloni MdM, Ramos E, Villarreal JZ, Pons A, Tur JA, et al. Metabolic Syndrome prevalence among Northern Mexican Adult Population. PLoS One. 2014;9(8):e105581. doi: 10.1371/journal.pone.0105581
https://doi.org/10.1371/journal.pone.010... ). MetS has become a significant health concern, especially in some developing countries. According to a recent report, the MetS rate in Mexico is an alarming 54.8% and rising annually (³3. Ortiz-Rodríguez MA, Yáñez-Velasco L, Carnevale A, Romero-Hidalgo S, Bernal D, Aguilar-Salinas C, et al. Prevalence of metabolic syndrome among elderly Mexicans. Arch Gerontol Geriatr. 2017;73:288–93. doi: 10.1016/j.archger.2017.09.001
https://doi.org/10.1016/j.archger.2017.0... ). Moreover, studies have shown that MetS is associated with a 5-fold risk of developing type-2 diabetes (T2D) and a 2-fold risk for cardiovascular disease (⁴4. Kaur J. Assessment and screening of the risk factors in Metabolic Syndrome. Med Sci. 2014;2(3):140–52. DOI: 10.3390/medsci2030140
https://doi.org/10.3390/medsci2030140... ).

Diagnosing MetS implies substantial costs, which in Mexico and other developing countries can be overwhelming for both the patient and the government health budget. The disorder is characterized by the presence of multiple risk factors: central obesity, hypertension, hyperglycemia, hypertriglyceridemia, and low plasma high-density lipoprotein (⁵5. Alberti KG, Zimmet P, Shaw J. Metabolic syndrome – a new world wide definition. A Consensus Statement from the International Diabetes Federation. Diabet Med. 2006;23(5):469–80. doi: 10.1111/j.1464-5491.2006.01850.x
https://doi.org/10.1111/j.1464-5491.2006... , ⁶6. Kaur J. A comprehensive review on Metabolic Syndrome. Cardiol Res Pract. 2014. doi: 10.1155/2014/943162
https://doi.org/10.1155/2014/943162... ). Measuring the last three components involves a blood sample drawn by a trained professional in a medical office, laboratory, or hospital; a blood pressure reading; and a follow-up appointment to discuss laboratory results. For the Instituto Mexicano del Seguro Social (Mexican Social Security Institute; IMSS), the cost of blood work to diagnose MetS is US$ 37.45 (19.46 pesos/US$) per patient; costs are higher at a private clinic (⁷7. XRATES. Currency calculator, historic lookup. Available from: https://www.x-rates.com/historical/?from=USD&amount=1&date=2010-09-09 Accessed 20 August 2018.
https://www.x-rates.com/historical/?from... ). Moreover, since the average Mexican makes US$ 292.50 per month (⁸8. Indeed. Salarios de INEGI en México. Available from: https://www.indeed.com.mx/cmp/Inegi/salaries Accessed 25 August 2018.
https://www.indeed.com.mx/cmp/Inegi/sala... ), most opt not to receive preventive medical care.

Given the expenses and resources required to make a MetS diagnosis, large-scale screenings are impractical. However, there is plenty of evidence to support the use of alternative, non-laboratorial diagnosis since MetS is associated with multiple alternative factors. For example, a diet high in carbohydrates (> 258 g daily) was shown to induce the glycemic index, which was positively associated with MetS development (⁹9. Lee YJ, Song S, Song Y. High-carbohydrate diets and food patterns and their associations with Metabolic Disease in the Korean population. Yonsei Med J. 2018;59(7):834–42. doi: 10.3349/ymj.2018.59.7.834
https://doi.org/10.3349/ymj.2018.59.7.83... ). Another factor was Acanthosis Nigricans, a dermatoses characterized by warty plaques with hyperkeratosis, papillomatosis, and hyperpigmentation, occurring mainly on the neck, armpits, and inguinal area (¹⁰10. Reissig F. Insulin resistance and Acanthosis Nigricans. In: Cohen Sabban EN, Puchulu FM, Cusi K, eds. Dermatology and Diabetes. Cham: Springer International Publishing; 2018. Pp. 105–15.). The presence of Acanthosis Nigricans on the neck has been shown to be a reliable clinical indicator of insulin resistance (IR), an important component of MetS (¹¹11. Choudhary SV, Saoji V, Singh A, Mane S. Acanthosis nigricans: a clinical marker of insulin resistance. Int J Res Dermatol. 2017;3(2):161–7. doi: 10.18203/issn.2455–4529.IntJResDermatol20171497
https://doi.org/10.18203/issn.2455–4529.... ). In addition, in males there is a strong association between MetS and erectile dysfunction (¹²12. Chaudhary RK, Shamsi BH, Tan T, Chen H-M, Xing J-P. Study of the relationship between male erectile dysfunction and type 2 diabetes mellitus/metabolic syndrome and its components. J Int Med Res. 2016;44(3):735–41. doi: 10.1177/0300060515623122
https://doi.org/10.1177/0300060515623122... ); and in females, between MetS and maternal body mass index (BMI) and fetal macrosomia (¹³13. Hakkarainen H, Huopio H, Cederberg H, Voutilainen R, Heinonen S. Future risk of metabolic syndrome in women with a previous LGA delivery stratified by gestational glucose tolerance: a prospective cohort study. BMC Pregnancy Childbirth. 2018;18(1):326. doi: 10.1186/s12884-018-1958-z.
https://doi.org/10.1186/s12884-018-1958-... ). Lastly, other anthropometric measurements, such as the waist-to-height ratio, have been correlated with MetS (¹⁴14. Sagun G, Oguz A, Karagoz E, Filizer AT, Tamer G, Mesci B. Application of alternative anthropometric measurements to predict metabolic syndrome. Clinics. 2014;69(5):347–53. doi: 10.6061/clinics/2014(05)09
https://doi.org/10.6061/clinics/2014(05)... ).

Due to the challenges of screening large populations, there is a new push to use simpler, less costly techniques, such as the self-reported questionnaire. The advantages of such a tool are that it is easy to administer, requires relatively little time, does not require a blood sample or specialized staff, and has lower costs associated with its application. Most self-reported questionnaires have been shown to have good predictive capacity; for example, the American Diabetes Association developed a questionnaire (¹⁵15. Li C-I, Chien L, Liu C-S, Lin W-Y, Lai M-M, Lee C-C, et al. Prospective validation of american diabetes association risk tool for predicting pre-diabetes and diabetes in taiwan–taichung community health study. PloS One. 2011;6(10):e25906. doi: 10.1371/journal.pone.0025906
https://doi.org/10.1371/journal.pone.002... , ¹⁶16. Scanlan AB, Maia CM, Perez A, Homko CJ, O’Brien MJ. Diabetes Risk Assessment in Latinas: Effectiveness of a brief diabetes risk questionnaire for detecting prediabetes in a community-based sample. Diabetes Spectr. 2017;31(1):31–6. doi: 10.2337/ds16-0051
https://doi.org/10.2337/ds16-0051... ) to determine risk for T2D, with satisfactory concordance (sensitivity = 79%; specificity = 65%). Use of a questionnaire for MetS diagnosis has recently been investigated. In Greece, the Finnish diabetes risk score (FINDRISC) questionnaire (¹⁷17. Makrilakis K, Liatis S, Grammatikou S, Perrea D, Stathi C, Tsiligros P, et al. Validation of the Finnish diabetes risk score (FINDRISC) questionnaire for screening for undiagnosed type 2 diabetes, dysglycaemia and the metabolic syndrome in Greece. Diabetes Metab. 2011;37(2):144–51. doi: 10.1016/j.diabet.2010.09.006
https://doi.org/10.1016/j.diabet.2010.09... ) was shown to predict MetS with high sensitivity (89% – 98%), but poor specificity (14% – 37%). Tan and colleagues (¹⁸18. Tan C, Sasagawa Y, Kamo K-i, Kukitsu T, Noda S, Ishikawa K, et al. Evaluation of the Japanese Metabolic Syndrome Risk Score (JAMRISC): a newly developed questionnaire used as a screening tool for diagnosing metabolic syndrome and insulin resistance in Japan. Environ Health Prev Med. 2016;21(6):470. doi: 10.1007/s12199-016-0568-5
https://doi.org/10.1007/s12199-016-0568-... ) demonstrated that the Japanese Metabolic Syndrome Risk Score (JAMRISC) questionnaire was superior to FINDRISC (sensitivity = 100%; specificity 72%). Others have tried to develop a MetS risk questionnaire, but have failed to attain the quality of the JAMRISC (¹⁹19. Yount BW, Wyrwich KW, Brownson RC. The reliability of a questionnaire-based metabolic syndrome surveillance tool. Metab Syndr Relat Disord. 2007;5(3):282–9. doi: 10.1089/met.2006.0036.
https://doi.org/10.1089/met.2006.0036... , ²⁰20. Norhafizah AM, Nor AY, Al-Safi I. Development of a questionnaire on prevention and risk of Metabolic Syndrome among a rural population in Kelantan. Int J Sci Res Publ. 2017;7(4):60–3.). The JAMRISC questionnaire, however, has questions not applicable in Latin America, such as consumption of sake.

Given the usefulness of questionnaires for MetS diagnosis, the high prevalence of MetS in Mexico, and the high cost of its diagnosis in the country, there was clearly a need for a Mexican MetS instrument. This study sought to evaluate the reliability of the newly developed Encuesta de Identificación de Sujetos Metabólicamente Comprometidos en Fase-I (ESF-I)—a questionnaire based on alternative, non-laboratorial parameters associated with an increased risk of developing MetS—for determining the presence of MetS among the population of central Mexico.

MATERIALS AND METHODS

Study design and participants

This was a cross-sectional study of males and non-pregnant females ≥ 18 years of age, recruited in June 2012 – August 2013, from the City of Puebla, State of Puebla, an urban area in Mexico's central region. The participants were attending the IMSS Family Medicine Clinic #2 and had come in for either a normal check-up (well visit) or were accompanying a family member, who was also eligible to attend the clinic. Due to IMSS policies, a possible 135 000 individuals can attend this clinic. For this study, individuals with or without diabetes could be included; those excluded had an incomplete clinical history or were suspected of having hypo- or hyperthyroidism, pituitary adenoma, rheumatoid arthritis, lupus, or any acute or chronic illness that would interfere with the analysis.

Clinical characterization

Participants were clinically evaluated according to a standardized protocol that included personal and family history. With the subjects in light clothing and without shoes, height and weight were determined using the Tanita Body Composition Analyzer (Model TBF-215, Tanita, Tokyo, Japan). BMI was calculated as weight/height² (kg/m²). Subjects were classified according to established criterion (²¹21. Ministry of Health of Mexico. Norma Oficial Mexicana NOM-015-SSA2-1994, Tratamiento y control de la diabetes mellitus en la atención primaria. Diario Oficial de la Federacion 2010. Available from: http://www.salud.gob.mx/unidades/cdi/nom/015ssa24.html Accessed 9 November 2018.
http://www.salud.gob.mx/unidades/cdi/nom... ) as normal weight (BMI: < 24.9 kg/m²), overweight (BMI: 25 – 29.9 kg/m²), or obese (BMI > 30 kg/m²). After resting for at least 5 minutes and in a seated position, each patient's blood pressure was taken on the right arm with a mercury-column sphygmomanometer or an electric blood pressure monitor with an appropriately sized cuff. The data are the average of two physician-obtained measurements.

Biochemical assays

Following a 10 – 12 hr overnight fast, whole blood samples were collected from the antecubital vein into a BD Vacutainer^® K2 (Becton, Dickinson & Co., Franklin Lakes, New Jersey, United States) containing ethylenediaminetetraacetic acid or BD Vacutainer^® Serum separator tubes with separator gel. The samples were kept at room temperature to allow clotting. The serum fraction was recovered and frozen at −20 °C until use. Samples were used for the following endpoints: fasting plasma glucose, insulin, glycated hemoglobin (HbA1c), total cholesterol, high-density lipoprotein, low-density lipoprotein, and triglycerides. An additional blood sample was obtained 2 hours after oral glucose administration (75 g) to determine tolerance. Fasting plasma glucose and oral glucose tolerance were determined, in duplicate, using the enzymatic method/spectrophotometric glucose oxidation (Beckman Coulter Inc., Brea, California, United States). The HbA1c levels were determined by the turbidimetric inhibition immunoassay. Insulin levels were determined by an automated immunoassay (Beckman Coulter Inc.). To determine total cholesterol, high-density lipoprotein, low-density lipoprotein, and triglycerides, serum samples were sent to the Central Laboratory of the Multidisciplinary Research Group, which is part of IMSS.

MetS definitions

To classify subjects as normal (MetS–) or having Metabolic Syndrome (MetS+), three definitions were used. The Harmonized definition (²²22. Alberti K, Eckel RH, Grundy SM, Zimmet PZ, Cleeman JI, Donato KA, et al. Harmonizing the metabolic syndrome. Circulation. 2009;120(16):1640–5. doi: 10.1161/CIRCULATIONAHA.109.192644
https://doi.org/10.1161/CIRCULATIONAHA.1... ) requires three of the following five criteria: (i) waist circumference: ≥ 90 cm for men and ≥ 80 cm for women; (ii) triglycerides: ≥ 150 mg/dL or drug treatment for elevated triglycerides; (iii) high-density lipoprotein: <40 mg/dL for men and < 50 mg/dL for women or drug treatment for high-density lipoprotein levels; (iv) hypertension: systolic blood pressure ≥ 130 mmHg or diastolic blood pressure ≥ 85 mmHg, or antihypertensive drug treatment; or (v) impaired glucose tolerance: fasting plasma glucose ≥ 100 mg/dL or drug treatment for elevated glucose. The National Cholesterol Education Program Adult Treatment Panel III (ATPIII) definition (²³23. Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults. Executive summary of the third report of the national cholesterol education program. JAMA. 2001;285(19):2486–97. doi: 10.1001/jama.285.19.2486
https://doi.org/10.1001/jama.285.19.2486... ) requires three of the following five criteria: (i) waist circumference: ≥ 102 cm for men or ≥ 88 cm for women; (ii) triglycerides: ≥ 150 mg/dL; (iii) high-density lipoprotein: < 40 mg/dL for men and < 50 mg/dL for women; (iv) hypertension: systolic blood pressure ≥ 130 mmHg or diastolic blood pressure ≥ 85 mmHg; or (v) impaired glucose tolerance: fasting plasma glucose ≥ 110 mg/dL or diabetes. The International Diabetes Federation (IDF) definition (²⁴24. Alberti G, Zimmet P, Shaw J, Grundy SM, eds. The Internacional Diabetes Federation (IDF) consensus worldwide definition of the metabolic syndrome. Proceedings of the workshop of the IDF Task Force on Epidemiology and Prevention. Available from: https://www.idf.org/e-library/consensus-statements/60-idfconsensus-worldwide-definitionof-the-metabolic-syndrome.html Accessed 9 November 2018.
https://www.idf.org/e-library/consensus-... ) requires central adiposity (waist circumference: ≥ 90 cm for men, ≥ 80 cm for women, or BMI > 30 kg/m²) and two of the following four criteria: (i) triglycerides ≥ 150 mg/dL or using triglycerides lowering medication; (ii) high-density lipoprotein: < 40 mg/dL for men, < 50 mg/dL for women, or using high-density lipoprotein augmenting medication; (iii) hypertension: systolic blood pressure ≥ 130 mmHg, diastolic blood pressure ≥ 85 mmHg, or treatment for hypertension; or (iv) impaired glucose tolerance: fasting plasma glucose ≥ 100 mg/dL or diagnosed with diabetes.

Study instrument

The ESF-I questionnaire was designed with 15 questions/criteria to assess key non-laboratorial measures that were associated with increased risk or prevalence of MetS. The study instrument is available upon request from the corresponding author. This questionnaire focused on anthropometric measurements and alternative clinical data, including erectile dysfunction, history of macrosomia, postprandial somnolence, Acanthosis Nigricans, hypertension, age, family history of T2D, blurry vision, polydipsia, polyuria, polyphagia, and dietary intake. Some questions were developed after searching for key factors associated with insulin resistance and MetS, whereas others were developed based on the knowledge and experience of a five-person panel. Some questions contained sex-specific cutoff-values for people in central Mexico. The final question was sex based, where a positive result for any of the sub-questions was considered a positive result for the whole question. Each positive response was given 1 point and summed to obtain a total value out of a maximum score of 15.

One of two doctors applied the questionnaire, reading the questions for the participant and taking measurements of the waist, hip, and height. When, a subject did not understand the question, this was noted, and depending on the question, the doctor gave a standard response. Worth noting is that the questions that provoked the most concern were those on consumption ≥ 300 grams of carbohydrates and Acantosis Nigricans. The total time required ranged from 10 – 15 minutes. Participants were then scheduled for a fasting blood-draw within 1 – 2 weeks.

Statistical analysis

Statistical analyses were performed using the IBM SPSS Statistics software, version 21 (SPSS Inc., an IBM company, Chicago, Illinois, United States). Normality of continuous variables was determined using Kolmogorov-Smirnov test. Levene's test was used to examine difference in the variances of continuous variables. Associations were determined by calculating Spearman coefficient rho (ρ) (²⁵25. Mukaka MM. Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Med J. 2012;24(3):69–71.). Cohen´s kappa (κ) was calculated to determine inter-assay agreement. Receiver-operating characteristic curve analysis was used to determine specificity and sensitivity between the ESF-I questionnaire and MetS. The area under the receiver-operating characteristic curve (AUC) was calculated using the method described by Hanley and McNeil (²⁶26. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi: 10.1148/radiology.143.1.7063747
https://doi.org/10.1148/radiology.143.1.... ). Using the sensitivity and specificity, the Youden's index (sensitivity + specificity – 1) was calculated and the highest score was considered the optimal cutoff value. Logistic regression was used to determine association (Odds Ratio [OR] and 95% confidence intervals [95%CI]) between the ESF-I questionnaire and MetS. Internal consistency was measured by Cronbach's α (α = 0.766). A post-hoc analysis was performed to determine if an acceptable power had been achieved. Using the MedCalc^® Version 13.3.3.0 software (Medcalc, Ostend, Belgium), the sample size of 72 was determined using AUC of 0.80 with a positive to negative ratio of 1, assuming an alpha and beta of 0.01. P values < 0.05 (two-tailed) were considered statistically significant. Results are expressed as the mean ± standard deviation.

Ethics. All participants provided written, informed consent, in accordance with the Declaration of Helsinki. To ensure the participant's confidentiality, the participant's IMSS identification number was used for any labs. Afterwards, the identification number was recoded to a Protocol Association number. The study protocol was approved by the Scientific Research and Ethics Committees of IMSS (IMSS Committee: Registry number: R-2014-2102-1).

RESULTS

Participants’ characteristics

Initially, 242 participants from 19 – 77 years of age were approached for this study (96 males and 136 non-pregnant females); however, 10 were excluded for incomplete laboratory reports. Thus, the study cohort consisted of 232 subjects, of which 26.3% were positive for T2D (Table 1); 5 (0.46%) were morbidly obese; 50.4% had attended or graduated from a university; 15.0% had completed a high school education; 9.4% had completed secondary (middle) school, whereas 25.2% had attained only a primary school education.

It is worth noting that most definitions of MetS do not require the presence of T2D to confirm it. Depending on which of the three definitions of MetS was used, the portion of T2D subjects negative for MetS ranged from 11% – 22%. The Harmonizing definition and the ATPIII definition had 93.1% agreement (κ = 0.860, P < 0.001), whereas the Harmonizing and IDF had only 90.8% agreement (κ = 0.800, P < 0.001). Interestingly, the IDF and the ATPIII had 95.2% agreement (κ = 0.903, P < 0.001). Using the three different MetS definitions, from 53.8% – 60.8% of the cohort was positive for MetS. As expected, the MetS+ group's age was significantly higher for the Harmonizing, ATPIII, and IDF definitions (P < 0.001). For the five criteria of MetS, each was significantly higher, independent of the MetS definitions used (P < 0.001). The anthropometric and biochemical characteristics are shown in Table 1. The average ESF-I questionnaire score was significantly higher in the MetS+ group for each definition (Harmonizing: 8.27 ± 3.10 versus 4.28 ± 2.35; ATPIII: 7.93 ± 3.17 versus 4.11 ± 2.30; and IDF: 8.16 ± 3.08 versus 4.23 ± 2.4; P < 0.001).

ESF-I correlates with MetS and its components

For the components of MetS, there was a moderate positive correlation between the ESF-I questionnaire and waist circumference (ρ = 0.611, P < 0.001, Figure 1) and a weak positive correlation for systolic blood pressure (ρ = 0.268, P < 0.001), diastolic blood pressure (ρ = 0.149, P = 0.021), triglycerides (ρ = 0.301, P < 0.001), and fasting plasma glucose (ρ = 0.319, P < 0.001). There was a weak negative correlation between the ESF-I questionnaire and high-density lipoprotein (ρ = −0.278, P < 0.001). Interestingly, there was a moderate positive correlation between the ESF-I questionnaire and MetS (Harmonizing definition: ρ = 0.591, ATPIII definition: ρ = 0.555, and IDF definition: ρ = 0.580, P < 0.001).

Receiver-operating characteristic curve analysis was used to compare the effectiveness of the ESF-I questionnaire for determining MetS (Figure 2). For the Harmonized definition, the AUC was 0.841 (95%CI: 0.790 – 0.892, P < 0.001) and analysis of the curve resulted in a suggested cutoff value of 7 (Youden index = 0.560, sensitivity = 72.8%, and specificity = 83.2%). For the ATPIII definition, the AUC was 0.827 (95%CI: 0.744 – 0.880, P < 0.001) and also a cutoff value of 7 was determined (Youden index = 0.520, sensitivity = 67.4%, and specificity = 84.6%). For the IDF definition, the AUC was 0.836 (95%CI: 0.785 – 0.887, P < 0.001) and a suggested cutoff value of 7 (Youden index = 0.520, sensitivity = 69.2%, and specificity = 81.4%).

ESF-I score indicates increased risk for MetS

Since the proposed cutoff value of 7 was determined for each MetS definition, the cohort was re-categorized. There was strong agreement between the ESF-I questionnaire and MetS, independent of the definition used (Harmonizing definition: test accuracy = 77.6%, positive prediction value (PPV) = 83.5%, negative prediction value (NPV) = 72.4%, κ = 0.554; ATPIII definition: test accuracy = 74.1%, PPV = 87.2%, NPV = 62.6%, κ = 0.489; and IDF definition: test accuracy = 74.6%, PPV = 82.6%, NPV = 67.5%, κ = 0.495, P < 0.001). For the Harmonizing definition, the ESF-I questionnaire was strongly predictive of MetS (OR = 13.2, 95%CI: 7.0 – 25.1, P < 0.001, Table 2), with similar results for the ATPIII definition (OR = 11.3, 95%CI: 5.8 – 22.2, P < 0.001) and for the IDF definition (OR = 9.8, 95%CI: 5.27 – 18.3, P = < 0.001). There was no difference when adjusting for gender (Model 1) or gender and age (Model 2).

DISCUSSION

MetS represents a significant risk marker for the development of T2D and cardiovascular disease. However, there is an overwhelming economical, medical, and social imperative for the early identification of individuals with MetS. Early identification allows lifestyle interventions and treatments, which many reports have shown to prevent the development of T2D and cardiovascular disease (²⁷27. Brotons C, Moral I, Fernández D, Puig M, Bonacho EC, Muñoz PM, et al. Estimation of lifetime risk of cardiovascular disease (IBERLIFERISK): a new tool for cardiovascular disease prevention in primary care. Rev Esp Cardiol. 2018;pii: S1885–57(18):30296–2. doi: 10.1016/j.rec.2018.05.028
https://doi.org/10.1016/j.rec.2018.05.02... , ²⁸28. Penn L, Rodrigues A, Haste A, Marques MM, Budig K, Sainsbury K, et al. NHS Diabetes Prevention Programme in England: formative evaluation of the programme in early phase implementation. BMJ Open. 2018;8(2):e019467. doi: 10.1136/bmjopen-2017-019467
https://doi.org/10.1136/bmjopen-2017-019... ). Taking into consideration the cost of diagnosing the components of MetS, this study demonstrates that using the ESF-I questionnaire, based on alternative signs of MetS, can specifically identify MetS-positive individuals.

Currently, IMSS spends 30% of its non-labor budget treating complications in 15% of patients suffering from T2D, heart disease, kidney failure, and other noncommunicable and degenerative diseases, such as MetS (²⁹29. González Anaya JA, García Cuéllar R. The transformation of the Mexican Social Security Institute (IMSS): progress and challenges. Health Systems & Reform. 2015;1(3):189–99. doi: 10.1080/23288604.2015.1061096
https://doi.org/10.1080/23288604.2015.10... ). As postulated by Gonzalez Anaya and Garcia Cuellar, by using early detection strategies and increased patient adherence, as well as implementing improved preventive efforts, many of these complications could be mitigated (²⁹29. González Anaya JA, García Cuéllar R. The transformation of the Mexican Social Security Institute (IMSS): progress and challenges. Health Systems & Reform. 2015;1(3):189–99. doi: 10.1080/23288604.2015.1061096
https://doi.org/10.1080/23288604.2015.10... ). Thus, a new strategy for prevention was designed to aid in early detection. Here, we have demonstrated that the ESF-I questionnaire was specific for MetS detection. The PPV suggests that the questionnaire can identify these individuals, which could improve treatment outcomes for patients with newly identified MetS.

The cost associated with MetS diagnosis makes large-scale screening prohibitive. But with a questionnaire such as this one, there is no need to analyze blood samples or use specialized staff. In addition, with this questionnaire high-risk individuals can be identified during a consultation or by an alternate route, such as a website, mobile app, email, and postal mail (still widely used in rural areas of Mexico). However, the ability to implement the questionnaire without the aid of trained staff is a concern. Here, most of the participants had a high school education, but in rural areas that cannot be expected. Moreover, most patients were unfamiliar with Acanthosis Nigricans, which suggests that photos should be added to the questionnaire for clarification. Nonetheless, this questionnaire could be used to identify apparently healthy individuals that are at a higher risk of developing cardiovascular disease, T2D, or other metabolic comorbidities. This could reduce the number of consultations required for truly healthy individuals; reduce the expense in the number of consultations due to unnecessary laboratory work and appointments; and save time and money by providing earlier detection. However, when MetS is suspected, this questionnaire is not a replacement for doctor consultations or future medical appointments. Its sole purpose is to identify possible positive individuals.

Regarding the structure of the questionnaire, each question was based on the fact that each category was associated with an increased risk of MetS: family history of T2D (OR = 1.44); advanced age (OR = 1.81); obesity (OR = 2.62) (³⁰30. Kim I, Song Y-M, Ko H, Sung J, Lee K, Shin J, et al. Educational disparities in risk for metabolic syndrome. Metab Syndr Relat Disord. 2018;16(8):416–24. doi: 10.1089/met.2017.0170
https://doi.org/10.1089/met.2017.0170... ); waist-to-height ratio (OR = 1.13); and waist circumference (OR = 1.95) (¹⁴14. Sagun G, Oguz A, Karagoz E, Filizer AT, Tamer G, Mesci B. Application of alternative anthropometric measurements to predict metabolic syndrome. Clinics. 2014;69(5):347–53. doi: 10.6061/clinics/2014(05)09
https://doi.org/10.6061/clinics/2014(05)... ). We also evaluated symptoms associated with the components of MetS, such as Acanthosis Nigricans (¹¹11. Choudhary SV, Saoji V, Singh A, Mane S. Acanthosis nigricans: a clinical marker of insulin resistance. Int J Res Dermatol. 2017;3(2):161–7. doi: 10.18203/issn.2455–4529.IntJResDermatol20171497
https://doi.org/10.18203/issn.2455–4529.... ), as an indicator of insulin resistance (OR = 3.35); erectile dysfunction (¹²12. Chaudhary RK, Shamsi BH, Tan T, Chen H-M, Xing J-P. Study of the relationship between male erectile dysfunction and type 2 diabetes mellitus/metabolic syndrome and its components. J Int Med Res. 2016;44(3):735–41. doi: 10.1177/0300060515623122
https://doi.org/10.1177/0300060515623122... ), associated with elevated triglyceride levels (OR = 1.98); and a history of fetal macrosomia (³¹31. Segovia Vázquez MR. Obesidad materna pregestacional como factor de riesgo para el desarrollo de macrosomía fetal. Rev Nac. 2014;6:8–15.), associated with obesity and gestational diabetes (OR = 2.56). Even though the odds ratios for each category were different, each question was assigned a value of 1. The sensitivity and specificity of the ESF-I questionnaire could be increased by adjusting the point values. Moreover, we did not consider any biological interactions, which could affect the point distributions. Therefore, current studies are underway to determine any biological interactions or clustering to improve the accuracy of the ESF-I questionnaire.

In Mexico, the population's ethnic diversity includes Native American, European, and African, and can differ by geographic area. Unlike other parts of the world where European colonialism resulted in a significant decline in the native population, about 10% of the population of Mexico is indigenous; therefore, the majority of Mexicans are mestizos (³²32. Moreno A, Sandoval K. Diversidad genómica en México: Pasado indígena y mestizaje. Cuicuilco. 2013;20(58):249–75.). Moreover, Moreno-Estrada and colleagues demonstrated that ethnic variation can affect disease development and prevalence (³³33. Moreno-Estrada A, Gignoux CR, Fernández-López JC, Zakharia F, Sikora M, Contreras AV, et al. The genetics of Mexico recapitulates Native American substructure and affects biomedical traits. Science. 2014;344(6189):1280–5. doi: 10.1126/science.1251688
https://doi.org/10.1126/science.1251688... ). In the State of Puebla, the study population was a mix of indigenous and European (³²32. Moreno A, Sandoval K. Diversidad genómica en México: Pasado indígena y mestizaje. Cuicuilco. 2013;20(58):249–75.); that being said, it represents the ethnicity of central Mexico. Therefore, since this questionnaire was only applied to central Mexicans and due to the great ethnic variations, we speculate that the results are specific to this geographic area.

This questionnaire is currently being tested in other areas of Mexico; however, ethnic-specific modifications are probably needed for successful application outside of Puebla, and especially in other countries of Latin America. With respect to non-Latin American countries, it remains unlikely that the ESF-I questionnaire could be utilized. As seen with the Japanese subjects, the JAMRISC questionnaire yielded superior identification for MetS than did the FINDRISC (¹⁸18. Tan C, Sasagawa Y, Kamo K-i, Kukitsu T, Noda S, Ishikawa K, et al. Evaluation of the Japanese Metabolic Syndrome Risk Score (JAMRISC): a newly developed questionnaire used as a screening tool for diagnosing metabolic syndrome and insulin resistance in Japan. Environ Health Prev Med. 2016;21(6):470. doi: 10.1007/s12199-016-0568-5
https://doi.org/10.1007/s12199-016-0568-... ). However, future studies are required to modify questions with region-specific cutoff values.

Limitations. This study has a few limitations. First, principle component analysis was not performed to group the questions into categories. This could affect the distribution of points. However, this initial study determined that with the crude result, the ESF-I questionnaire was specific and sensitive for determining MetS. Second, the differences between female and male criteria could influence the result; however, we used gender-specific criteria to minimalize any gender bias. Third, as mentioned above, there are many non-laboratorial signs and symptoms that could have been used. We selected ones based on risk levels, reviews of the literature, and personal experiences. However, including more questions could improve the questionnaire's specificity and sensitivity; this idea is currently being investigated.

Conclusions

The ESF-I questionnaire was able to identify individuals with MetS with exceptional sensitivity among the population of central Mexico. This questionnaire can lead to earlier diagnoses, reduce the number of consultations, and lower costs with easier application.

Acknowledgements

The authors would like to express their gratitude to the participants of this study; to Cristhian G. Balderas Carreto (UMF-2- Instituto Mexicano de Seguridad Social, Puebla, Puebla, Mexico) and Blanca G. Baez-Duarte (Benemérita Universidad Autónoma de Puebla, Puebla, Puebla, Mexico) for their aid in collecting data; and to Ricardo Villegas-Tovar (Benemérita Universidad Autónoma de Puebla, Puebla, Puebla, Mexico) for his help finding information for the preparation of the manuscript.

REFERENCES

Publication Dates

History