INVESTIGACIÓN ORIGINAL ORIGINAL RESEARCH
Aceptabilidad y fiabilidad de un cuestionario sobre el comportamiento de riesgo de los adolescentes aplicado con el apoyo de audio y computadora
Juan Pablo GutiérrezI, II; Pilar Torres-PeredaIII,1
IDivision of Health Economics, National Institute of Public Health (INSP), Cuernavaca, Mexico
IIHealth Policy Unit, London School of Hygiene and Tropical Medicine, London, United Kingdom
IIICenter for Health Systems Research, INSP, Cuernavaca, Mexico
OBJECTIVE: To test acceptability and reliability of audio computer-assisted self-interviewing (ACASI) system administered at the household level to gather data on risk behaviors among adolescents in Mexico.
METHODS: Answers to sensitive behavior-related questions obtained using ACASI systems were compared with answers to the same questions obtained through face-to-face interviews (FFIs). Focus groups were conducted to explore ACASI acceptability among adolescents and its feasibility in obtaining better self-reported data on sensitive behaviors.
RESULTS: ACASI was adolescents' preferred method for reporting risk behaviors, particularly sexual behavior, and did not pose greater challenges to respondents versus FFIs. Also, more risk behaviors were reported through ACASI versus FFIs.
CONCLUSION: The use of ACASI systems in disadvantaged households to obtain data on adolescent risk behavior is not only feasible but may also improve data quality in the case of complex questionnaires, compared with FFIs, and should therefore be considered as an alternative survey method.
Key words: Confidentiality, risk-taking, adolescent behavior, Mexico.
OBJETIVO: Evaluar la aceptabilidad y fiabilidad de un sistema de autoentrevista asistido por audio y computadora (ACASI) aplicado en los hogares para recopilar información sobre el comportamiento de riesgo de los adolescentes en México.
MÉTODOS: Se compararon las respuestas obtenidas mediante el sistema ACASI y entrevistas cara a cara (FFI) sobre aspectos sensibles del comportamiento. Se analizó en grupos focales la aceptabilidad de ACASI para los adolescentes y la factibilidad de obtener mejor información en los autoinformes sobre aspectos sensibles de su conducta.
RESULTADOS: Los adolescentes prefirieron el sistema ACASI para informar sobre conductas de riesgo, en particular las relacionadas con el sexo, y no constituyó un mayor reto para los entrevistados en comparación con las FFI. Se informaron más comportamientos de riesgo mediante ACASI que en las FFI.
CONCLUSIONES: El empleo del sistema ACASI para obtener información sobre las conductas de riesgo de adolescentes en hogares con dificultades económicas no solo es factible, sino que en el caso de cuestionarios complejos puede mejorar la calidad de la información en comparación con las FFI y debe considerarse como un método alternativo de encuesta.
Palabras clave: Confidencialidad, asunción de riesgos, conducta del adolescente, México.
In Mexico, high rates of health-risk behaviors or their consequences have been documented among adolescents. For example, various studies have reported high seroprevalence of herpes type 2 (HSV-2) in specific populations, and official statistics show a high pregnancy rate among adolescents (1-3). These data suggest that risk behaviors are highly prevalent among Mexican adolescents.
Nevertheless, limited information exists regarding the magnitude and distribution of these behaviors throughout the country. A major limitation to gathering representative data from this population, which includes both in-school and out-of-school adolescents, is related to complications in obtaining sensitive information (i.e., reporting of risk behaviors) from the respondents in the household setting-the optimal sampling unit.
The literature suggests that, in general, individuals tend to underreport behaviors that are culturally sensitive in face-to-face interviews (FFIs), and this problem increases if the interview is conducted in an environment with low confidentiality, as could be the case in the study participants' households, where these type of behaviors are normally hidden (4, 5). The use of self-administered questionnaires as an alternative to FFIs ensures better household confidentiality but poses other constraints, such as the need for respondents to follow complicated questionnaire instructions independently, and the inherent limitations to the participation of illiterate individuals.
The use of information technology such as audio computer-assisted self-interviewing (ACASI) systems significantly reduces these complications, while maintaining the advantages of a selfadministered instrument (6-9).
Even so, utilizing computerized questionnaires for surveys with a large sample size, especially in a disadvantaged population, can be a challenge. The use of a computer must not only be acceptable to both the study participants and their families, but also logistically functional. Some researchers have suggested that the use of ACASI systems in populations with limited formal education is inappropriate because it can pose great challenges for low-educated individuals (10, 11).
This report presents the results of the first qualitative and quantitative evaluation of an ACASI system in a large, household-based survey, using data collected in a national survey in poor households throughout Mexico.
MATERIALS AND METHODS
During the last quarter of 2004, with-in the context of an evaluation of a national-level poverty alleviation program, a questionnaire on risk behaviors was administered to a sample of adolescents between the ages of 15 and 21 years in 157 towns distributed throughout 13 of the 32 states in Mexico.
The questionnaire was based on a previously conducted survey and contained questions previously tested in the study population. It was programmed in Questionnaire Development System Version 2.1 for Windows (Nova Research, Bethesda, MD, USA), a commercial software package for the development of ACASI systems.
The computer-aided questionnaire was designed to present questions visually (on the computer screen) and through a narrated voiceover heard through the use of headphones. The audio version of the questions was narrated by a professional female radio announcer from a popular music radio station, and recorded at a professional studio. The use of an audio recording ensured that the questions were presented under standardized conditions throughout the study sample, and helped maintain the survey participant's confidentiality, even if the interview was conducted in his/her household, by requiring the use of headphones. Responses were collected using a touch-screen computer monitor to facilitate participants' navigation of the system.
Three techniques were used to evaluate the acceptability and reliability of the survey instrument. The first involved a series of questions for the interviewer, who, once the self-administration of the questionnaire was complete, reported the number of interruptions experienced during the interview, and the number of questions asked by the survey participant, as well as his/her apparent comprehension of the questions, and computer literacy.
The second technique involved a series of focus groups conducted among Mexican adolescents, most of whom (58%) had been previously surveyed on risk behaviors using the ACASI system. The non-ACASI users were included to help review and assess the clarity of the ACASI questionnaire. In addition to evaluating the study participants' understanding of the ACASI questions and identifying any difficulties they may have had in answering them, the focus groups explored ACASI users' perceptions about the ACASI system, among other topics. Ten different focus groups were conducted (four in the central area of the country and six in the south). Each focus group, which lasted for 2 hours, was conducted by a trained anthropologist and observed by a social scientist who took notes on participants' nonverbal expressions. Average attendance per focus group was eight adolescents from either sex. Group discussions were recorded on magnetic tapes, with the subjects' consent, and subsequently transcribed to a Microsoft Word document. The information was then coded and analyzed utilizing the ATLAS-ti program (ATLAS.ti Scientific Software Development GmbH, Berlin, Germany).
In the third technique, using the ACASI system, survey participants were asked questions identical to those asked in previous FFIs, allowing for a comparison of the responses provided to the face-to-face interviewers to those entered directly into the computer through the ACASI application. A total of 3 060 males and 3 228 females were interviewed using both techniques, and the proportions of adolescents reporting risk behaviors were compared by interview technique, using a mean difference test (adjusting for cluster correlation) as well as odds ratios. The odds ratios were estimated using logistic regression models that incorporated survey participants' individual identification information, controlling for age and sex and considering paired observations.
The study design, questionnaire, and informed consent forms were approved by the institutional review boards of the National Institute of Public Health (INSP) in Mexico.
A total of 6 288 adolescents, with an average age of 17 years (data not shown), were interviewed through the ACASI system; slightly more than half of the subjects were female (51%).
According to the interviewers (based on their survey evaluation), 93% of the ACASI survey participants did not appear to have problems using the computers, and only 10% demonstrated difficulty in understanding the questions on the questionnaire (data not shown). The evaluation was conducted by the interviewers and was based on the type of questions participants asked while they answered the questionnaire. On average, the survey participants only called the ACASI administrators once with questions regarding the questionnaire, and only 10% of the participants were interrupted by their relatives or friends while answering the questionnaire (data not shown).
A total of 86 adolescents from 10 locations participated in the focus groups.
At the time of their participation in the focus groups, 73% of the youths were single and 19% were married (living together or separated), with the remaining 8% living in a civil union. In terms of occupations, 49% were students, 20% were workers, and 24% were house-keepers (data not shown). Of these 86 participants, 58% had participated in the ACASI survey (as mentioned above, non-ACASI participants were also included in the focus group studies to help determine if the computer survey questions were understandable). Additional details (e.g., the focus group study locations and number of participants) can be found in Table 1.
As indicated in the focus group responses below, the survey participants viewed the ACASI system as an attractive and novel technique that gave them the opportunity to express themselves freely and easily while maintaining their confidentiality:
[Informant 1:] . . . [I]t's the first time that they [provide us] with a computer [for answering] questions that they no longer ask us directly. . . .
[Informant 2:] As if you were chatting [online] . . . .
[Informant 3:] You can answer more freely . . . .
[Informant 4:] . . . [W]ithout lies.
[Informant 1:] It was very fun.
29 November 2004
[Informant 1:] . . . [S]aying the same thing [in the FFIs] makes you feel ashamed or embarrassed . . . doesn't it?
[Informant 2:] . . . [Using the ACASI system] I was hearing the questions.
[Informant 3:] Hearing and answering I expressed myself more freely.
-San Cristóbal de las Casas, Chiapas,
15 November 2004
The participants repeatedly described the decreased self-consciousness and increased trust in confidentiality that they experienced when answering the questionnaire on the computer. In San Cristóbal de las Casas (focus group study no. 8, 15 November 2004), one participant said the ACASI system had provided him with the first opportunity to be honest in a survey of this type. Similarly, adolescents in Coatepec, Veracruz, said that listening to the survey questions using headphones and answering them on the computer had allowed them to provide honest answers for the first time in response to questions about topics such as tobacco consumption, sexual behavior, and religion. They also said the direct (face-to-face) interview method posed enormous barriers to confidentiality and disclosure, and that these barriers were eliminated through the use of the ACASI system.
[Informant 1:] [Responding to the survey through] a computer is good, because you feel [less self-conscious versus face-to-face interviews], right?
[Informant 2:] With [the face-to-face interviewers], since you do not know them, [you feel self-conscious and distrustful].
[Informant 3:] Like, for example, [when the face-to-face interviewers] ask me, "Have you had any sexual relations?" ... I am [not] going to answer that! But on the computer, I would answer.
29 November 2004
[Informant 1:] We answer more freely . . . [using the computer versus the face-to-face interviews, in which], a lot of the time, we invent everything.
[Informant 2:] For example, on the topic of religion . . . .
[Informant 3:] Exactly.
[Informant 1:] . . . [A]lcoholism and drug addiction, those are the [topics about which] you [tend to] feel a fear of family, because, for example, we are good, right? And that's when the problem starts [when they acknowledge their behavior with their family].
29 November 2004
Some survey participants said that they had never used a computer before and/or that their reading and writing capability was low. Nevertheless, the large majority of participants said that it was easy to understand the survey questions as well as the method for responding to them.
[Informant 1:] Sometimes I read it, and sometimes I listened to it. If it was really long, well, I listened. . . . If it was short, well, I read it.
-San Cristóbal de las Casas,
Chiapas, 16 November 2004
Despite its overall appeal, however, the ACASI system does have limitations, especially for females with a low level of education and/or from the more rural localities, who explained that when they did not understand a question they simply answered "No."
Table 2 shows the proportion of adolescents that reported risk behaviors through the ACASI system versus the FFIs. On average, the FFIs were carried out three months before the ACASI.
As shown in the table, for the majority of variables, risk-behavior reporting was consistently greater in ACASI compared to FFIs. Males, in particular, reported higher rates of sexual activity and less use of condoms in the ACASI system versus the FFIs. Among females, the percentage of those who reported being sexually active was also greater via the ACASI system versus FFIs, but there was no difference in their reported use of condoms. Among both males and females, the percentage of participants that reported smoking and drinking was significantly greater in responses reported by means of the ACASI system versus the FFIs.
Table 3 presents the odd ratios for reporting risk behaviors by interview technique. For this analysis, surveys conducted via the ACASI system were coded as 1, with any odd ratios above 1 indicating a higher level of reporting, and any odd ratios below 1 indicating a lower level of reporting. Consistent with the differences in the respective proportions of survey participants reporting various risk activities mentioned above, significantly more sexual activity, drinking, and current smoking and less condom use was reported in the ACASI survey versus the FFI survey.
Overall, a comparison of the results obtained by means of the ACASI system with those obtained by FFIs, and the testimonies of the survey participants in the subsequent focus groups, appear to confirm previous findings that in FFIs, particularly those conducted in the home, risk behaviors are underreported (6, 9, 12, 13).
The results of the current study are also in line with previous studies that indicate the ACASI system is more suitable than FFIs in diverse cultural settings (14, 15). For the population surveyed in the current study, for example, the use of the ACASI system significantly reduced the underreporting experienced with the use of FFIs. The ACASI system also turned out to be logistically viable, suggesting it is an appropriate alternative for surveys collecting sensitive information.
In addition, in contrast to previous analyses that suggested ACASI may not be suitable for low-educated populations (11), the results of the current study suggest that even in a population with little computer experience and/or that is illiterate, the ACASI survey is highly acceptable, and easy to understand and complete. The study participants described ACASI as a fun way to answer survey questions that was highly confidential regarding sensitive topics and preferable to direct (face-to-face) surveys conducted by an interviewer.
It should be noted, however, that the ACASI system is limited by the fact that some participants, when they do not understand a survey question, may respond nonetheless, advancing through the questionnaire rather than resolving their confusion with the system administrators. Nevertheless, the advantages of computer-aided self-interviewing seem to outweigh the potential constraints. One of these advantages is the fact that, according to results of this study, the ACASI system can be used successfully to conduct surveys in populations with limited resources.
Acknowledgments. The authors express their gratitude to all of the survey and focus group participants who provided the information reported here. They also thank Aurora Franco and Manuel Castro, who were responsible for data collection; their field teams; and Ana Emilia Quiroz, who helped with the focus groups.
Funding for this study was provided by the National Institute of Public Health (INSP) in Mexico, a grant from the Mexican Council for Science & Technology, and a grant from the University of California at Berkeley (Dr. Paul Gertler) as part of National Institutes of Health (NIH) research grant R01-HD44146-04.
1. Abraham CD, Conde-González CJ, Cruz-Valdez A, Sánchez-Zamorano L, Hernández-Márquez C, Lazcano-Ponce E. Sexual and demographic risk factors for herpes simplex virus type 2 according to schooling level among Mexican youths. Sex Transm Dis. 2003;30(7):549-55.
2. Conde-González CJ, Lazcano-Ponce E, Hernández-Girón C, Juárez-Figueroa L, Smith JS, Hernández-Ávila M. Seroprevalencia de la infección por el virus herpes simplex tipo 2 en tres grupos poblacionales de la Ciudad de México. Salud Publica Mex. 2003;45 Suppl 5:S608-16.
3. Instituto Nacional de Estadística, Geografía e Informática (MX). México: porcentaje de nacimientos registrados de madres adolescentes (menores de 20 años), 1990-2005 [statistics on the Internet]. México (DF): INEGI; 2006.
4. Leigh BC, Gillmore MR, Morrison DM. Comparison of diary and retrospective measures for recording alcohol consumption and sexual activity. J Clin Epidemiol. 1998;51(2): 119-27.
5. Youn G. Sexual activities and attitudes of adolescent Koreans. Arch Sex Behav. 1996; 25(6):629-43.
6. Des-Jarlais DC, Paone D, Milliken J, Turner CF, Miller H, Gribble J, et al. Audio-computer interviewing to measure risk behaviour for HIV among injecting drug users: a quasi-randomised trial. Lancet. 1999;353(9165):1657-61.
7. Ghanem KG, Hutton He, Zenilman JM, Zimba R, Erbelding EJ. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect. 2005;81(5):421-5.
8. Hewett PC, Mensch BS, Erulkar AS. Consistency in the reporting of sexual behaviour by adolescent girls in Kenya: a comparison of interviewing methods. Sex Transm Infect. 2004; 80 Suppl 2:ii43-8.
9. Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280(5365):867-73.
10. Potdar R, Koening MA. Does Audio-CASI improve reports of risky behaviour? Evidence from a randomized field trial among urban men in India. Stud Fam Plann. 2005;36(2): 107-16.
11. van de Wijgert J, Padian N, Shiboski S, Turner C. Is audio computer-assisted self-interviewing a feasible method of surveying in Zimbabwe? Int J Epidemiol. 2000;29(5):885-90.
12. Metzger DS, Koblin B, Turner C, Navaline H, Valenti F, Holte S, et al. Randomized controlled trial of audio computer-assisted self-interviewing: utility and acceptability in longitudinal studies. HIVNET Vaccine Preparedness Study Protocol Team. Am J Epidemiol. 2000; 152(2):99-106.
13. Murphy DA, Durako S, Muenz LR, Wilson CM. Marijuana use among HIV-positive and high-risk adolescents: a comparison of self-report through audio computer-assisted self-administered interviewing and urinalysis. Am J Epidemiol. 2000;152(9):805-13.
14. Simoes AA, Bastos FI, Moreira RI, Lynch KG, Metzger DS. Acceptability of audio computer-assisted self-interview (ACASI) among substance abusers seeking treatment in Rio de Janeiro, Brazil. Drug Alcohol Depend. 2006;82 Suppl 1:S103-7.
15. Simoes AA, Bastos FI, Moreira RI, Lynch KG, Metzger DS. A randomized trial of audio computer and in-person interview to assess HIV risk among drug and alcohol users in Rio De Janeiro, Brazil. J Subst Abuse Treat. 2006;30(3):237-43.
Manuscript received on 3 December 2007.
Revised version accepted for publication on 22 July 2008.
1 Send correspondence and reprint requests to: Pilar Torres, Researcher, Center for Health Systems Research, INSP, Av. Universidad 655, Sta. María Ahuacatitlán, Cuernavaca, 62508, Mexico; e-mail: firstname.lastname@example.org