On-line version ISSN 1680-5348
Print version ISSN 1020-4989
Rev Panam Salud Publica vol.25 n.5 Washington May. 2009
INVESTIGACIÓN ORIGINAL ORIGINAL RESEARCH
Estudio de validación de la versión en portugués del cuestionario sobre calidad de vida: satisfacción y placer
Carlos ZubaranI, 1; Katia ForestiII; Mariana Rossi ThorellIII; Paulo Roberto FranceschiniIII; Wagner HomeroII
ISchool of Medicine, University of Western Sydney, Australia
IISchool of Medicine, University of Caxias do Sul, Caxias do Sul, Brazil
IIIHospital Santa Casa de Misericordia, Porto Alegre, Brazil
OBJECTIVE: The aim of this study was to demonstrate the validity and reliability of the Portuguese version of the Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q) among substance users in southern Brazil.
METHODS: The original English version of the Q-LES-Q was translated into Portuguese taking into account semantic and conceptual equivalence. The Portuguese versions of the Q-LES-Q and the World Health Organization Quality of Life Instrument (WHOQOL-BREF) were used to assess quality of life among crack cocaine and inhalant users. Factorial and reliability analyses as well as correlation studies were conducted to establish the psychometric properties of the Portuguese version of the Q-LES-Q.
RESULTS: All areas of the Q-LES-Q achieved significant Cronbach's coefficients. Significant Pearson correlation coefficients were obtained between various subscales of the Q-LES-Q and the four dimensions of the WHOQOL-BREF. Correlations were significant when crack and inhalant users were analyzed separately or conjointly. Inhalant users presented significantly higher scores than crack users in the social (p = 0.035) and general (p = 0.005) subscales of the Q-LES-Q.
CONCLUSION: The Portuguese version of the Q-LES-Q possesses sound internal validity. The English version of the Q-LES-Q was adequately adapted to Portuguese. Inhalant users may present lower levels of satisfaction with social life and overall quality of life than crack users. The Portuguese version of the Q-LES-Q is a reliable questionnaire for future research and clinical use with substance users in Brazil.
Key words: Quality of life, reliability and validity, questionnaires, crack cocaine, solvents, Brazil.
OBJETIVO: Demostrar la validez y consistencia de la versión en portugués del cuestionario sobre calidad de vida: satisfacción y placer (Q-LES-Q) en consumidores de drogas del sur de Brasil.
MÉTODOS: Se tradujo la versión original en inglés del Q-LES-Q al portugués tomando en cuenta las equivalencias semántica y conceptual. Se utilizaron la versión en portugués del Q-LES-Q y la escala de calidad de vida de la Organización Mundial de la Salud (WHOQOL-BREF) para evaluar la calidad de vida de personas que consumían basuco (crack) e inhalantes. Se realizaron análisis factoriales y de consistencia y estudios de correlación para establecer las propiedades psicométricas de la versión en portugués del Q-LES-Q.
RESULTADOS: Todas las áreas del Q-LES-Q alcanzaron valores significativos del coeficiente de Cronbach. Se obtuvieron coeficientes de correlación de Pearson significativos entre varias subescalas del Q-LES-Q y las cuatro dimensiones de la WHOQOL-BREF. La correlación fue significativa para los consumidores de basuco y de inhalantes por separado y en conjunto. Los consumidores de inhalantes presentaron puntuaciones significativamente mayores que los consumidores de basuco en las subescalas social (P = 0,035) y general (P = 0,005) del Q-LES-Q.
CONCLUSIONES: La versión en portugués del Q-LES-Q posee una sólida validez interna. La versión en inglés del Q-LES-Q se adaptó adecuadamente al portugués. Los consumidores de inhalantes pueden presentar niveles más bajos de satisfacción con la vida social y la calidad general de vida que los consumidores de basuco. La versión en portugués del Q-LES-Q es un instrumento confiable tanto para investigaciones futuras como para uso clínico en consumidores de drogas en Brasil.
Palabras clave: Calidad de vida, reproducibilidad de resultados, cuestionario, cocaína crocante, solventes, Brasil.
During the latter part of the 20th century, the subjective expressions of patients became a salient topic in health care (1). The field of health status and quality-of-life measurement has been evolving as a formal discipline with structured theoretical foundations and specific methodology for more than 30 years (2). In the late 1980s, some authors adopted health-screening procedures to examine the limitations imposed by the disease process on patients' well-being and social functioning (3, 4). Although the concept of health has been subject to cultural and historical adjustment, according to the World Health Organization health can be defined as a state of complete physical, mental, and social well-being (5). Yet, there is a current tendency to reexamine the conceptual boundaries of health to include elements such as sociocultural conditions as well as factors contributing to mental and physical health, which ideally will transcend the rather circumscribed dichotomy of the health-disease process (6). It is currently acknowledged that efforts to investigate and evaluate quality of life, social functioning, health status, and well-being are valid enterprises within clinical and research contexts (7).
Quality of life has been defined as the perception of the individual about one's position in life in the context of cultural and value systems in relation to one's objectives, beliefs, and expectations (8). Some prefer the concept of "health-related quality of life" to simply quality of life because it focuses on health-in view of the fact that the former construct refers not only to physical, emotional, and social well-being but more precisely to the "impact that health conditions and their symptoms have on an individual's quality of life, and, in the context of healthcare" (9). The measurement of quality of life provides a benchmark against which the impact of disease and different treatments at a personal level can be measured (10). An improvement in health status and quality of life is an important primary outcome in determining therapeutic benefit (11).
Quality of life is also becoming an important clinical and research outcome within the context of drug and alcohol abuse. After all, besides knowingly affecting general health standards, substance abuse produces unstable life patterns in a significant proportion of users as well as impaired interpersonal, social, and professional skills. The association between quality of life and drug abuse has been investigated via a public health perspective by Danish researchers by way of a questionnaire-based survey sent to a representative sample randomly selected from a public database and a cohort registry (12). This study revealed that, of those who returned the questionnaires, opiate users and those who mix alcohol and tranquilizers present the lowest quality-of-life scores.
Quality of life has also been investigated within the context of research on treatment for drug abuse. A recent study found that heroin users under pharmacological maintenance treatment acknowledged a beneficial effect of taking both methadone and buprenorphine in terms of satisfaction with quality of life (13). In the latter study, a specific instrument designed to investigate the quality of life of injection drug users was tested and validated as an effective research tool for this purpose (14).
An additional research instrument was tested among alcohol abusers and endorsed as a valid tool with sound psychometric properties for use in clinical practice as a "diagnosis and management support instrument," which may also "be useful in research for evaluating treatment efficacy" (15). It has been advocated that the effort to assess the impact of drug dependence and its treatment on the quality of life of patients has been insufficient (16). Yet, self-reported information obtained from quality-of-life questionnaires provides valuable information about the burden of treatment that drug-addicted persons experience.
In Brazil, only a limited number of studies investigated quality of life among substance users. A pilot study that used the World Health Organization Quality of Life Instrument (WHOQOL-BREF) revealed that users with low to moderate levels of drug dependence show higher scores in all domains of this questionnaire (17). In a similar study with young smokers at two public Brazilian universities, never-smokers presented higher health-related quality-of-life scores than smokers in all domains (18).
The limited number of studies investigating the interplay between drug dependence and quality of life in Brazil may be due to the limited availability of quality-of-life instruments validated to Portuguese. The widely used and recognized Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q), for instance, has not been tested in Brazil. The Q-LES-Q has been used to investigate different degrees of enjoyment and satisfaction that other instruments cannot detect easily (19). Recently, the Italian version of the Q-LES-Q has been validated in a multicenter study with patients receiving treatment for anxiety disorders (20). The Italian version of the Q-LES-Q proved to be as valid and reliable as the original English version.
Clinicians and policymakers now agree on the importance of measuring quality of life (21). Because of the increment in the number of multinational and multicultural research projects, there has been a proportional increase in the number of studies designed to test and adapt a series of health status instruments to different countries and languages (22). Most questionnaires were originally developed in English-speaking countries. However, the methodological details of translation and adaptation of questionnaires originally developed in English for future use in other countries have received surprisingly little attention. It is now consensus that, when tested across different cultures, measures must not only be well translated linguistically but also be well adapted culturally to guarantee the content validity of the instrument (22, 23).
The main objective of this study is to test in Brazil the Portuguese version of the Q-LES-Q. The original English version of this assessment tool was translated and adapted to Portuguese according to well-established methodologies and then tested among crack and inhalant drug users in southern Brazil in 2005.
MATERIALS AND METHODS
The study investigated two groups of adult volunteers of equal size (n = 50): a group of crack cocaine users and a group of inhalant users. Both groups of drug users were interviewed as inpatient volunteers under psychiatric treatment in specific psychiatric wards of tertiary care psychiatric treatment facilities and higher education training sites. Drug users fulfilled the diagnostic criteria (24) of one of the drug dependence formulations under scrutiny, according to a psychiatric interview conducted during the admission processes by certified psychiatrists who were unaware of this research protocol. Although patients with polydrug dependence were not excluded from the study, they had to satisfy the inclusion criteria of identifying one of the two drugs as (1) the main addictive substance and (2) the one causing the most salient dependence. In both groups of substance users, variable frequencies of nicotine, alcohol, and cannabis use were also observed. Yet, no diagnosis of withdrawal from any of these substances was established during the admission process of volunteers from both groups.
All patients who volunteered to participate in this investigation were assisted via the Brazilian public health system. Taking into account that a substantial fraction of the sample was illiterate or semi-illiterate, all patients completed the questionnaires under minimal guidance from trained examiners, who followed standardized instructional procedures. All volunteers have Portuguese as their mother tongue.
Research volunteers completed a socioeconomic status (SES) scale, which was developed and tested in Brazil (25). This instrument classifies SES into six categories: lower-lower, upper-lower, lower-middle, upper-middle, lower-upper and upper-upper class. The educational background of volunteers was not assessed.
Research volunteers were recruited via word of mouth by members of the research team in the days after their admission process to one of the mental health facilities mentioned above. Patients were invited to participate in this study after achieving a physically and psychologically stable condition as determined by their clinical team in the public mental health facilities. Recruitment announcements did not refer to "healthy" or "normal" controls in order to decrease the potential for underreporting psychiatric and medical illness.
This study was endorsed by the institutional Ethics and Research Committees at the University of Caxias do Sul (UCS), which are regulatory bodies that control all scientific activities developed at different campuses of UCS. All volunteers signed a consent form to declare a voluntary agreement with all procedures implicated in this project. A certificate of confidentiality was guaranteed to all research respondents, whose participation was needed for scientific purposes only. All respondents were treated anonymously.
Research volunteers completed the long form of the Q-LES-Q (19). The Q-LES-Q is a self-report instrument composed of 93 items, with 91 of them grouped into eight summary subscales reflecting satisfaction with physical health, subjective feelings, work, household duties, school, leisure activities, social relationships, and general activities. Two additional items in the general activities subscale measure satisfaction with medication and overall life satisfaction and contentment.
Each of the 93 items is rated on a 5-point scale, which indicates the degree of enjoyment or satisfaction experienced during the week before the assessment. High scores on the Q-LES-Q indicate greater enjoyment or satisfaction. Raw summary scores are calculated as a percentage of the maximum possible score to facilitate comparisons across different areas of functioning. Some investigators have reported their data in terms of raw mean scores, while others have calculated the maximum percentage scores (26). The general activities subscale is frequently used as a short form of the Q-LES-Q (Q-LES-Q-SF). It is composed of 14 items covering a wide array of life issues in addition to the two other items mentioned above.
Research volunteers also completed the short version of the WHOQOL-BREF as a benchmark against which Q-LES-Q was compared. The WHOQOL-BREF is a cross-culturally valid assessment of well-being and quality of life (27). This tool was developed through an international collaboration of various sites working in their own national language. The WHOQOL-BREF collaboration pooled information throughout the project, which facilitated a high level of semantic and conceptual equivalence in the process of generating multilingual instruments (28). Initial analysis of the WHOQOL-BREF-generated data indicated that a four-factor structure best fitted this tool, and the WHOQOL-BREF was therefore developed in the context of four domains of WHOQOL-BREF: physical, psychological, social, and environment (28). The Portuguese version of the WHOQOL-BREF instrument has already been tested and validated in Brazil (17).
Adaptation and translation procedures
The English version of the Q-LES-Q was translated into Portuguese, taking into account semantic, idiomatic, experiential, cultural, and conceptual equivalence between the source and the target instruments (29). Two investigators proficient in both Portuguese and English developed the final Portuguese version of the Q-LES-Q. Each investigator translated and adapted the questionnaires from one language to the other (translation and back-translation). The final adapted version of the instrument was established by a committee of specialists by comparing the translation and the back-translation of each instrument. This committee consisted of professionals fully cognizant of the subject under investigation. Many of them were versed in both languages. The various drafts of both questionnaires, in each language, were progressively improved by using relevant information obtained from a pilot study conducted with a different subset of health care professionals and patients not included in the current research protocol and analysis (8, 30).
The internal consistency of Q-LES-Q subscales was determined via Cronbach's alpha coefficients. The decrease in Cronbach's alpha coefficients at item deletion was computed when pertinent. The correlations between the areas of the Q-LES-Q and the WHOQOL-BREF were examined by using Pearson product-moment correlation coefficients. Independent-samples t tests were used to compare the Q-LES-Q scores of crack and inhalant users. A Mann-Whitney U test was used to compare SES of the two groups. Statistical analysis was conducted with SPSS® software.
The mean ages of crack users and inhalant users were 22.8 and 17.3 years, respectively. The group of crack users consisted of 14% female and 86% male volunteers. The gender distribution among female and male inhalant users was 10% and 90%, respectively.
As demonstrated in Table 1, in terms of SES, most volunteers in the crack (56%) and inhalant (58%) user groups were in the lower-middle category.
Internal structure information
The internal consistency (correlation of the items to the total score) of the Portuguese version of the Q-LES-Q, considering crack and inhalant users conjointly, is shown in Table 2. The reliability analysis of each subscale, including all specific questions, generated a Cronbach's alpha per area of the Q-LES-Q.
All subscales of the Q-LES-Q attained a significant Cronbach's alpha of 0.78 or higher.
Pearson product-moment correlation coefficients
Correlation coefficients were computed among the scores of each subscale of the Q-LES-Q and the four dimensions of the WHOQOL-BREF in the groups of crack and inhalant users taken conjointly. Table 3 illustrates Pearson correlations achieved between various subscales of the Q-LES-Q and the areas of WHOQOL-BREF. Significant (p < 0.01) correlations among all areas of the WHOQOL-BREF and four subscales of the Q-LES-Q were detected: physical (n = 99), psychological (n = 100), social (n = 100), and environment (Q-LES-Q-SF; n = 100). The leisure subscale also attained significant correlations with WHOQOL-BREF areas related to physical health (p < 0.01), psychological factors (p < 0.01), and social relationships (p < 0.05). The WHOQOL-BREF area entitled "environment" attained significant (p < 0.01) correlation with the Q-LES-Q area related to household duties.
A Mann-Whitney U test was conducted to investigate a hypothesis of distinct SES between the two groups of substance users. This nonparametric test revealed no significant difference between crack and inhalant users according to different SES categories (z = -1.10, p = 0.27).
Independent-samples t tests were conducted to evaluate the hypothesis of differential means of quality-of-life maximum percentage scores between crack and inhalant users in the different Q-LES-Q areas. Inhalant users presented significantly higher scores than crack users in the social (t(98) = 1.88; p = 0.035) and general (t(98) = 1.74; p = 0.005) subscales of the Q-LES-Q. No significant difference was detected between inhalant and crack users in any other areas of the Q-LES-Q.
The Portuguese version of the Q-LES-Q proved to have sound psychometric properties. Internal consistency data revealed significant Cronbach's alpha coefficients for all eight areas of the Q-LES-Q. Likewise, the short form of the Portuguese version of the Q-LES-Q presented significant Cronbach's alpha coefficients, which attests to the substantial uniformity of the abbreviated version as well. The internal consistency of each dimension assessed by the Portuguese version of the Q-LES-Q reflects the accuracy of the measurement process. The reliability analysis of data generated in this research was conducted according to parameters described in similar studies developed by other investigators (20, 26).
Various significant correlations among distinct WHOQOL-BREF and Q-LES-Q areas were observed, which demonstrated that both quality-of-life questionnaires are conceptually congruent or equivalent, as expected. The short form of the Portuguese version of the Q-LES-Q presented significant Pearson correlation scores with all areas of the WHOQOL, which indicates constructural congruency between the two quality-of-life assessment instruments. The Q-LES-Q coherently investigates the construct of quality of life as measured uniformly by its items.
The validation study of the Italian version of the Q-LES-Q demonstrated similar results. In that study, a significant correlation was observed between the Italian version of the Q-LES-Q and the work and social adjustment scale (20). Although our study also found significant correlations among specific areas of the Q-LES-Q and the WHOQOL-BREF, the magnitude of the overall correlation between the two instruments may have been limited by a reduced number of responders, particularly in the areas related to school (n = 15) and work (n = 23). The fact that many respondents were unable to complete the subscales related to school and work indicates a substantial impact of substance use on the specific areas encompassed under the broader concept of quality of life. In the group of crack users, 72% and 96% of participants did not work and did not attend any school-related activities, respectively, whereas in the group of inhalant users the percentages were 82% and 74%, respectively.
The reduced number of responders in the latter two subscales is an identifiable limitation in the current study. Yet, in our study, raw summary scores were expressed as a percentage of the maximum possible score to facilitate comparisons across different areas of functioning. Since the minimum score for a given item is 1 rather than 0, the maximum percentage score is calculated as follows: (raw score - minimum possible score)/ (maximum possible score - minimum possible score) (26). Although the impact of crack and inhalant use on educational performance has been well documented (31), crack and inhalant use with a less severe form of addictive disorders (noninpatient samples) could present a smaller impact of substance use on the work and the school subscales.
In addition, there is an unequal gender distribution, with a higher number of male volunteers. Yet, this discrepancy was evenly distributed in both groups. Additional studies encompassing different areas in the country might be necessary to generate a more representative picture of both regional and national health standards among distinct samples of patients and the general population.
This study demonstrates that the original English version of the Q-LES-Q was successfully adapted to Portuguese as methodologically demonstrated here. This study demonstrates that the two questionnaires, both conceptually designed to assess quality of life, present significant constructural correlations and are therefore congruent. The internal consistency of the Portuguese version of the Q-LES-Q, as indicated by the results of the Cronbach's alpha coefficients, was satisfactory, which demonstrates that the instrument coherently investigates the quality of life as measured uniformly by its items. The Portuguese version of the Q-LES-Q was well accepted by all volunteers. They answered the questions with relative ease and rapidity, which demonstrates the ease of use of the instrument. The Portuguese version of the Q-LES-Q constitutes a reliable research instrument for evaluating quality of life among substance users in Brazil.
Acknowledgments. We acknowledge the collaboration of several students in our research group during initial phases of this research. We also acknowledge the contribution of Professor Jean Endicott to the interpretation of results obtained in this research. This study was partially supported by grant 000605-25.00/03-8 from the Secretaria de Ciência e Tecnologia do Estado do Rio Grande do Sul, Brazil.
1. von Engelhardt D. Patient vs. disease in medicine: historical perspectives and contemporary concerns. J Nephrol. 2004;17(4):611-8. [ Links ]
2. Lohr KN. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193-205. [ Links ]
3. Marks A. Aspects of biosocial screening and health maintenance in adolescents. Pediatr Clin North Am. 1980;27(1):153-61. [ Links ]
4. Roeske NC. Quality of life and factors affecting the response to hysterectomy. J Fam Pract. 1978;7(3):483-8. [ Links ]
5. The WHOQOL Group. The development of the World Health Organisation quality of life assessment instrument: The WHOQOL. In: Orley J and Kuyken W, eds. Quality of life assessment: international perspectives. Berlin: Springer-Verlag Telos; 1994. Pp. 41-60. [ Links ]
6. Khushf G. An agenda for future debate on concepts of health and disease. Med Health Care Philos. 2007;10(1):19-27, discussion 29-32. [ Links ]
7. Berlim MT, Fleck MP. "Quality of life": a brand new concept for research and practice in psychiatry. Rev Bras Psiquiatr. 2003;25(4):249-52. [ Links ]
8. Mezzich JE, Ruiperez MA, Perez C, Yoon G, Liu J, Mahmud S. The Spanish version of the quality of life index: presentation and validation. J Nerv Ment Dis. 2000;188(5):301-5. [ Links ]
9. Thompson DR, Yu CM. Quality of life in patients with coronary heart disease-I: assessment tools. Health Qual Life Outcomes. 2003; 1(1):42. [ Links ]
10. Thompson DR, Roebuck A. The measurement of health-related quality of life in patients with coronary heart disease. J Cardiovasc Nurs. 2001;16:28-33. [ Links ]
11. Treasure T. The measurement of health related quality of life. Heart. 1999;81(4):331-2. [ Links ]
12. Ventegodt S, Merrick J, Andersen NJ. Quality of life theory I. The IQOL theory: an integrative theory of the global quality of life concept. ScientificWorldJournal. 2003;3:1030-40. [ Links ]
13. Ponizovsky AM, Grinshpoon A. Quality of life among heroin users on buprenorphine versus methadone maintenance. Am J Drug Alcohol Abuse. 2007;33(5):631-42. [ Links ]
14. Hubley AM, Palepu A. Injection drug user quality of life scale (IDUQOL): findings from a content validation study. Health Qual Life Outcomes. 2007;5:46. [ Links ]
15. Malet L, Llorca PM, Beringuier B, Lehert P, Falissard B. AlQoL 9 for measuring quality of life in alcohol dependence. Alcohol Alcohol. 2006;41(2):181-7. [ Links ]
16. Vanagas G, Bihari-Axelsson S. Cross-sectional study on quality of life, work demands and psychosocial stress of Lithuanian general practitioners. Eur J Gen Pract. 2004;10(4):169-70. [ Links ]
17. da Silva Lima AF, Fleck M, Pechansky F, de Boni R, Sukop P. Psychometric properties of the World Health Organization quality of life instrument (WHOQOL-BREF) in alcoholic males: a pilot study. Qual Life Res. 2005;14(2): 473-8. [ Links ]
18. Martinez JAB, Mota GA, Vianna ESO, Filho JT, Silva GA, Rodrigues AL Jr. Impaired quality of life of healthy young smokers. Chest. 2004;125(2):425-8. [ Links ]
19. Endicott J, Nee J, Harrison W, Blumenthal R. Quality of life enjoyment and satisfaction questionnaire: a new measure. Psychopharmacol Bull. 1993;29(2):321-6. [ Links ]
20. Rossi A, Rucci P, Mauri M, Maina G, Pieraccini F, Pallanti S, et al. Validity and reliability of the Italian version of the Quality of Life, Enjoyment and Satisfaction Questionnaire. Qual Life Res. 2005;14(10):2323-8. [ Links ]
21. Jane-Llopis E. Mental health promotion: concepts and strategies for reaching the population. Health Promot J Austr. 2007;18(3):191-7. [ Links ]
22. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25(24):3186-91. [ Links ]
23. Ferraz MB. Cross cultural adaptation of questionnaires: what is it and when should it be performed? J Rheumatol. 1997;24(11):2066-8. [ Links ]
24. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington, DC: American Psychiatric Association; 2000. [ Links ]
25. Barros MB. The use of the concept of social class in the epidemiological profiles studies:a proposal. Rev Saude Publica. 1986;20(4): 269-73. [ Links ]
26. Schechter D, Endicott J, Nee J. Quality of life of 'normal' controls: association with lifetime history of mental illness. Psychiatry Res. 2007; 152(1):45-54. [ Links ]
27. Skevington SM, Lotfy M, O'Connell KA. The World Health Organization's WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual Life Res. 2004;13(2):299-310. [ Links ]
28. The WHOQOL Group. The World Health Organization Quality of Life Assessment (WHOQOL): development and general psychometric properties. Soc Sci Med. 1998;46(12): 1569-85. [ Links ]
29. Koller M, Aaronson NK, Blazeby J, Bottomley A, Dewolf L, Fayers P, et al. Translation procedures for standardised quality of life questionnaires: The European Organisation for Research and Treatment of Cancer (EORTC) approach. Eur J Cancer. 2007;43(12):1810-20. [ Links ]
30. Mezzich J, Ruiperez M. Presentation and validation of the Latino version of personal health scale (PHS-La) in New York. WPA International Congress, Caracas, October 2003. [ Links ]
31. Shamblen SR, Ringwalt C. The effects of Project SUCCESS on student academic performance: a quasi-experimental study. J Drug Educ. 2008;38(1):1-14. [ Links ]
Manuscript received on 26 February 2008.
Revised version accepted for publication on 23 November 2008.
1 Send correspondence and reprint requests to: Carlos Zubaran, Department of Psychiatry, Sydney West Area Health Service, PO Box 6010, Blacktown, New South Wales, Australia 2148; e-mail: email@example.com