Evaluation of different school-based preventive interventions for reducing the use of psychotropic substances among students: a randomized study

Marcelo Oliveira do Nascimento Denise De Micheli About the authors

Abstract

Although many schools perform preventive interventions for reducing drug use, little is known about their implementation processes and results. Accordingly, this study aimed to evaluate the effectiveness of three different types of preventive intervention performed in the school setting for reducing substance use among students. The study comprised 1316 students from the 9th year of elementary school to the 3rd year of secondary school in 8 public schools in the city of Guarulhos, São Paulo state, Brazil. Students were randomly assigned to the intervention groups or a control group and were evaluated at two different time-points regarding substance use. The results indicated that interventions performed by teachers were most effective in reducing both substance use and the severity of substance-associated problems. Interventions performed by experts were partially effective, although they only reduced use among experimental users. The results of “Single lecture” interventions revealed that such approaches are counterproductive. In summary, preventive actions that were contextualized to the student's reality and the school environment and that included the active involvement of both teacher and student were most effective at reducing the prevalence of substance use and the severity of associated problems in students.

Key words
Drug abuse; Adolescents; Prevention; Teachers; Randomized clinical trial

Introduction

Adolescence is a stage of life during which individuals are naturally exposed to increased challenges. This increased exposure makes them more vulnerable to several risk behaviors, including the use of psychotropic substances11. Nascimento MO, De Micheli D. Prevalência do uso de drogas entre adolescentes nos diferentes turnos escolares. Adolescência & Saude 2013; 10(4):41-49.. Several studies have indicated that drug users generally have their first contact with drugs during this stage of life22. De Micheli D, Formigoni ML. Drug use by Brazilian students: associations with family, psychosocial, health, demographic and behavioral characteristics. Addiction 2004; 99 (5):570-578.,33. Silva EA, De Micheli D, Camargo BMV, Buscatti D, Alencar MAP, Formigoni MLO. Drogas na adolescência: temores e reações dos pais. Psicologia: Teoria e Prática 2006; 8(1):41-54.. According to data from the VI National Survey on Psychotropic Drug Use among Elementary and Secondary School Students, which involved students from both public and private school systems in 27 Brazilian capitals, 42.5% of students between 13 and 15 years old had used alcohol, 10% had used tobacco and 15.5% had used illegal drugs44. Carlini EA, Noto AR, Sanchez ZM, Carlini CMA, Locatelli DP, Amato TC, Opaleye ES, Tondowski CS, Moura YG . VI Levantamento nacional sobre o consumo de drogas psicotrópicas entre estudantes do ensino fundamental e médio das redes públicas e privadas de ensino nas 27 capitais brasileira. São Paulo: CEBRID; 2010.. Another study by De Micheli and Formigoni22. De Micheli D, Formigoni ML. Drug use by Brazilian students: associations with family, psychosocial, health, demographic and behavioral characteristics. Addiction 2004; 99 (5):570-578. reinforced the strong association between early drug use and dependence. Similarly, neurobiological studies have shown that early drug use affects brain maturation processes during adolescence and may cause cognitive, behavioral and emotional changes55. Andrade ALM, Fisberg M, De Micheli D. Cognitive Aspects of Fetal Alcohol Syndrome in Young Adults: Two Case Studies. Interação Psicol 2013; 17(2):217-223.,66. Andrade ALM, De Micheli D, Silva EA, Souza-Formigoni MLO, Goeldner FO. Desenvolvimento Neural na adolescência. In: De Micheli D, Andrade ALM, Silva EA, Souza-Formigoni MLO, editores. Neurociências do Abuso de drogas na adolescência: o que sabemos? São Paulo: Ed. Atheneu; 2014. that can compound already existing difficulties. Taken together, these findings strongly suggest that adolescents are a priority target group for drug use intervention and prevention programs77. Rebolledo EAO, Medina NMO, Pillon SC. Factores de riesgo asociados al uso de drogas en estudiantes adolescentes. Rev. Latino-Americana de Enfermagem 2004; 12(N. esp):369-375.,88. Schenker M, Minayo MCS. A implicacao da familia no uso abusivo de drogas: uma revisao critica. Cien Saude Colet 2003; 8(1):299- 306..

It is well known that social interactions are established at school at the same time that young individuals are exercising their autonomy and forming their conceptions about life. Schenker and Minayo88. Schenker M, Minayo MCS. A implicacao da familia no uso abusivo de drogas: uma revisao critica. Cien Saude Colet 2003; 8(1):299- 306. state that the school settings presents many unique factors that can encourage adolescent substance use, including unattractive and inefficient teaching methods that decrease motivation to learn and increase absenteeism, ultimately leading to poor school performance.

Despite continuous changes, the Brazilian drug legislation (Law No. 11.343 from 08/23/2006)99. Brasil. Lei n° 11.343, de 23 de agosto de 2006. Institui o Sistema Nacional de Políticas Públicas sobre Drogas – Sisnad; prescreve medidas para prevenção do uso indevido, atenção e reinserção social de usuários e dependentes de drogas; estabelece normas para repressão à produção não autorizada e ao tráfico ilícito de drogas; define crimes e dá outras providências. Diário Oficial da União 2006; 24 ago. states in Article 19 that, among other things, activities to prevent drug abuse should include the continuing training of teachers in this area of knowledge at all three educational levels listed in the National Education Guidelines and Framework Law (Lei de Diretrizes e Bases da Educação Nacional - LDBEN)1010. Brasil. Lei n° 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Diário Oficial da União 1996; 23 dez.. Accordingly, it is understood that teachers from elementary and secondary schools are important potential prevention agents.

According to Paulo Freire1111. Freire P. Pedagogia da autonomia. 33a ed. São Paulo: Paz e Terra; 1996., teachers occupy a unique position because they can simultaneously inform and teach through their bond with students. Indeed, according to Demo1212. Demo MIS. Vínculo transferencial professor/aluno: importância no processo educativo [tese]. Ijuí: Universidade Regional do Noroeste do Estado do Rio Grande do Sul; 2011., very few professionals have such a strong influence over students as teachers. Similarly, McBride et al.1313. McBride N, Farringdon F, Midford R, Meuleners L, Phillips M. Harm minimisation in school drug education: final results of the School Health and Alcohol Harm Reduction Project. Addiction 2004; 99(3):278291. state that teachers represent an important variable in the effectiveness of several school-based prevention programs, particularly when they are properly trained and knowledgeable. In addition, according to Freire1111. Freire P. Pedagogia da autonomia. 33a ed. São Paulo: Paz e Terra; 1996., teachers must consider that in addition to teaching real, practical knowledge necessary for facing adversity, they should also focus on training critical, autonomous and active individuals1111. Freire P. Pedagogia da autonomia. 33a ed. São Paulo: Paz e Terra; 1996.. Sodelli1414. Sodelli M. Uso de drogas e prevenção: da desconstrução da postura proibicionista às ações redutoras de vulnerabilidade. São Paulo: Iglu; 2011. highlights the close relationship between the harm reduction approach and the dialogic pedagogy of Paulo Freire, stating that the former establishes that prevention and education are directly related. Accordingly, it should be appreciated that education that enables students to increase their critical thinking and autonomy is by its very nature preventive.

However, despite the important role of teachers in providing information and training to students, relatively few voluntarily address the topic psychotropic drugs. Studies have shown that social representations associated with negative and stigmatizing aspects, such as impotence, fear and poor training, are predominant among teachers, hindering the implementation of preventive actions1515. Sodelli M. A prevenção em nova perspectiva: ações redutoras de vulnerabilidade ao uso nocivo de drogas. Rev Portuguesa Internacional de Saúde Mental 2007; 9(2):3-58.1919. Dalbosco C. Representações sociais de educadores de escolas públicas sobre situações-problema relacionadas ao uso de álcool e outras drogas [tese]. Brasília: Universidade de Brasília; 2011..

Despite inefficient and unproductive results2020. Muller AC, Paul L, Santos NIS. Prevenção às drogas nas escolas: uma experiência pensada a partir dos modelos de atenção em saúde. Estudos em Psicologia 2008; 25(4):607-616.,2121. Moreira A, Vóvio CL, De Micheli D. Drug abuse prevention in school: challenges and possibilities for the role of the educator. Rev Educação e Pesquisa 2014; 41(1):119-135., many schools still choose to outsource drug education to outside professionals or experts, delegating the responsibility to address this issue separately with the students. Such approaches are aimed at avoiding confrontation with students or wearing down the student-teacher relationship. However, it has long been known that the development of trust and affection between teachers and adolescents is essential to any successful intervention program, and otherwise, little progress is achieved1515. Sodelli M. A prevenção em nova perspectiva: ações redutoras de vulnerabilidade ao uso nocivo de drogas. Rev Portuguesa Internacional de Saúde Mental 2007; 9(2):3-58.,2121. Moreira A, Vóvio CL, De Micheli D. Drug abuse prevention in school: challenges and possibilities for the role of the educator. Rev Educação e Pesquisa 2014; 41(1):119-135..

Considering the distressing reality of alcohol and drug use among adolescents and the negative consequences of such behavior, the proper utilization of teachers to address this issue is essential11. Nascimento MO, De Micheli D. Prevalência do uso de drogas entre adolescentes nos diferentes turnos escolares. Adolescência & Saude 2013; 10(4):41-49.,1515. Sodelli M. A prevenção em nova perspectiva: ações redutoras de vulnerabilidade ao uso nocivo de drogas. Rev Portuguesa Internacional de Saúde Mental 2007; 9(2):3-58.. Accordingly, several training courses for teachers focused on drug use prevention have been proposed in recent years. However, Sodelli1414. Sodelli M. Uso de drogas e prevenção: da desconstrução da postura proibicionista às ações redutoras de vulnerabilidade. São Paulo: Iglu; 2011. emphasizes that most of these training courses do not promote significant behavioral changes and/ or do not help teachers to feel more prepared and motivated to take on this task. Most likely, these problems are due to the passive role assigned to the students and the burden felt by the teacher, who must address a topic laden with personal prejudices1616. Nascimento MO, De Micheli D, Vitalle MAS. Visão e temores dos educadores ante ao uso abusivo de substâncias psicoativas por adolescentes no ambiente escolar. Rev. Magistro 2012; 2(1):5-21.1919. Dalbosco C. Representações sociais de educadores de escolas públicas sobre situações-problema relacionadas ao uso de álcool e outras drogas [tese]. Brasília: Universidade de Brasília; 2011.. Thus, it is thought that continuing education programs and teacher training can be more effective when using a participatory-constructivist approach, instilling in teachers – based on the dialogic pedagogy proposed by Paulo Freire1111. Freire P. Pedagogia da autonomia. 33a ed. São Paulo: Paz e Terra; 1996. – that “one also learns when teaching.”

As teachers are so fundamental to the process of drug use prevention, numerous studies22. De Micheli D, Formigoni ML. Drug use by Brazilian students: associations with family, psychosocial, health, demographic and behavioral characteristics. Addiction 2004; 99 (5):570-578.,1313. McBride N, Farringdon F, Midford R, Meuleners L, Phillips M. Harm minimisation in school drug education: final results of the School Health and Alcohol Harm Reduction Project. Addiction 2004; 99(3):278291.,1515. Sodelli M. A prevenção em nova perspectiva: ações redutoras de vulnerabilidade ao uso nocivo de drogas. Rev Portuguesa Internacional de Saúde Mental 2007; 9(2):3-58.,2020. Muller AC, Paul L, Santos NIS. Prevenção às drogas nas escolas: uma experiência pensada a partir dos modelos de atenção em saúde. Estudos em Psicologia 2008; 25(4):607-616.,2121. Moreira A, Vóvio CL, De Micheli D. Drug abuse prevention in school: challenges and possibilities for the role of the educator. Rev Educação e Pesquisa 2014; 41(1):119-135. warn that if teachers do not have a plan for general prevention and do not put this plan into action, the result will be merely theory disconnected from practice, which cannot reduce substance use among adolescents in or outside the school environment.

Accordingly, this study aimed to evaluate the impact of three different types of school-based preventive interventions on reducing drug use among students.

Materials and methods

Sample: This study involved 1316 students attending from the 8th year of elementary school to the 3rd year of secondary school from both the day and night shifts of 8 public schools from the eastern and western regions of the city of Guarulhos-São Paulo state, Brazil.

Eligibility criteria: twelve of the students who attended only one of the evaluations (T0 or T1) were not eligible to participate in the study. Additionally, 7 questionnaires were not included in the study due to erasures or filling errors that compromised the reliability of the information.

Selection criteria for school participation: the geographical distribution of schools in the city of Guarulhos is quite heterogeneous2222. Nascimento MO. O impacto de diferentes modalidades preventivas na redução do consumo de substancias psicoativas associado ao estudo da influencia parental entre adolescentes no ambiente escolar [tese]. São Paulo: Universidade Federal de São Paulo; 2012., with the schools being primarily concentrated in the eastern and western regions; therefore, only schools located in these two regions were selected. Initially, 14 schools from both regions were selected using convenience sampling and sociodemo-graphic matching, and the school principals were then contacted through a formal letter describing the entire project accompanied by a cover letter. Among the 14 contacted schools, only 10 (represented by their principals and/or coordinators) were willing to schedule a meeting with the coordinators of this study to obtain the details of the project. The other schools did not respond to the initial contact. At the end of the study, the coordinators from 2 from the 10 schools restated the importance of the study, but they were unwilling to participate due to lack of time, as their curriculum schedules were already completely filled for the school year. Therefore, 8 schools participated in this study, 4 of which were from the western region and 4 from the eastern region of Guarulhos.

Instruments: the Drug Use Screening Inventory (DUSI) was used to evaluate substance use and associated problems. This instrument was validated in Brazil by De Micheli and Formigoni22. De Micheli D, Formigoni ML. Drug use by Brazilian students: associations with family, psychosocial, health, demographic and behavioral characteristics. Addiction 2004; 99 (5):570-578.. This instrument consists of a table that assesses the frequency of alcohol, tobacco and other substance use in the previous month, followed by 10 DUSI domains that assess substance use and associated aspects (family, friends, school, leisure, etc.). This study used only the frequency table for substance use, as well as domain 1, which screens for substance use. Domain 1 consists of 15 questions that address problems associated with substance use, including compulsion or craving to use, tolerance symptoms, withdrawal, or involvement in accidents under the influence of alcohol and/or other drugs. The DUSI questions are answered with a Yes or No, with “yes” answers corresponding to the presence of problems. Absolute Problem Density was used to assess the severity of the substance use-related problems. This severity indicator proposed by the DUSI is calculated from the percentage of “yes” responses in the area.

Classification of participants according to substance use pattern

The cutoff points proposed during the DUSI validation were used to classify the patterns of substance use among the participants: 0-2 “yes” answers in the substance use domain corresponded to experimental drug users; 3-7 “yes” answers corresponded to drug abusers; and more than 7 “yes” answers corresponded to drug-dependent individuals. According to De Micheli & Formigoni2323. De Micheli D, Formigoni ML. Psychometric properties of the Brazilian version of the Drug Use Screening Inventory. Alcohol Clin Exp Res 2002; 26(10):1523-1528., these cutoffs have 72% sensitivity and 97% specificity in the classification of users and nonusers. Participants who did not score on the substance use domain (domain 1) and who did not mention substance use in the past month were classified as “nonusers”. Participants who used alcohol and/or tobacco no more than 1-2 times a month and/or had no more than 2 “yes” answers in the substance use domain were classified as “experimental drug users”. Participants who used alcohol and/or tobacco no more than 9 times in the past month, and/or used illegal substances no more than 9 times in the past month, and/or had 3-7 “yes” answers in the substance use domain were classified as “drug abusers”. Participants who used alcohol and/or tobacco more than 20 times in the past month, used illegal substances more than 10 times in the past month, and/or had 7 or more “yes” answers in the substance use domain were classified as “drug-dependent individuals”.

Procedures

The 8 participating schools signed an Informed Consent form and were then randomly included in either the control or experimental group. Since the beginning of contact, all school managers were informed that the study would be conducted at two different time-points to assess the effectiveness of the interventions.

Allocation of schools into the experimental and control groups

After agreeing to participate in the study, each school was assigned a random number. This number corresponded to the preventive intervention to be tested (experimental group) as well as the control group. The preventive intervention denominated “Single Lecture” corresponded to number 01; the preventive intervention denominated “Prevention performed by teachers” corresponded to number 02; the preventive intervention denominated “Prevention performed by experts” corresponded to number 03; and finally, the control group corresponded to number 00.

Control group (n = 339): One school from each region (2 schools in total) was randomly selected to be part of the control group. Students from this group only answered the questionnaires at the two assessment time-points of the study (T0 and T1). As initially agreed with the managers of the two schools, at the end of the study, teachers and students attended a lecture given by the principal investigators of this study to address issues such as quality of life, violence, sexuality, and use/abuse of psychotropic substances, with this last topic being the main focus of the lecture.

Experimental group (n=907): three schools from each region (6 schools in total) received different preventive interventions based on the previous randomization scheme and at the dates scheduled with the school administration:

  1. Single Lecture (n = 338): in the week following the completion of the questionnaires (T0), a lecture (100 minutes long) was given by experts to the students from both shifts. The content covered in this lecture included quality of life, violence, sexuality, and use/abuse of psychotropic substances, with this last topic being the main focus of the lecture. The theoretical reference used was “Harm Reduction-Based Prevention” (HR)1414. Sodelli M. Uso de drogas e prevenção: da desconstrução da postura proibicionista às ações redutoras de vulnerabilidade. São Paulo: Iglu; 2011.,2020. Muller AC, Paul L, Santos NIS. Prevenção às drogas nas escolas: uma experiência pensada a partir dos modelos de atenção em saúde. Estudos em Psicologia 2008; 25(4):607-616.,2424. Moreira FG, Silveira DX, Andreoli SB. Redução de Danos do Uso Indevido de Drogas no Contexto da Escola Promotora de Saúde. Cien Saude Colet 2006; 11(3):807816., which was also used in the interventions performed by the teachers (see below).

  2. Preventive interventions performed by teachers (PITEA) (n = 301): teachers from the schools were previously trained by experts to perform 6 preventive interventions that addressed quality of life, violence, sexuality, and use/abuse of psychotropic substances, with the last topic being the main focus of the interventions. These interventions began during the week following the completion of the questionnaires by the students (T0); they consisted of two sequential classes once a week for 6 consecutive weeks.

    Training of teachers: training was performed over 4 consecutive weeks during working hours and was scheduled according to “Collective Pedagogical Work classes” (Aulas de Trabalho Pedagógico Coletivo - ATPC), which primarily targets the continuing training of teachers. Training was conducted by experts associated with this study, who covered topics related to quality of life, violence, sexuality and drug use – providing teachers with the tools required to address these topics with the students. During this training period, 2-4 teachers from each school were selected to act as multipliers and perform the 6 preventive interventions. The criteria used to select the teachers that would perform the interventions were based on interest in this activity, willingness to be a volunteer, motivation and natural engagement with the students.

    Theoretical reference used in the training of teachers: in this study, we used the propositions of the dialogic relationship and problem-based education proposed by Paulo Freire1111. Freire P. Pedagogia da autonomia. 33a ed. São Paulo: Paz e Terra; 1996., whose premise is that a horizontal relationship between teacher and student is most effective for knowledge construction. According to Freire (1996), the dialogic practice must occur on a daily basis as an ongoing process between teacher and student in the way they relate, talk and think. This process should occur without impositions and without the oppressor-oppressed relationship that can arise in a vertical relationship. This perspective avoids the conception that the teacher owns the knowledge and primarily considers the knowledge that students bring from their previous experiences. According to Freire, the latter is essential for a dialogic relationship, and the teacher should always value the student's knowledge and consider their worldview. Similarly, problem-based education is based on the dialogue between student and teacher, who then build knowledge together. In other words, there is no complete and definitive knowledge, but rather knowledge that is built based on the teacher-student relationship. Within this concept, teachers challenge students to understand and critically think about the content that is being worked on; thereby, the students become active agents in the construction of knowledge. According to this theoretical perspective, the teachers were encouraged during training to use HR in their interventions with students. The theoretical construction of this educational approach is based on the expansion and deepening of knowledge and information about drugs and their use, on the effective strengthening of the subjects, and on the ability to choose based on potential consequences. In this approach, the drug is no longer the main focus, but rather the subject in his/her complexity, his/her biopsychosocial dimension and his/her citizenship become the focus.

  3. Preventive interventions performed by experts (PIEXP) (n = 338): After students completed the questionnaires (T0), a group of three professionals experts in chemical dependency (2 Psychologists and 1 Biologist) associated with this study (but not with the schools) offered preventive lectures to students for 6 consecutive weeks covering topics associated with quality of life, violence, sexuality, and use/abuse of psychotropic substances, with this last topic being the main focus of the lectures.

Random selection of the classes

One class from each school year participating in the study (one class from the 9th year of elementary school and one class of each year of secondary school (1st, 2nd and 3rd years)) were randomly selected in the presence of the school management team. This random selection was performed for all participating schools, totaling 7-8 classes per school (4 classes/school for the day shift and 3 or 4 classes/school for the night shift). This difference in the number of classes/school/ shift included in the study occurred because the 9th year is not offered during the night shift in all schools. Based on input from the school managers and the questionnaire applications, if the randomly selected class could not participate in the study, a new class was randomly selected.

Application of questionnaires and the start of preventive interventions

For the three types of intervention (experimental group), the various preventive interventions began during the week following the questionnaire applications (T0). Two months after the completion of the interventions (T1), the same questionnaires were reapplied to both groups (control and experimental) to assess the impact of each type of preventive intervention.

The questionnaires (both at To and T1) were applied in the classroom, without the presence of the teacher, by researchers affiliated with this study who directly approached the students. The students were informed about the goals of the study, assured as to the confidentiality of the information provided, and instructed about the correct way to complete the questionnaires. Application of the questionnaires lasted approximately 50 minutes.

To facilitate comparisons of the results from the two different time-points (T0 = before; T1 = after), the questionnaires given to students were labeled with numbers corresponding to the roll call of each class. In this way, each questionnaire was directed to a particular student (for example, questionnaire number 21 was given to the student number 21 in the roll call of that class). All students were instructed to deposit their completed questionnaires in a container near the classroom door.

Ethical aspects

This study was approved by the Research Ethics Committee of the Federal University of São Paulo.

Data analysis

The statistical software program SPSS version 20.0 was used in this study. Descriptive and bivariate exploratory data analyses were initially performed to identify possible differences between the experimental and control groups. Chisquare tests were used to assess the differences between categorical variables, and Student's t-test or analysis of variance (ANOVA) were used to compare the means of the numerical and categorical variables. ANOVA for repeated measures with a fixed factor (group) was used to compare each parameter between the control and experimental groups. To evaluate the effectiveness of the intervention, the Wilcoxon test was used to compare the means of the Absolute Problem Density (DUSI), considering the first and second application of the questionnaire (T0 and T1). A significance level of 5% was adopted in all analyses.

Results

The distributions of the sociodemographic variables for the 1316 students sampled is shown in Table 1. Considering the total sample, there were relatively homogeneous distributions for gender (46% males vs. 54% females) and school shift (43% day shift vs. 57% night shift), with the mean age being 15. Regarding the distribution of students by school year (elementary school and secondary school), there was a higher percentage of students in the first year of secondary school, as classes in this school year are larger (Table 1).

Table 1
Distributions of the sociodemographic variables for the 1316 students sampled

Table 2 shows a comparison of the impact of the preventive interventions on either reducing or increasing substance use for the two evaluated time-points (T0 and T1).

Table 2
Impact of the preventive interventions on either reducing or increasing substance use in the last month, considering two evaluated time-points (T0 and T1). (n = 1316)

Among students who attended the Single Lecture, the second evaluation revealed a significant 88% reduction in alcohol in those students who reported usage of 3-9 times in the past month. By contrast, there was a significant 61% increase in intranasal cocaine use for students reporting use 3-9 times a month, and a 113% increase for students reporting use of more than 20 times a month. Similarly, the second evaluation revealed a significant 128% increase in crack use for the students reporting use between 10-20 times a month and a 58% increase in cannabis use for the students reporting use 3-9 times a month.

Among students who received preventive interventions performed by the teachers, significant decreases were observed for the use of various substances at the second evaluation. There was a 70% reduction in alcohol use for the those reporting use 3-9 times a month, a 71.5% reduction in those reporting use 10-20 times a month, and an 88% reduction in those reporting use more than 20 times in the past month. Regarding intranasal cocaine use, there was a significant 56% reduction in those reporting use 1-2 times a month and a 75% reduction in those reporting use more than 20 times a month. Similarly, there was a 47% reduction in crack use in those reporting use 10-20 times a month and a 38% reduction in those reporting use more than 20 times a month. Regarding cannabis use, the second evaluation revealed a 50% reduction in those reporting use 10-20 times a month and an 84% reduction for those reporting use more than 20 times a month. Regarding tobacco use, a 53% reduction was observed in those reporting use 10-20 times a month and a 33% reduction in those reporting use more than 20 times in the past month. For solvents, the second evaluation also revealed a 46% reduction in those reporting use 10-20 times in the past month and an 80% reduction for those reporting use 20 times or more in the past month.

Among students who received preventive interventions performed by the experts, a significant 73% reduction was observed in alcohol use for those reporting use 3-9 times a month, a 50% reduction in alcohol use for those reporting use 10-20 times a month, and a 58% reduction in alcohol use for those reporting use 20 times or more a month. For cannabis, a 25% reduction was in those using 3-9 times a month, and a 42.5% reduction was observed for those using 10-20 times a month.

Among the students of the control group, the second evaluation revealed a significant increase of 28% in the use of alcohol for those reporting use 3-9 times a month, and a reduction of 54% in the use of alcohol in those reporting use 10-20 times a month. There was also an increase of 80% in use of intranasal cocaine 1-2 times a month and an increase of 58% in the use of tobacco 3-9 times a month.

Table 3 describes the severity of the problems related to substance use (Absolute Density) of the DUSI for the two time-points evaluated (T0 and T1). The comparison of the results obtained for the first (T0) and second evaluations (T1) revealed a significant reduction in the severity of problems in the groups that received preventive intervention performed by both teachers and experts. Comparing only the results obtained in the second evaluation (T1) between experimental and control groups, students who received interventions by experts (T1 = 3.9) or teachers (T1 = 3.8) showed a significant reduction in the severity of problems compared with those who attended only a single lecture (T1 = 7.9) or were in the control group (T1 = 7.2).

Table 3
Comparison of severity of the problems related to substance use (Absolute Density) of the DUSI for the two time-points evaluated (T0 and T1), considering Experimental Group and Control Group. (n = 1316)

Table 4 shows the impact of the different preventive interventions on the patterns of drug use observed at the second evaluation. A total of 20% of the sample did not use substances (nonusers), 48% were experimental drug users, 27% were drug abusers, and 5% were classified as drug dependents. Among students that were considered nonusers, interventions performed by the experts had the most impact on the second evaluation, increasing the percentage of nonuser students (increase of 14% in T1), followed by interventions performed by teachers (increase of 37% in T1). Moreover, a significant 21% reduction in the number of nonusers was observed at the second evaluation for the control group, which was significantly different from the group that attended only the single lecture, which showed a 34% reduction in the number of nonusers; these findings indicate that a significant percentage of students were no longer “nonusers” after receiving this type of preventive intervention. Similarly, interventions performed by experts (reduction of 9% in T1) and interventions performed by teachers (reduction of 12% in T1) were the most successful at decreasing the percentage of experimental drug users at the second evaluation, with interventions performed by teachers showing a significantly greater impact than those performed by experts. By contrast, there was a significant increase of 9% in the number of experimental drug users in the control group between the first and second evaluations; however, this increase was significantly smaller than that observed for this category in the group that attended a single lecture (increase of 15% in T1). For the category of drug abuse, only preventive interventions performed by teachers reduced the percentage of students in this group. Although no statistically significant differences were observed, a slight reduction in the number of drug abusers was observed in the group that attended the single lecture. The other types of preventive intervention showed no impact in changing this pattern of use. A significant increase in the number of drug-dependent individuals who received interventions performed by experts or attended a single lecture was observed at the second evaluation, with the increase for the single lecture being significantly higher (increase of 15% vs. 33%).

Table 4
Impact of the different preventive interventions on the patterns of drug use, considering two time-points evaluated. (n = 1316)

Discussion

Alcohol and/or other substance use among students in Brazil and around the world are significant concerns for health care professionals and educators in general, due to the biopsychosocial repercussions associated with these behaviors. Accordingly, numerous strategies to prevent drug use have been implemented. Although some schools in Brazil perform preventive interventions for drug use, little is known about their implementation processes and results. Overall, prevention efforts in the school setting are executed in an ad hoc and discontinuous manner, contributing to low compliance rate among students1515. Sodelli M. A prevenção em nova perspectiva: ações redutoras de vulnerabilidade ao uso nocivo de drogas. Rev Portuguesa Internacional de Saúde Mental 2007; 9(2):3-58.,2424. Moreira FG, Silveira DX, Andreoli SB. Redução de Danos do Uso Indevido de Drogas no Contexto da Escola Promotora de Saúde. Cien Saude Colet 2006; 11(3):807816.,2525. Midford R. Drug prevention programmes for young people: where have we been and where should we be going? Addiction 2010; 105(10):1688-1695.. Several studies have indicated that schools and their social actors are one of the main places to establish positive choices for health and prevention, as well as to promote changes in potential unhealthy, risky behaviors and life-styles2323. De Micheli D, Formigoni ML. Psychometric properties of the Brazilian version of the Drug Use Screening Inventory. Alcohol Clin Exp Res 2002; 26(10):1523-1528.,2626. Tavares T, Bonito J, Oliveira M. Caraterização do consumo de álcool entre os escolares de 12 a 21 anos de idade do distrito de Beja. In: Pereira B, Cunha C, Anastácio Z, Carvalho G, coordenadores. Atas do IX seminário internacional de educação física, lazer e saúde. Braga: Instituto de Educação da Universidade do Minho; 2013. 2.° Vol. p. 339-358. Therefore, the dedicated and genuine involvement of teachers is requited to achieve positive results in these processes. However, when topics about the prevention of drug use are addressed, social representations associated with both negative and stigmatizing aspects, such as impotence, fear and feelings of unpreparedness, are predominant among teachers, reducing the effectiveness of preventive actions1717. Ferreira TCD, Sanchez ZM, Ribeiro LA, Oliveira LC, Nappo AS. Percepção e atitudes de professores de escolas públicas e privadas perante o tema drogas. Interface (Botucatu) 2010; 14(34):551-562.,1818. Araldi JC, Njaine K, Oliveira MC, Ghizoni AC. Teachers’ social representations of abusive use of alcohol and other drugs during adolescence: repercussions on preventive actions in schools. Interface (Botucatu) 2012; 16(40):135-146.,2424. Moreira FG, Silveira DX, Andreoli SB. Redução de Danos do Uso Indevido de Drogas no Contexto da Escola Promotora de Saúde. Cien Saude Colet 2006; 11(3):807816.,2727. Placco VMNS. Modelos de Prevenção do Uso de Drogas para Adolescentes: Concepções e Ações de Professores. In: Silva EA, Micheli D, organizadores. Adolescência, Uso e Abuso de Drogas: Uma Visão Integrativa. São Paulo: Fap-Unifesp; 2011. p. 657 - 678. According to Sodelli1414. Sodelli M. Uso de drogas e prevenção: da desconstrução da postura proibicionista às ações redutoras de vulnerabilidade. São Paulo: Iglu; 2011.,2828. Sodelli M. A abordagem de Redução de Danos Libertadora na Prevenção: ações redutoras de vulnerabilidade. In: Silva EA, De Micheli D, organizadores. Adolescência - uso e abuso de drogas: uma visão integrativa. São Paulo: Fap-Unifesp; 2012. p. 11-787., one aspect that is neglected in this educational area is understanding the relationship between “teaching” and “preventing”. Historically, the prevention of substance use and risk behavior has been attributed to professionals not directly associated with education (e.g., physicians, psychologists, and policemen). It is possible that, over time, this fact has reinforced the belief that prevention is not part of the educational area2929. Placco V. Representações Sociais de professores do Ensino Médio quanto Aids, Drogas, Violência e Prevenção. São Paulo: Relatório de pesquisa Fapesp; 2006.. Thus, an attempt was made to deconstruct this belief through teacher training using the Harm Reduction preventive approach, which is based on the dialogic pedagogy of Paulo Freire and encourages teachers to implement preventive actions.

Accordingly, the results of this study demonstrated that preventive intervention performed by teachers had the greatest impact, not only in reducing the frequency of the use for several substances over a one month period but also in reducing the severity of the problems associated with these usage patterns. Additionally, following interventions performed by teachers, we observed a significant increase in the percentage of nonuser students. According to Moreira et al.2424. Moreira FG, Silveira DX, Andreoli SB. Redução de Danos do Uso Indevido de Drogas no Contexto da Escola Promotora de Saúde. Cien Saude Colet 2006; 11(3):807816., the type of bond between teacher and students who use legal or illegal substances may be the watershed between interrupting the process of drug experimentation and migration to other usage patterns. With respect to the teacher's role in this context, Pavani et al.3030. Pavani RAB, Silva EF, Moraes MS. Avaliação da informação sobre drogas e sua relação com o consumo de substâncias entre escolares. Rev. Brasileira de Epidemiologia 2009; 12(2):204-216. evaluated a sample of 1041 secondary school students in the city of São Jose do Rio Preto with aim of determining their opinions about different drug information sources. Their results showed that students who saw the teacher as a reliable source of drug information showed lower rates of alcohol, tobacco, cannabis, intranasal cocaine and crack use. Similarly, Mc-Bride1313. McBride N, Farringdon F, Midford R, Meuleners L, Phillips M. Harm minimisation in school drug education: final results of the School Health and Alcohol Harm Reduction Project. Addiction 2004; 99(3):278291. conducted a systematic literature review to identify elements that increase the effectiveness of interventions in the school environment. The author concluded that properly trained teachers who are connected with the students is the distinctive feature of most successful school-based drug prevention programs.

It should be emphasized that despite the substantial role of the teacher, he/she is not solely responsible for preventing substance use. It is important to be careful and not assume that the educational institution is the only organ responsible for solving problems resulting from drug use/abuse. That is, a teacher who is properly prepared to address issues related to substance use does not exempt other social actors (i.e., parents and health care professionals) involved in different educational spaces of the adolescent's life from being committed to this matter.

Similar to the interventions performed by teachers, interventions performed by experts also showed satisfactory results. Comparing the two periods evaluated, the students from this group also showed significant reductions in the frequency of use for various substances. Compared with the results obtained from students who attended a single lecture and students from the control group, students who received interventions performed by experts showed a significant reduction in the severity of problems associated with substance use. In addition, this approach significantly increased the percentage of nonuser students and significantly reduced the percentage of students considered to be experimental drug users. It is also notable that in this type of intervention, the experts were in close contact with the students for 6 consecutive weeks, which did not occur in the group that attended only a single lecture. Therefore, one might assume that the experts had sufficient time to develop effective and trusting bonds with the students. Presumably, it was this bond that was built over the 6 weeks that was responsible for the impact this type of intervention. According to Schenker and Minayo3131. Schenker M, Minayo MCS. A implicação da família no uso abusivo de drogas: uma revisão crítica. Cien Saude Colet 2003; 8(1):299-306., adolescent substance users respond favorably to interventions that are contextualized in their reality. Thus, one could infer that it was not only the theoretical framework used but also, and perhaps especially, the connection established with the students that represented the key reason for their behavioral changes. De Micheli and Formigoni22. De Micheli D, Formigoni ML. Drug use by Brazilian students: associations with family, psychosocial, health, demographic and behavioral characteristics. Addiction 2004; 99 (5):570-578. assessed the risk and protective factors among groups of substance users and nonusers and found that those who mentioned having a strong emotional or trust bond with important people in their life (e.g., aunts/uncles, teachers, friends, parents) had lower rates of substance use. These authors showed that a strong emotional bond with important people was a protective factor associated with the interruption and/ or nonuse of substance among adolescents. In the present study, comparisons of the interventions performed by experts and teachers revealed that both had a good impact, although there were some interesting differences. Regarding the frequency of substance use in the previous month, students who received interventions performed by teachers showed reductions in the frequency of use for all six substances evaluated. By contrast, the students who received interventions performed by experts only showed significant reductions for alcohol, tobacco and cannabis. With respect to changes in usage pattern, although both types of intervention were effective in increasing the percentage of nonusers and in reducing the experimental drug users, only interventions performed by teachers were significantly more effective, and this was the only approach that could reduce the drug abuse pattern. These data suggest that despite the time spent by the experts with the students, they were unable to establish a bond with and/or become close with those who were using those other substances (intranasal cocaine, crack and solvents) or were using at the highest frequencies. It is also possible that these students avoided proximity with experts due to drug use itself. Studies have shown that the internalized stigma among many substance users is a strong barrier against searching for expert help due to the fear and/or shame of their condition3232. Felicissimo FB, Ferreira GCL, Soares RG, Silveira OS, Ronzani TM. Estigma internalizado e autoestima: uma revisão sistemática da literatura. Psicologia, Teoria e Prática 2013; 15(1):116-119.. Considering this, the longer-lasting student-teacher relationship can be a consistent and positive factor for these intervention processes.

Regarding the “single lecture” intervention type, we actually observed a significant increase in the frequency of illicit substance use, and indeed, analysis revealed a significant reduction in the percentage of nonusers for the second evaluation. In other words, some percentage of students that were classified as nonusers at the first evaluation had began using some substance after receiving this preventive intervention. Moreover, compared with students in the control group, there was a significant increase in the percentage of students in this group who began to use substances with both the “experimental use” and “drug dependency” patterns. These results are consistent with previous studies showing that such programs are inefficient and can in fact motivate adolescents to experiment and/or minimize fear in those adolescents who already using drugs, thereby having the opposite of the desired effect,33. Silva EA, De Micheli D, Camargo BMV, Buscatti D, Alencar MAP, Formigoni MLO. Drogas na adolescência: temores e reações dos pais. Psicologia: Teoria e Prática 2006; 8(1):41-54.,1212. Demo MIS. Vínculo transferencial professor/aluno: importância no processo educativo [tese]. Ijuí: Universidade Regional do Noroeste do Estado do Rio Grande do Sul; 2011.,2121. Moreira A, Vóvio CL, De Micheli D. Drug abuse prevention in school: challenges and possibilities for the role of the educator. Rev Educação e Pesquisa 2014; 41(1):119-135.,2727. Placco VMNS. Modelos de Prevenção do Uso de Drogas para Adolescentes: Concepções e Ações de Professores. In: Silva EA, Micheli D, organizadores. Adolescência, Uso e Abuso de Drogas: Uma Visão Integrativa. São Paulo: Fap-Unifesp; 2011. p. 657 - 678,2828. Sodelli M. A abordagem de Redução de Danos Libertadora na Prevenção: ações redutoras de vulnerabilidade. In: Silva EA, De Micheli D, organizadores. Adolescência - uso e abuso de drogas: uma visão integrativa. São Paulo: Fap-Unifesp; 2012. p. 11-787.,3333. Sanchez ZVDM. O papel da informação como medida preventiva ao uso de drogas entre jovens em situação de risco. Cien Saude Colet 2009; 15(2):621-630.. This fnding highlights that the actual contributions of these preventive programs should be rigorously assessed. De Micheli et al.3434. De Micheli D, Fisberg M, Formigoni MLOS. Study on the effectiveness of brief intervention for alcohol and other drug use directed to adolescents in a primary health care unit. Rev. Assoc. Med. Bras. 2004; 50(3):305-313. assessed the effectiveness of a lecture on substance use prevention targeting adolescents who did not use any substances that was performed in a primary health care setting, and they found an increase in the frequency of drug use after six months. Another study in a different setting showed a similar outcome, or in other words, increased frequencies of substance use after attending the orientation lecture. According to Muller et al.2020. Muller AC, Paul L, Santos NIS. Prevenção às drogas nas escolas: uma experiência pensada a partir dos modelos de atenção em saúde. Estudos em Psicologia 2008; 25(4):607-616., occasional meetings with experts who do not belong to the school and are not connected with the adolescents prevent the commitment of the school to wider and more effective programs. Therefore, it is important to plan such programs systematically and implement actions that go beyond occasional meetings.

One limitation of this study was the convenience sampling strategy adopted to select the sample, which does not represent the entire study population and therefore does not allow for generalization of the observed results. By contrast, the sample size is satisfactory and paves the road for future studies with more representative samples.

Taken together, the results of the present study indicate the effectiveness of preventive interventions performed by teachers in reducing the frequency of substance use and the severity of problems associated with substance use among students. One likely reason for these results is the training offered to teachers before the interventions, undermining the idea that prevention is not part of the educational role, thus closing the gap between the act of teaching and the act of preventing. These results reinforce the idea that effective prevention strategies are more in line with liberating approaches to education, characterized by a horizontal teacher-student relationship in which both parties are part of the educational practice. We believe that encouraging leadership and autonomy in students and promoting critical and contextualized reflection are associated with the results of this study, as these practices nurture conscious attitudes on the part of the students. The present study also demonstrates the effectiveness of intervention performed by experts. However, the results we obtained showed a lower effectiveness for this type of intervention, given that it reached only students who did not use substances or were only experimental users.

It is clear that there is no single, specific prevention model for substance use that is fully effective. However, we believe that an educational model incorporating dynamism, contextualization of topics, respect and mutual learning within the teacher-student relationship is more effective, interesting and engaging to those involved.

References

  • 1
    Nascimento MO, De Micheli D. Prevalência do uso de drogas entre adolescentes nos diferentes turnos escolares. Adolescência & Saude 2013; 10(4):41-49.
  • 2
    De Micheli D, Formigoni ML. Drug use by Brazilian students: associations with family, psychosocial, health, demographic and behavioral characteristics. Addiction 2004; 99 (5):570-578.
  • 3
    Silva EA, De Micheli D, Camargo BMV, Buscatti D, Alencar MAP, Formigoni MLO. Drogas na adolescência: temores e reações dos pais. Psicologia: Teoria e Prática 2006; 8(1):41-54.
  • 4
    Carlini EA, Noto AR, Sanchez ZM, Carlini CMA, Locatelli DP, Amato TC, Opaleye ES, Tondowski CS, Moura YG . VI Levantamento nacional sobre o consumo de drogas psicotrópicas entre estudantes do ensino fundamental e médio das redes públicas e privadas de ensino nas 27 capitais brasileira São Paulo: CEBRID; 2010.
  • 5
    Andrade ALM, Fisberg M, De Micheli D. Cognitive Aspects of Fetal Alcohol Syndrome in Young Adults: Two Case Studies. Interação Psicol 2013; 17(2):217-223.
  • 6
    Andrade ALM, De Micheli D, Silva EA, Souza-Formigoni MLO, Goeldner FO. Desenvolvimento Neural na adolescência. In: De Micheli D, Andrade ALM, Silva EA, Souza-Formigoni MLO, editores. Neurociências do Abuso de drogas na adolescência: o que sabemos? São Paulo: Ed. Atheneu; 2014.
  • 7
    Rebolledo EAO, Medina NMO, Pillon SC. Factores de riesgo asociados al uso de drogas en estudiantes adolescentes. Rev. Latino-Americana de Enfermagem 2004; 12(N. esp):369-375.
  • 8
    Schenker M, Minayo MCS. A implicacao da familia no uso abusivo de drogas: uma revisao critica. Cien Saude Colet 2003; 8(1):299- 306.
  • 9
    Brasil. Lei n° 11.343, de 23 de agosto de 2006. Institui o Sistema Nacional de Políticas Públicas sobre Drogas – Sisnad; prescreve medidas para prevenção do uso indevido, atenção e reinserção social de usuários e dependentes de drogas; estabelece normas para repressão à produção não autorizada e ao tráfico ilícito de drogas; define crimes e dá outras providências. Diário Oficial da União 2006; 24 ago.
  • 10
    Brasil. Lei n° 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Diário Oficial da União 1996; 23 dez.
  • 11
    Freire P. Pedagogia da autonomia 33a ed. São Paulo: Paz e Terra; 1996.
  • 12
    Demo MIS. Vínculo transferencial professor/aluno: importância no processo educativo [tese]. Ijuí: Universidade Regional do Noroeste do Estado do Rio Grande do Sul; 2011.
  • 13
    McBride N, Farringdon F, Midford R, Meuleners L, Phillips M. Harm minimisation in school drug education: final results of the School Health and Alcohol Harm Reduction Project. Addiction 2004; 99(3):278291.
  • 14
    Sodelli M. Uso de drogas e prevenção: da desconstrução da postura proibicionista às ações redutoras de vulnerabilidade São Paulo: Iglu; 2011.
  • 15
    Sodelli M. A prevenção em nova perspectiva: ações redutoras de vulnerabilidade ao uso nocivo de drogas. Rev Portuguesa Internacional de Saúde Mental 2007; 9(2):3-58.
  • 16
    Nascimento MO, De Micheli D, Vitalle MAS. Visão e temores dos educadores ante ao uso abusivo de substâncias psicoativas por adolescentes no ambiente escolar. Rev. Magistro 2012; 2(1):5-21.
  • 17
    Ferreira TCD, Sanchez ZM, Ribeiro LA, Oliveira LC, Nappo AS. Percepção e atitudes de professores de escolas públicas e privadas perante o tema drogas. Interface (Botucatu) 2010; 14(34):551-562.
  • 18
    Araldi JC, Njaine K, Oliveira MC, Ghizoni AC. Teachers’ social representations of abusive use of alcohol and other drugs during adolescence: repercussions on preventive actions in schools. Interface (Botucatu) 2012; 16(40):135-146.
  • 19
    Dalbosco C. Representações sociais de educadores de escolas públicas sobre situações-problema relacionadas ao uso de álcool e outras drogas [tese]. Brasília: Universidade de Brasília; 2011.
  • 20
    Muller AC, Paul L, Santos NIS. Prevenção às drogas nas escolas: uma experiência pensada a partir dos modelos de atenção em saúde. Estudos em Psicologia 2008; 25(4):607-616.
  • 21
    Moreira A, Vóvio CL, De Micheli D. Drug abuse prevention in school: challenges and possibilities for the role of the educator. Rev Educação e Pesquisa 2014; 41(1):119-135.
  • 22
    Nascimento MO. O impacto de diferentes modalidades preventivas na redução do consumo de substancias psicoativas associado ao estudo da influencia parental entre adolescentes no ambiente escolar [tese]. São Paulo: Universidade Federal de São Paulo; 2012.
  • 23
    De Micheli D, Formigoni ML. Psychometric properties of the Brazilian version of the Drug Use Screening Inventory. Alcohol Clin Exp Res 2002; 26(10):1523-1528.
  • 24
    Moreira FG, Silveira DX, Andreoli SB. Redução de Danos do Uso Indevido de Drogas no Contexto da Escola Promotora de Saúde. Cien Saude Colet 2006; 11(3):807816.
  • 25
    Midford R. Drug prevention programmes for young people: where have we been and where should we be going? Addiction 2010; 105(10):1688-1695.
  • 26
    Tavares T, Bonito J, Oliveira M. Caraterização do consumo de álcool entre os escolares de 12 a 21 anos de idade do distrito de Beja. In: Pereira B, Cunha C, Anastácio Z, Carvalho G, coordenadores. Atas do IX seminário internacional de educação física, lazer e saúde Braga: Instituto de Educação da Universidade do Minho; 2013. 2.° Vol. p. 339-358
  • 27
    Placco VMNS. Modelos de Prevenção do Uso de Drogas para Adolescentes: Concepções e Ações de Professores. In: Silva EA, Micheli D, organizadores. Adolescência, Uso e Abuso de Drogas: Uma Visão Integrativa São Paulo: Fap-Unifesp; 2011. p. 657 - 678
  • 28
    Sodelli M. A abordagem de Redução de Danos Libertadora na Prevenção: ações redutoras de vulnerabilidade. In: Silva EA, De Micheli D, organizadores. Adolescência - uso e abuso de drogas: uma visão integrativa São Paulo: Fap-Unifesp; 2012. p. 11-787.
  • 29
    Placco V. Representações Sociais de professores do Ensino Médio quanto Aids, Drogas, Violência e Prevenção São Paulo: Relatório de pesquisa Fapesp; 2006.
  • 30
    Pavani RAB, Silva EF, Moraes MS. Avaliação da informação sobre drogas e sua relação com o consumo de substâncias entre escolares. Rev. Brasileira de Epidemiologia 2009; 12(2):204-216.
  • 31
    Schenker M, Minayo MCS. A implicação da família no uso abusivo de drogas: uma revisão crítica. Cien Saude Colet 2003; 8(1):299-306.
  • 32
    Felicissimo FB, Ferreira GCL, Soares RG, Silveira OS, Ronzani TM. Estigma internalizado e autoestima: uma revisão sistemática da literatura. Psicologia, Teoria e Prática 2013; 15(1):116-119.
  • 33
    Sanchez ZVDM. O papel da informação como medida preventiva ao uso de drogas entre jovens em situação de risco. Cien Saude Colet 2009; 15(2):621-630.
  • 34
    De Micheli D, Fisberg M, Formigoni MLOS. Study on the effectiveness of brief intervention for alcohol and other drug use directed to adolescents in a primary health care unit. Rev. Assoc. Med. Bras 2004; 50(3):305-313.

Publication Dates

  • Publication in this collection
    Aug 2015

History

  • Received
    01 July 2014
  • Reviewed
    19 Nov 2014
  • Accepted
    21 Nov 2014
ABRASCO - Associação Brasileira de Saúde Coletiva Rio de Janeiro - RJ - Brazil
E-mail: revscol@fiocruz.br