Is the negative evaluation of dental services among the Brazilian elderly population associated with the type of service?

Andréa Maria Eleutério de Barros Lima Martins Lorena Amaral Jardim João Gabriel Silva Souza Carlos Alberto Quintão Rodrigues Raquel Conceição Ferreira Isabela Almeida Pordeus About the authors

Abstracts

This study aimed at identifying the prevalence of the negative evaluation of dental services among elderly Brazilians and at investigating whether the prevalence was higher among those using public or philanthropic provider services than among those paying privately or using private health plans. Additionally, factors associated with this negative assessment were identified. Interview and survey data were collected in the residences of participants by trained and calibrated examiners as part of a national epidemiological survey of oral health conditions of the Brazilian population in 2002/2003. The dependent variable was obtained in response to questions regarding whether the participant had ever used dental services, the frequency of use, and the quality of this service. Potential responses to the questions regarding the quality of service were very poor or poor, fair, and good or very good. The main independent variable was the system of health care used with potential responses being health plan or private, public, and philanthropic services. We conducted univariate (linear tendency χ2test) and multiple descriptive analyses, and the partial proportional Odds model for ordinal logistic regression. Among the elderly, 196 (3.7%) evaluated the provided services negatively (very poor or poor). Participants with the following responses were more likely to evaluate the services negatively: those who had used public or philanthropic services, men, those with higher education, the ones who had not received information about preventing dental problems, those who perceived pain in their teeth and gums in the last six months, and those who self-reported their oral health and speech was poor. In conclusion, elderly Brazilian users of public and philanthropic services were more likely than users of private or insurance-based plans to evaluate their dental services negatively, regardless of the other investigated variables.

Evaluation; Dental care; Elderly; Oral health; Users


INTRO DUCTION

The increased longevity of the Brazilian population and the reduced mortality and fecundity rates have changed the demographic profile of the country, which now has one of the fastest aging populations in the world11. Schmidt MI, Duncan BB, Azevedo e Silva G, Menezes AM, Monteiro CA, Barreto SM, et al. Chronic non-communicable diseases in Brazil: burden and current challenges. Lancet 2011; 377(9781): 1949-61.. It is estimated that in 2020, Brazil will have the sixth largest population of elderly people in the world22. Lima-Costa MF, Veras R. Saúde pública e envelhecimento. Cad Saúde Pública 2003; 19(3): 700-1.. The demands and needs of the elderly Brazilian population are increasing, especially in the health field33. Benedetti TRB, Mello ALSF, Gonçalves LHT. Idosos de Florianópolis: autopercepção das condições de saúde bucal e utilização de serviços odontológicos. Cienc Saúde Coletiva 2007; 12(6): 1683-90., which leads to the need for more oral health attention among this population, and this does not always happen44. Colussi CF, Freitas SFT. Aspectos epidemiológicos da saúde bucal do idoso no Brasil. Cad Saúde Pública 2002; 18(5): 1313-20.. Therefore, knowing the needs, determining factors of the use of services and making the access to qualified health services viable for the elderly present as new challenges55. Moreira MM. O envelhecimento da população brasileira em nível regional; 1940-2050. In: XI Encontro Nacional de Estudos Populacionais. Anais... Caxambu: Associação Brasileira de Estudos Populacionais; 1998. p. 3030-3124.. Besides, the evaluation of health services from the perspective of the users has been used in the attempt to organize and order resources, with the objective of meeting the health needs of the population properly; therefore, this would be an important tool for the development of management strategies for the sector66. Gouveia GC, Souza WV, Luna CF, Souza-Júnior PRB, Szwarcwald CL. Satisfação dos usuários do sistema de saúde brasileiro: fatores associados e diferenças regionais. Rev Bras Epidemiol 2009; 12(3): 281-96..

Studies that evaluated health services stood out in literature, especially in the 1970s, in countries like the United States and England. The improvement of health services is reached when one of the goals is user satisfaction. In Brazil, these studies were conducted from the 1990s on, especially with the strengthening social control in the scope of the Unified Health System (SUS), by means of the participation of the community in planning and evaluation processes77. Favaro P, Ferris LE. Program evaluation with limited fiscal and human resources. Cad Saúde Pública 1991; 11(3): 425-38..

The opinion of users as to the quality of care has been suggested with the objective of promoting subsidies for those who administer such services and for the staff that provides care, therefore allowing the detected limitations to be overcome88. Robles ACC, Grosseman S, Bosco VL. Satisfação com o atendimento odontológico: estudo qualitativo com mães de crianças atendidas na Universidade Federal de Santa Catarina. Cienc Saúde Coletiva 2008; 13(1): 43-9.. It helps, since the participation of users in the assessment of services surpasses its mere passive use, and provides essential information to define quality standards in the provided services, since this perspective enables to complement the technical quality evaluations with a shared view of the perception of individuals who receive care99. Bottan ER, Sperb RAL, Telles PS, Uriarte Neto M. Avaliação de serviços odontológicos: a visão dos pacientes. Rev ABENO 2006; 6(2): 128-33..

The need to assess health services, besides improving the performance of service providers, gives favorable conditions to employees and users, improves working conditions and the quality of life of people, therefore providing more efficiency and effectivity to the system1010. Chiavenato I. Gerenciando pessoas: o passo decisivo para a administração participativa. 2 ed. São Paulo: Makron Books; 1994.. The quality of health services can be considered as a result of several factors, such as professional competence, accessibility, effectivity, efficiency and user satisfaction1111. Gattinara BC, Ibacache J, Puente CT, Giaconi J, Caprara A. Percepción de la comunidad acerca de la calidad de los servicios de salud públicos en los Distritos Norte e Ichilo, Bolivia. Cad Saúde Pública 1995; 14(1): 61-70.. In this context, the dental health service should also be assessed. This was first developed in a representative sample of the elderly population in Brazil, in the epidemiological survey of oral health conditions of the Brazilian population (Project SB Brasil). It was a population-based study conducted by the Ministry of Health (MH) in 2002 and 2003. This study generated a descriptive report about oral health conditions of the Brazilian population, which also contemplated information about dental cavities, periodontal conditions, edentulism, dental and facial anomalies, fluorosis, socioeconomic characterization, access to dental services and oral health self-perception. In the mentioned report, it was observed that 3.9% of the elderly assessed oral services negatively (very poor/poor)1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004..

No studies characterizing the Brazilian elderly population who assessed the services negatively were identified. Therefore, identifying if the prevalence of negative evaluation regarding dental services among the Brazilian elderly population was higher among users of public or philanthropic services, as well as factors associated with this analysis, can subsidize changes in public health and assistance policies in the field, with the implantation of rehabilitating, preventive and health promotion actions1313. Silva SRC, Fernandes RAC. Autopercepção das condições de saúde bucal por idosos. Rev Saúde Pública 2001; 35(4): 349-55..

With the assessment of health systems, matters such as accessibility, coverage and equity were evaluated. Quality is frequently seen as a primordial factor to be considered for the assessment of health services1414. Donabedian A. Evaluating the quality of medical care. Milbank Quarterly 2005; 83(4): 691-729. , 1515. Vuori HV. Quality Assurance of Health Services. Concepts and Methodology. Copenhagen: WHO Regional office for Europe; 1982.. Dobabedian1414. Donabedian A. Evaluating the quality of medical care. Milbank Quarterly 2005; 83(4): 691-729. proposes that the best strategy to assess such services should involve a set of indicators representing the triad: structure-process-results. Structure refers to characteristics involving providers, instruments and resources, as well as physical and organization conditions of services; the process would correspond to the relationship established between professionals and patients during the performed activities; and the results would be the changes observed in health status, knowledge, behavior, and user satisfaction resulting from the provided care1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60..

The most publicized theoretical model to analyze the associated factors regardless of dental services was the one by Andersen and Davidson1717. Andersen RM, Davidson PL. Ethnicity, aging, and oral health outcomes: a conceptual framework. Adv Dent Res 1997; 11(2): 203-9.. Its expanded version was specifically developed to assess the use of dental services among adults and elders in the United States, in 1997. This version proposes that primary determinants of oral health (external and personal characteristics of the population and system of oral health care) influence behavior (use of dental services and personal oral hygiene practices). These on the other hand, influence oral health outcomes (oral health condition assessed by a professional or self-perceived by individuals and user satisfaction). In this sense, one of the oral health outcomes of the referred model is the assessment of health services from the perspective of the users.

Based on the exposed, this study aimed at identifying the prevalence of negative evaluation of dental services and analyzing if it was higher among users of public or philanthropic services in relation to those who use private or insurance-based plans. It also aimed at identifying the factors associated with the negative assessment of dental services among the Brazilian elderly population, using data from Project SB Brasil 2002/20031212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004.. The previously described model1818. Martins AMEL, Barreto SM, Pordeus IA. Características associadas ao uso de serviços odontológicos entre idosos dentados e edentados no Sudeste do Brasil: Projeto SB Brasil. Cad Saúde Pública 2008; 24(1): 81-92. proposed by Andersen and Davidson in 19971717. Andersen RM, Davidson PL. Ethnicity, aging, and oral health outcomes: a conceptual framework. Adv Dent Res 1997; 11(2): 203-9. was used as theoretical reference and considered the triad proposed by Dobanedian1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60..

METHODS

The project SB Brasil 2002/2003 involved 108,921 individuals living in 250 cities of the give Brazilian macroregions, selected by cluster probability sample by raffle. Interviews and examinations were conducted in the households and schools. Data collection was performed by dental surgeons who were trained and calibrated from the Kappa statistics calculation (> 0.61) and/or percentage concordance (variable % according to the assessed oral condition) This was made in order to allow examiners to go to field, however, the organizers of SB Brasil 2002/2003 did not publicize the concordance values obtained from calibration. Examinations were conducted under natural lighting, with the assistance of a mouth mirror, wooden spatulas and periodontal probe1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004.. According to a proposal by the World Health Organization (WHO)1919. World Health Organization. Oral health surveys: basic methods. 4 ed. Geneva: ORH/EPID; 1997., besides sociodemographic conditions, as well as those of evaluation and use of dental services, the survey investigated the self-perceived need for dental care and oral health, as well as health conditions (cavity, need for treatment, use need for prosthesis, changes in soft tissues) among the elderly1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004..

In the analysis, 5,349 elderly individuals were examined and interviewed (aged from 65 to 74 years old). This investigation used a piece of the data base, considering elderly people (65 to 74 years old) who used dental services at least once in life and who answered the question about the evaluation of dental services, accounting for 5,013 participants.

The triad proposed by Donabedian1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60. involves a set of indicators that can be used to assess health services: structure-process-results. In this research, no variables concerning the structure indicator were found. Process was referred to as the question concerning access to information about how to avoid oral problems. The indicator 'results' was assessed from user satisfaction.

The studied dependent variable "Assessment of Dental Services" was obtained from sequential answers to the questions: "Have you ever been to the dentist? How long ago? (have never been to the dentist/less than one year/ from one to two years/ three years or more); How do you evaluate assistance?" (Very poor / Poor / Fair / Good / Very Good). This investigation excluded participants who have never been to the dentist, and therefore, did not evaluate dental services. The answers to the last question were separated into three categories, respectively: positive assessment - those who considered dental services to be good/very good; intermediate assessment - the ones who considered the service to be fair; and negative assessment - as poor/very poor.

The main independent variable was: used health care system (insurance health plan/private; public/philanthropic). Besides, the variables described as independent proposed in the theoretical model by Andersen and Davidson1717. Andersen RM, Davidson PL. Ethnicity, aging, and oral health outcomes: a conceptual framework. Adv Dent Res 1997; 11(2): 203-9. were the ones gathered in five groups: external environment, individual characteristics, health-related behavior, normative and subjective health conditions. External environment included: Brazilian macroregions (Southeast; South; Center-West; Northeast; North) and place of residency (urban; rural zones). The investigated individual characteristics were: age in years (65 to 69; 70 to 74 years old), sex (female; male), race/self-declared color (white and yellow/brown/black/indigenous), schooling in years (five years or more; one to four years; zero) and access to information about how to avoid oral problems (yes; no).

The availability of resources was evaluated by per capita household income in reais in tertiles (R$ 201.00 or more; R$ 100.00 to 200.00; R$ 0.00 to 99.00). The variable referring to health-related behavior included the use of dental services (three years ago or less; more than three years ago). Normative health conditions referred to the presence of changes in soft tissues (no; yes), number of permanent teeth (10 or more; one to nine; zero), number of permanent teeth with cavity (zero; one to three; four or more), edentulism (no; yes) and use of prosthesis (no; yes). Subjective conditions related to oral health included self-perceived pain in teeth and gum in the past six months (no; yes), oral health (very good/good/fair; poor/very poor), appearance of gums (very good/good/fair; poor/very poor), mastication (very good/good/fair; poor/very poor), speech as to teeth and gums (very good/good/fair; poor/very poor), relationship status due to oral health (does not affect it / affects it a little; affects it more or less/ affects it a lot) and the need for treatment (no; yes).

Statistical analyses were conducted with the statistical softwares Predictive Analytics Software (PASW(r), version 17 for Windows) and Stata (StataCorp LP, version 12.0). Initially, the Brazilian elderly people were characterized according to the evaluation of dental services, external environment, individual characteristics, health-related behavior, and normative and subjective health conditions. Afterwards, univariate analyses were conducted to identify factors associated with the negative evaluation of dental services among the Brazilian elderly population, from the χ2tendency test (significance level < 0.25). The score test was used for each independent variable in order to check if the proportional Odds assumption had been violated2020. Brant R. Assessing proportionality in the proportional odds model for ordinal logistic regression. Biometrics 1990; 46(4): 1171-8.. Violation was found for the schooling (p = 0.00503), per capita income (p = 0.02914), self-perceived pain (p = 0.02464, oral health (p = 0.0005), need for dental treatment (p = 0.01075), relationship status (p = 0.00214) and speech (p = 0.00712) variables.

For the other independent variables, p values ranged from 0.1256 and 0.8409. Therefore, the partial proportional Odds model was chosen in the multiple analysis, which was conducted by using the gologit2 command in Stata2121. Williams R. Generalized Ordered Logit/ Partial Proportional Odds Models for Ordinal Dependent Variables. The Stata Journal 2006; 6(1): 58-82. , 2222. Abreu MNS, Siqueira AL, Caiaffa WT. Regressão logística ordinal em estudos epidemiológicos. Rev Saúde Pública 2009; 43(1): 183-94.. This software tests the assumption of proportional Odds by means of the autofit option, and adjusts coefficients for the several categories in which this assumption is violated. The model enables independent variables to be modeled by obtaining coefficients for each compared category.

All of the variables that were associated with a 5% significance level remained in the models. Correction based on the design effect was made by using the svy command in Stata to analyze data coming from complex samples. This adjustment was necessary because the sample of the project SB Brasil was conducted by clusters, and estimates that do not consider the cluster organization of the sample tend to overestimate the loss of estimation precision. All of the people who were selected to be part of the study signed an informed consent form according to the ethical principles of the Declaration of Helsinki, which are present in the resolution by the National Health Council 196/96, approved by the National Commission of Research Ethics - CONEC (report 581/2000).

RESULTS

Out of the 5,349 elderly participants in the project SB Brasil, 5,013 (93.7%) used dental services and answered the question about the evaluation of dental services, therefore, they were included in this study. Out of the 5,013 elderly people, 196 (3.7%) assessed dental services negatively (poor/very poor), 496 (10.7%), intermediately, and 4,321 (85.6%), positively.

Most elderly participants were female, aged between 65 and 69 years old, who were self-declared as non-white, with one to four schooling years, who had no access to information about how to avoid oral problems, with declared per capita income of R$ 100.00 to 200.00, living in the South and Northeast macroregions of the country, mostly concentrated in the urban zone. From the participants, 31.8% used dental services three years ago or less, and 68.2% of the elderly used it more than three years ago. As to the used health care system, the private type was prevalent (Table 1).

Table 1
Characterization of the elderly, according to the evaluation of dental services, external environment, individual characteristics, health-related behaviors, health conditions normative, and subjective health conditions, SB Brazil, 2002/2003 (n = 5,013).

In the univariate analyses, it was observed that several factors were associated with the negative evaluation of dental services with significance lower or equal to 0.25, among them, the adopted health care system (Table 2).

Table 2
Univariate analysis to identify the factors associated with negative evaluation (bad or poor) of dental services among the elderly, SB Brazil, 2002/2003.

In the multiple analysis, the prevalence of the negative evaluation of dental services was higher among users of public or philanthropic services in relation to Brazilian elderly people who attended private services or insurance-based plans. Besides, six other variables remained associated with the negative assessment of dental services (Table 3).

Table 3
Multiple analysis of factors associated with negative evaluation of dental services among the elderly, SB Brazil, 2002/2003.

DISCUSSION

The velocity of the demographic and epidemiological transition process brings a series of crucial matters for administrators and researchers of the health system, with repercussions for the society as a whole, especially in a context of severe social inequity, poverty and fragile institutions2323. Veras R. Envelhecimento populacional contemporâneo: demandas, desafios e inovações. Rev Saúde Pública 2009; 43(3): 548-54.. Identifying the prevalence of the negative evaluation of dental services, observing if it is higher among users of public or philanthropic services in comparison to the elderly individuals who use private services or insurance-based plans, as well as identifying the factors associated with the negative evaluation of these services from the perspective of the user are important for the attempt to overcome the limitations of the biomedical service. Since the disease is considered to be a biological process in this model, sociocultural and community questions and the perception of people about their health conditions are not taken into account in the evaluation of the health-disease process2424. Roseiro MNV, Takayanagui AMM. Novos indicadores no processo saúde-doença. Saude 2007; 33(1): 37-42..

The precariousness of the oral health condition of the elderly and the fact that most of them need some sort of treatment1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004. may be responsible for the negative evaluation of dental services. On the other hand, edentulism that affects more than half of the elderly1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004. seems to have been accepted as a natural phenomenon. Despite the precarious health conditions and the fact of being toothless1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004., most of these elders self-evaluated their oral health positively2525. Martins AMEBL, Barreto SM, Pordeus IA. Auto-avaliação de saúde bucal em idosos: análise com base em modelo multidimensional. Cad Saúde Pública 2009; 25(2): 421-35.. However, it is important to emphasize that these people come from a time when losing teeth was normal, and oral health conditions reflected the model of health care adopted in the country, with little accessibility, lack of equity, inefficient basic care and mutilating treatments2626. Moreira RS, Nico LS, Tomita NE, Ruiz T. A saúde bucal do idoso brasileiro: revisão sistemática sobre o quadro epidemiológico e acesso aos serviços de saúde bucal. Cad Saúde Pública 2005; 21(6): 1665-75.. Therefore, the low prevalence of negative evaluations observed in this investigation may be based on the acceptance of these precarious oral health conditions, which is based on a cultural matter.

A few studies approached the assessment of dental services. When the accessibility dimension was considered, as well as some aspects involving the physical environment of dental services, most users were not satisfied2727. Lima ACS, Cabral ED, Vasconcelos MMVB. Patient satisfaction at Specialized Dental Clinics, in Recife, Pernambuco, State, Brazil. Cad Saúde Pública 2010; 26(5): 991-1002.. In this investigation, questions referring to accessibility and the physical environment were not assessed. When general health services were compared to dental services, most users were mostly dissatisfied with dental services2828. Moimaz SAS, Marques JAM, Saliba O, Garbin CAS, Zina LG, Saliba NA. Satisfação e percepção dos usuários do SUS sobre o serviço Público de Saúde. Physis Revista de Saúde Coletiva 2010; 20(4): 1419-40.. In spite of that, regardless of the several forms of conceiving and measuring user satisfaction, such as in this investigation, most studies had low dissatisfaction prevalence2727. Lima ACS, Cabral ED, Vasconcelos MMVB. Patient satisfaction at Specialized Dental Clinics, in Recife, Pernambuco, State, Brazil. Cad Saúde Pública 2010; 26(5): 991-1002. , 2929. Esperedião M, Trad LAB. Avaliação da satisfação de usuários: considerações teórico-conceituais. Cad Saúde Pública 2006; 22(6): 1267-76. , 3030. Santos SAS, Meneghim MC, Pereira AC. Análise da organização da demanda e grau de satisfação do profissional e usuário de serviço público odontológico do Município de Campos de Goytacazes/RJ/Brasil. Rev Odontol UNESP 2007; 36(2): 169-74.. This fact is known as the "elevation" effect of satisfaction rates, and is reported even when expectations about the services are negative. It could be expected that users in developing countries, like Brazil, where access to qualified health care is still precarious, would mostly be dissatisfied with services that are provided by SUS or by philanthropic organizations2828. Moimaz SAS, Marques JAM, Saliba O, Garbin CAS, Zina LG, Saliba NA. Satisfação e percepção dos usuários do SUS sobre o serviço Público de Saúde. Physis Revista de Saúde Coletiva 2010; 20(4): 1419-40.. On the other hand, the results found in this article can be a consequence of good and resolutive provided dental services offered by health insurances or private entities.

The multiple analysis showed that, from the five groups proposed by Andersen and Davidson1717. Andersen RM, Davidson PL. Ethnicity, aging, and oral health outcomes: a conceptual framework. Adv Dent Res 1997; 11(2): 203-9., only health-related behaviors and normative health conditions were not associated with the negative evaluation of dental services. Also among the questions related to the structure-process-results triad, which would be important measures to evaluate these services1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60., no variable concerning structure was found. With regard to process, the investigation of access to information about how to avoid oral problems was assessed according to the proposal by Donabedian1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60.. It is believed that the lack of statistical significance (and associations) between the dependent variables and those proposed in the theoretical model is due to the absence of variables collected in the project SB Brasil12 12. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004.that could represent all of the theoretical dimensions of the proposed model1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60.. Therefore, it was not possible to evaluate the questions concerning structure. In the data base of the project SB Brasil, only one question could be assessed to identify the association between the negative evaluation of dental services and the "process" (access to information about how to avoid oral problems)1212. Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004.. However, it is known that the assessment of the relationship between professionals and users would require the analysis of other questions, such as respect, waiting time, freedom to ask questions, among others1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60..

The evaluation of the quality of health care is an important tool for the planning and management of health services and systems3131. Westaway MS, Rheeder P, Zyl DGV, Seager JR. Interpersonal and organizational dimensions of patient satisfaction: the moderating effects of health status. Int J Qual Health Care 2003; 15(4): 337-44.. From that perspective, the idea of good quality in a service is considered when the problem is solved, as well as in situations in which the care provided to the user leads to satisfaction both concerning physical structure and the relationship between professionals and users1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60.. Some studies were conducted with the objective of assessing the level of user satisfaction with the health services66. Gouveia GC, Souza WV, Luna CF, Souza-Júnior PRB, Szwarcwald CL. Satisfação dos usuários do sistema de saúde brasileiro: fatores associados e diferenças regionais. Rev Bras Epidemiol 2009; 12(3): 281-96. , 88. Robles ACC, Grosseman S, Bosco VL. Satisfação com o atendimento odontológico: estudo qualitativo com mães de crianças atendidas na Universidade Federal de Santa Catarina. Cienc Saúde Coletiva 2008; 13(1): 43-9. , 3232. Oliveira DF, Arieta CEL, Temporini ER, Kara-Jose N. Quality of health care: patient satisfaction in a university hospital. Arq Bras Oftalmol 2006; 69(5): 731-6. , 3333. Trad LAB, Bastos ACS, Santana EM, Nunes MO. Estudo etnográfico da satisfação do usuário do Programa de Saúde da Família (PSF) na Bahia. Cienc Saúde Colet 2002; 7(3): 581-9.. Approaching this situation means to evaluate the characteristics of the services, therefore, their quality. Thus, the perspective of users provides important information to improve the quality of services3434. Atkinson SJ. Anthropology in research on the quality of health services. Cad Saúde Pública 1993; 9(3): 283-99.. No associations were found between the negative assessment of services and normative conditions of oral health. According to the proposal by Donabedian1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60., such services could be considered as non-satisfactory, once the results, that is, the oral health of the elderly, were negative. It is important to mention that these conditions are consequences of services provided throughout the lives of the elderly, and that they probably analyzed the services used more recently, since the evaluation of dental services was based on the questions: "Have you ever been to the dentist?"; "How long ago?"; and "How do you evaluate care?". However, statistically significant associations were observed with subjective questions, which can also be considered as health conditions, that is, results. According to the Health Department of the United Kingdom, oral health involves a set of attributes that guarantees mastication, speech and the socialization of individuals, besides the non-existence of disease or discomfort, aiming at the general well-being3232. Oliveira DF, Arieta CEL, Temporini ER, Kara-Jose N. Quality of health care: patient satisfaction in a university hospital. Arq Bras Oftalmol 2006; 69(5): 731-6.. Therefore, the consideration of subjective questions is also a type of health measure. In this context, when the proposal is to evaluate oral health services, it is necessary to understand the oral health conditions of the users2626. Moreira RS, Nico LS, Tomita NE, Ruiz T. A saúde bucal do idoso brasileiro: revisão sistemática sobre o quadro epidemiológico e acesso aos serviços de saúde bucal. Cad Saúde Pública 2005; 21(6): 1665-75., as well as their perception about these conditions.

The prevalence of a negative assessment was higher among the elderly who used public and/or philanthropic services. Studies that approached user satisfaction with public health services point out to the need for improvement in relation to assistance, humanization, reception and the acquisition of physical and material resources2828. Moimaz SAS, Marques JAM, Saliba O, Garbin CAS, Zina LG, Saliba NA. Satisfação e percepção dos usuários do SUS sobre o serviço Público de Saúde. Physis Revista de Saúde Coletiva 2010; 20(4): 1419-40.. In this investigation, questions concerning humanization, reception and the acquisition of physical and material resources were not evaluated, which may have compromised the presented results. The restructure of public health services is important to meet the requirements of these patients, who are extremely in need of care3737. Melo I, Costa D, Maciel SML, Cavalcanti AL. Acesso aos serviços odontológicos e motivos da procura por atendimento por pacientes idosos em Campina Grande - PB. Odontol Clin Cientif 2008; 7(4): 331-5..

The negative assessment of dental services was higher among men, since this perception can be influenced by gender2828. Moimaz SAS, Marques JAM, Saliba O, Garbin CAS, Zina LG, Saliba NA. Satisfação e percepção dos usuários do SUS sobre o serviço Público de Saúde. Physis Revista de Saúde Coletiva 2010; 20(4): 1419-40.. The male participant points out to oral health problems as limiting factors in the performance of activities more than women3838. Pinheiro RS, Viacava F, Travassos C, Brito AS. Gênero, morbidade, acesso e utilização de serviços de saúde no Brasil. Cienc Saúde Colet 2002; 7(4): 687-707., even though women use these services more frequently3737. Melo I, Costa D, Maciel SML, Cavalcanti AL. Acesso aos serviços odontológicos e motivos da procura por atendimento por pacientes idosos em Campina Grande - PB. Odontol Clin Cientif 2008; 7(4): 331-5.

38. Pinheiro RS, Viacava F, Travassos C, Brito AS. Gênero, morbidade, acesso e utilização de serviços de saúde no Brasil. Cienc Saúde Colet 2002; 7(4): 687-707.

39. Matos DL, Giatti L, Lima-Costa MF. Fatores sócio-demográficos associados ao uso dos serviços odontológicos entre idosos brasileiros: um estudo baseado na Pesquisa Nacional de Amostra de Domicílios. Cad Saúde Pública 2004; 20(5): 1290-7.
- 4040. Baldani MH, Brito WH, Lawder JAC, Mendes YBE, Silva FFM, Antunes JLF. Determinantes individuais da utilização de serviços odontológicos por adultos e idosos de baixa renda. Rev Bras Epidemiol 2010; 13(1): 150-62.. The fact that men use the services and care less for their health can be connected to sociocultural issues, to the difficulty to access assistance networks during working hours and to recognize their health needs. Besides, services and communication strategies are mostly addressed to women4141. Gomes R, Nascimento EF, Araújo FC. Por que os homens buscam menos os serviços de saúde do que as mulheres? As explicações de homens com baixa escolaridade e homens com ensino superior. Cad Saúde Pública 2007; 23(3): 565-74.. It is suggested that the higher prevalence of negative assessment of dental services among men can be a result of the presented oral health conditions and the restrictions they generate, as well as of the difficulty to access these services in this stratum during lifetime, once the elderies were investigated3939. Matos DL, Giatti L, Lima-Costa MF. Fatores sócio-demográficos associados ao uso dos serviços odontológicos entre idosos brasileiros: um estudo baseado na Pesquisa Nacional de Amostra de Domicílios. Cad Saúde Pública 2004; 20(5): 1290-7.. It is known that most public or philanthropic services are provided at times when men are at work, therefore, public health policies addressed to men are recommended to ensure the access to such services during the night, which could change this scenario.

Schooling can be reflected on socioeconomic conditions. In this sense, it is believed that income interferes in the evaluation of health services. In this investigation, the higher prevalence of negative evaluation among those with higher schooling may have been found due to the homogeneity of the population as to income4242. Martins AMEBL, Barreto SM, Silveira MF, Santa-Rosa TT, Pereira RD. Autopercepção da saúde bucal entre idosos brasileiros. Rev Saúde Pública 2010; 44(5): 912-22.. It is suggested that the higher prevalence of negative assessment among those with higher schooling, regardless of the type of used service, is due to the fact that the elderly people with more schooling years are more demanding concerning the provided services. Besides, the health-related education can influence the evaluation of services1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60.. The elders who were informed about how to avoid oral problems reported being more dissatisfied with the provided services4343. Oliveira RS, Magalhães BG, Gaspar GS, Rocha RACP, Góes PSA. Avaliação do grau de satisfação dos usuários nos serviços de saúde bucal da Estratégia de Saúde da Família. Rev Bras Pesq Saúde 2009; 11(4): 34-8. , 4444. Mialhe FL, Oliveira CSR, Silva DD. Acesso e avaliação dos serviços de saúde bucal em uma localidade rural da região sul do Brasil. Arq Cienc Saúde Unipar 2006; 10(3): 145-9.. Health-related education contributes with the awareness of individuals for the valorization of oral health, which would facilitate the practice of healthy habits and behaviors4545. Saliba NA, Pereira AA, Moimaz SAS, Garbin CAS, Arcieri RM. Programa de educação em saúde bucal: A experiência da Faculdade de Odontologia de Araçatuba - UNESP. Odontol Clin Cientif 2003; 2(3): 197-200.. In this investigation, the negative evaluation was more common among those who were not informed about how to avoid oral problems, regardless of the type of used service. Therefore, the recommendation is that in order for the low percentage of elderly people who negatively assessed dental services (3.7%) to change their perspective, one of the changes to be considered is the improvement of information provided about how to avoid oral problems, especially in public and philanthropic services.

The subjective conditions related to health may interfere in the outcome of the evaluation of health services1616. Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60.. Among the seven investigated conditions in this study, three of them were associated with the negative evaluation of dental services. No previous studies were found that identified such associations. However, it is known that self-perception among the elderly is influenced by the perception of several oral health aspects4646. Martins AMEBL, Barreto SM, Pordeus IA. Factors associated to self perceived need of dental care among Brazilian elderly. Rev Saúde Pública 2008; 42(3): 487-96.

47. Ekanayake L, Perera I. Perceived need for dental care among dentate older individuals in Sri Lanka. Spec Care Dentist 2005; 25(4): 199-205.
- 4848. Heft MW, Gilbert GH, Shelton BJ, Ducan RP. Relationship of dental status, sociodemographic status, and oral symptoms to perceived need for dental care. Community Dent Oral Epidemiol 2003; 31(5): 351-60.. The positive perception of oral health, even among elderly people with precarious oral health conditions, suggests that normative conditions are poor self-perception predictors4848. Heft MW, Gilbert GH, Shelton BJ, Ducan RP. Relationship of dental status, sociodemographic status, and oral symptoms to perceived need for dental care. Community Dent Oral Epidemiol 2003; 31(5): 351-60.

49. Benyamini Y, Leventhal H, Leventahal EA. Self rated oral health as an independent predictor of self rated general health, self esteem and life satisfaction. Soc Sci Med 2004; 59(5): 1109-16.

50. Reisine ST, Bailit HL. Clinical oral health status and adult perceptions of oral health. Soc Sci Med 1980; 14(6): 597-605.
- 5151. Matthias RE, Atchison KA, Lubben JE, De Jong F, Schweitzer SO. Factors affecting self-ratings of oral health. J Public Health Dent 1995; 55(4): 197-204.. This probably occurs because most oral diseases are asymptomatic and unknown by the patients, who only consider being sick in face of the acute manifestation of oral diseases5050. Reisine ST, Bailit HL. Clinical oral health status and adult perceptions of oral health. Soc Sci Med 1980; 14(6): 597-605., or even because normative conditions measure only the disease, while subjective questions assess human health experiences5252. Locker D, Slade G. Association between clinical and subjective indicators of oral health status in an older adult population. Gerodontology 1994; 11(2): 108-14..

The limitations in this study refer to the dependent variable that considers the evaluation of dental services from a single question; and the dynamic process of association between the assessment of dental services and the investigated variables, once causes and effects certainly vary throughout life. Since this is a sectional study, it is not possible to establish a temporal relationship between the observed associations. Besides, it is important to mention that in studies with complex samples it is recommended to perform correction based on the design effect, in statistical analyses5353. Queiroz RCS, Portela MC, Vasconcellos MTL. Pesquisa sobre as Condições de Saúde Bucal da População Brasileira (SB Brasil 2003): seus dados não produzem estimativas populacionais, mas há possibilidade de correção. Cad Saúde Pública 2009; 25(1): 47-58.. Such corrections lead to the lower impact in inferential statistics, which is objective of this investigation, in relation to descriptive ones5454. Narvai PC, Antunes JLF, Moysés SJ, Frazão P, Peres MA, Glazer K, et al. Validade científica de conhecimento epidemiológico gerado com base no estudo Saúde Bucal Brasil 2003. Cad Saúde Pública 2010; 26(4): 647-70., so part of it was made by the svy command of Stata to analyze data from complex samples. Weighing was not performed since the necessary data to estimate the different probabilities of sample composition were not made available.

CONCLUSIONS

The knowledge about the evaluation of dental services from the perspective of the users should be considered in health policies, in health-promotion, preventive and rehabilitating actions, which try to improve the quality of life of the population. The prevalence of negative evaluation was low, and higher among users of public and philanthropic users, regardless of the other investigated variables. Associations with variables of three from the five subgroups proposed in the theoretical model were found.

ACKNOWLEDGEMENTS

We would like to thank the Oral Health Coordination of the Ministry of Health and the team of the survey field, Faculdades Unidas do Norte de Minas (FUNORTE/SOEBRAS) and Universidade Estadual de Montes Claros (UNIMONTES) for the logistical support, as well as the participants of the study. Martins is a post-doc scholarship participant from the National Council of Scientific and Technological Development (CNPq); Souza was a scholarship participant of scientific initiation at FUNORTE/SOEBRAS; Ferreira is a scholarship participant of productivity of Fundação de Amparo à Pesquisa do estado de Minas Gerais (FAPEMIG), and Pordeus is a scholarship participant of productivity of CNPq.

REFERENCES

  • 1
    Schmidt MI, Duncan BB, Azevedo e Silva G, Menezes AM, Monteiro CA, Barreto SM, et al. Chronic non-communicable diseases in Brazil: burden and current challenges. Lancet 2011; 377(9781): 1949-61.
  • 2
    Lima-Costa MF, Veras R. Saúde pública e envelhecimento. Cad Saúde Pública 2003; 19(3): 700-1.
  • 3
    Benedetti TRB, Mello ALSF, Gonçalves LHT. Idosos de Florianópolis: autopercepção das condições de saúde bucal e utilização de serviços odontológicos. Cienc Saúde Coletiva 2007; 12(6): 1683-90.
  • 4
    Colussi CF, Freitas SFT. Aspectos epidemiológicos da saúde bucal do idoso no Brasil. Cad Saúde Pública 2002; 18(5): 1313-20.
  • 5
    Moreira MM. O envelhecimento da população brasileira em nível regional; 1940-2050. In: XI Encontro Nacional de Estudos Populacionais. Anais... Caxambu: Associação Brasileira de Estudos Populacionais; 1998. p. 3030-3124.
  • 6
    Gouveia GC, Souza WV, Luna CF, Souza-Júnior PRB, Szwarcwald CL. Satisfação dos usuários do sistema de saúde brasileiro: fatores associados e diferenças regionais. Rev Bras Epidemiol 2009; 12(3): 281-96.
  • 7
    Favaro P, Ferris LE. Program evaluation with limited fiscal and human resources. Cad Saúde Pública 1991; 11(3): 425-38.
  • 8
    Robles ACC, Grosseman S, Bosco VL. Satisfação com o atendimento odontológico: estudo qualitativo com mães de crianças atendidas na Universidade Federal de Santa Catarina. Cienc Saúde Coletiva 2008; 13(1): 43-9.
  • 9
    Bottan ER, Sperb RAL, Telles PS, Uriarte Neto M. Avaliação de serviços odontológicos: a visão dos pacientes. Rev ABENO 2006; 6(2): 128-33.
  • 10
    Chiavenato I. Gerenciando pessoas: o passo decisivo para a administração participativa. 2 ed. São Paulo: Makron Books; 1994.
  • 11
    Gattinara BC, Ibacache J, Puente CT, Giaconi J, Caprara A. Percepción de la comunidad acerca de la calidad de los servicios de salud públicos en los Distritos Norte e Ichilo, Bolivia. Cad Saúde Pública 1995; 14(1): 61-70.
  • 12
    Brasil. Ministério da Saúde. Projeto SB Brasil 2003: condições de saúde bucal da população brasileira 2002-2003: resultados principais. Brasília: Ministério da Saúde; 2004.
  • 13
    Silva SRC, Fernandes RAC. Autopercepção das condições de saúde bucal por idosos. Rev Saúde Pública 2001; 35(4): 349-55.
  • 14
    Donabedian A. Evaluating the quality of medical care. Milbank Quarterly 2005; 83(4): 691-729.
  • 15
    Vuori HV. Quality Assurance of Health Services. Concepts and Methodology. Copenhagen: WHO Regional office for Europe; 1982.
  • 16
    Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin 1992; 18: 356-60.
  • 17
    Andersen RM, Davidson PL. Ethnicity, aging, and oral health outcomes: a conceptual framework. Adv Dent Res 1997; 11(2): 203-9.
  • 18
    Martins AMEL, Barreto SM, Pordeus IA. Características associadas ao uso de serviços odontológicos entre idosos dentados e edentados no Sudeste do Brasil: Projeto SB Brasil. Cad Saúde Pública 2008; 24(1): 81-92.
  • 19
    World Health Organization. Oral health surveys: basic methods. 4 ed. Geneva: ORH/EPID; 1997.
  • 20
    Brant R. Assessing proportionality in the proportional odds model for ordinal logistic regression. Biometrics 1990; 46(4): 1171-8.
  • 21
    Williams R. Generalized Ordered Logit/ Partial Proportional Odds Models for Ordinal Dependent Variables. The Stata Journal 2006; 6(1): 58-82.
  • 22
    Abreu MNS, Siqueira AL, Caiaffa WT. Regressão logística ordinal em estudos epidemiológicos. Rev Saúde Pública 2009; 43(1): 183-94.
  • 23
    Veras R. Envelhecimento populacional contemporâneo: demandas, desafios e inovações. Rev Saúde Pública 2009; 43(3): 548-54.
  • 24
    Roseiro MNV, Takayanagui AMM. Novos indicadores no processo saúde-doença. Saude 2007; 33(1): 37-42.
  • 25
    Martins AMEBL, Barreto SM, Pordeus IA. Auto-avaliação de saúde bucal em idosos: análise com base em modelo multidimensional. Cad Saúde Pública 2009; 25(2): 421-35.
  • 26
    Moreira RS, Nico LS, Tomita NE, Ruiz T. A saúde bucal do idoso brasileiro: revisão sistemática sobre o quadro epidemiológico e acesso aos serviços de saúde bucal. Cad Saúde Pública 2005; 21(6): 1665-75.
  • 27
    Lima ACS, Cabral ED, Vasconcelos MMVB. Patient satisfaction at Specialized Dental Clinics, in Recife, Pernambuco, State, Brazil. Cad Saúde Pública 2010; 26(5): 991-1002.
  • 28
    Moimaz SAS, Marques JAM, Saliba O, Garbin CAS, Zina LG, Saliba NA. Satisfação e percepção dos usuários do SUS sobre o serviço Público de Saúde. Physis Revista de Saúde Coletiva 2010; 20(4): 1419-40.
  • 29
    Esperedião M, Trad LAB. Avaliação da satisfação de usuários: considerações teórico-conceituais. Cad Saúde Pública 2006; 22(6): 1267-76.
  • 30
    Santos SAS, Meneghim MC, Pereira AC. Análise da organização da demanda e grau de satisfação do profissional e usuário de serviço público odontológico do Município de Campos de Goytacazes/RJ/Brasil. Rev Odontol UNESP 2007; 36(2): 169-74.
  • 31
    Westaway MS, Rheeder P, Zyl DGV, Seager JR. Interpersonal and organizational dimensions of patient satisfaction: the moderating effects of health status. Int J Qual Health Care 2003; 15(4): 337-44.
  • 32
    Oliveira DF, Arieta CEL, Temporini ER, Kara-Jose N. Quality of health care: patient satisfaction in a university hospital. Arq Bras Oftalmol 2006; 69(5): 731-6.
  • 33
    Trad LAB, Bastos ACS, Santana EM, Nunes MO. Estudo etnográfico da satisfação do usuário do Programa de Saúde da Família (PSF) na Bahia. Cienc Saúde Colet 2002; 7(3): 581-9.
  • 34
    Atkinson SJ. Anthropology in research on the quality of health services. Cad Saúde Pública 1993; 9(3): 283-99.
  • 35
    United Kingdom. Department of Health. An oral health strategy for England. London; 1994.
  • 36
    Matos DL, Lima-Costa MF, Guerra HL, Marcenes W. Projeto Bambuí: avaliação de serviços odontológicos privados, públicos e de sindicato. Rev Saúde Pública 2002; 36(2): 237-43.
  • 37
    Melo I, Costa D, Maciel SML, Cavalcanti AL. Acesso aos serviços odontológicos e motivos da procura por atendimento por pacientes idosos em Campina Grande - PB. Odontol Clin Cientif 2008; 7(4): 331-5.
  • 38
    Pinheiro RS, Viacava F, Travassos C, Brito AS. Gênero, morbidade, acesso e utilização de serviços de saúde no Brasil. Cienc Saúde Colet 2002; 7(4): 687-707.
  • 39
    Matos DL, Giatti L, Lima-Costa MF. Fatores sócio-demográficos associados ao uso dos serviços odontológicos entre idosos brasileiros: um estudo baseado na Pesquisa Nacional de Amostra de Domicílios. Cad Saúde Pública 2004; 20(5): 1290-7.
  • 40
    Baldani MH, Brito WH, Lawder JAC, Mendes YBE, Silva FFM, Antunes JLF. Determinantes individuais da utilização de serviços odontológicos por adultos e idosos de baixa renda. Rev Bras Epidemiol 2010; 13(1): 150-62.
  • 41
    Gomes R, Nascimento EF, Araújo FC. Por que os homens buscam menos os serviços de saúde do que as mulheres? As explicações de homens com baixa escolaridade e homens com ensino superior. Cad Saúde Pública 2007; 23(3): 565-74.
  • 42
    Martins AMEBL, Barreto SM, Silveira MF, Santa-Rosa TT, Pereira RD. Autopercepção da saúde bucal entre idosos brasileiros. Rev Saúde Pública 2010; 44(5): 912-22.
  • 43
    Oliveira RS, Magalhães BG, Gaspar GS, Rocha RACP, Góes PSA. Avaliação do grau de satisfação dos usuários nos serviços de saúde bucal da Estratégia de Saúde da Família. Rev Bras Pesq Saúde 2009; 11(4): 34-8.
  • 44
    Mialhe FL, Oliveira CSR, Silva DD. Acesso e avaliação dos serviços de saúde bucal em uma localidade rural da região sul do Brasil. Arq Cienc Saúde Unipar 2006; 10(3): 145-9.
  • 45
    Saliba NA, Pereira AA, Moimaz SAS, Garbin CAS, Arcieri RM. Programa de educação em saúde bucal: A experiência da Faculdade de Odontologia de Araçatuba - UNESP. Odontol Clin Cientif 2003; 2(3): 197-200.
  • 46
    Martins AMEBL, Barreto SM, Pordeus IA. Factors associated to self perceived need of dental care among Brazilian elderly. Rev Saúde Pública 2008; 42(3): 487-96.
  • 47
    Ekanayake L, Perera I. Perceived need for dental care among dentate older individuals in Sri Lanka. Spec Care Dentist 2005; 25(4): 199-205.
  • 48
    Heft MW, Gilbert GH, Shelton BJ, Ducan RP. Relationship of dental status, sociodemographic status, and oral symptoms to perceived need for dental care. Community Dent Oral Epidemiol 2003; 31(5): 351-60.
  • 49
    Benyamini Y, Leventhal H, Leventahal EA. Self rated oral health as an independent predictor of self rated general health, self esteem and life satisfaction. Soc Sci Med 2004; 59(5): 1109-16.
  • 50
    Reisine ST, Bailit HL. Clinical oral health status and adult perceptions of oral health. Soc Sci Med 1980; 14(6): 597-605.
  • 51
    Matthias RE, Atchison KA, Lubben JE, De Jong F, Schweitzer SO. Factors affecting self-ratings of oral health. J Public Health Dent 1995; 55(4): 197-204.
  • 52
    Locker D, Slade G. Association between clinical and subjective indicators of oral health status in an older adult population. Gerodontology 1994; 11(2): 108-14.
  • 53
    Queiroz RCS, Portela MC, Vasconcellos MTL. Pesquisa sobre as Condições de Saúde Bucal da População Brasileira (SB Brasil 2003): seus dados não produzem estimativas populacionais, mas há possibilidade de correção. Cad Saúde Pública 2009; 25(1): 47-58.
  • 54
    Narvai PC, Antunes JLF, Moysés SJ, Frazão P, Peres MA, Glazer K, et al. Validade científica de conhecimento epidemiológico gerado com base no estudo Saúde Bucal Brasil 2003. Cad Saúde Pública 2010; 26(4): 647-70.

  • Financial support: National Council for Scientific and Technological Development (CNPq) and Minas Gerais State Research Foundation (Fapemig).

Publication Dates

  • Publication in this collection
    Mar 2014

History

  • Received
    23 May 2013
  • Reviewed
    01 Sept 2013
  • Accepted
    18 Nov 2013
Associação Brasileira de Pós -Graduação em Saúde Coletiva São Paulo - SP - Brazil
E-mail: revbrepi@usp.br