ORIGINAL ARTICLES

User satisfaction with psychosocial healthcare services, Southern Brazil

Satisfacción de los usuarios de los centros de atención psicosocial de la región Sur de Brasil

Luciane Prado KantorskiI; Vanda da Rosa JardimI; Christine WetzelII; Agnes OlschowskyII; Jacó Fernando SchneiderII; Rita Maria HeckI; Valquíria de Lourdes BielemannI; Eda SchwartzI; Valéria Cristina Christello CoimbraI; André Luis Alves de QuevedoI; Suélen dos Santos SaraivaI

IDepartamento de Enfermagem. Faculdade de Enfermagem e Obstetrícia. Universidade Federal de Pelotas. Pelotas, RS, Brasil
IIDepartamento de Assistência e Orientação Profissional. Universidade Federal do Rio Grande do Sul. Pelotas, RS, Brasil

Correspondence

ABSTRACT

OBJECTIVE: To evaluate user satisfaction with psychosocial healthcare services.
METHODS: Qualitative and quantitative study conducted in psychosocial healthcare services in the states of Paraná, Santa Catarina and Rio Grande do Sul, Southern Brazil, in 2006. The study combined quantitative and qualitative methods. Quantitative epidemiological data from a cross-sectional study including 1,162 users of 30 psychosocial healthcare services was used. The instrument used to evaluate users' satisfaction was the Brazilian version of the WHO Users' Satisfaction Scale (SATIS-BR), a 1 to 5-point scale. Qualitative data was collected from five case studies using a fourth generation approach. Information was obtained from field observations and interviews (between ten and 13 users in each field, totaling 57 users) and presented to users in validation and negotiation workshops.
RESULTS: The SATIS-BR scale showed that users positively evaluated all items, overall mean 4.4 (SD=0.4). Communication and relationship with psychosocial healthcare services staff had mean 4.5 (SD=0.5), and access to information through staff had mean 4.8. Satisfaction with care service was the lowest, mean 4.1, and general service infrastructure had mean 3.9. The qualitative study revealed that, according to users, the quality of treatment provided was good and the outcome was satisfactory. Their new status, better access to services, ending of their physical and social isolation, response to their demands and helping them reorganize their lives - all contributed to users' satisfaction.
CONCLUSIONS: The complementary results of both study approaches showed that users are satisfied with care provided at the psychosocial healthcare services studied.

Descriptors: Mental Health Services. Patient Satisfaction. Health Services Evaluation. Case Studies. Cross-Sectional Studies.

RESUMEN

OBJETIVO: Evaluar la satisfacción de los usuarios con la atención en los centros de atención psicosocial.
MÉTODOS: El estudio fue realizado en centros de atención psicosocial de los estados del Sur de Brasil: Paraná, Santa Catarina y Rio Grande do Sul, en 2006. La estrategia de investigación fue dividida en métodos cuantitativo y cualitativo. Para evaluación cuantitativa fueron utilizados datos epidemiológicos del tipo transversal de muestra 1.162 usuarios de 30 centros de atención psicosocial. El instrumento utilizado para evaluar la satisfacción de los usuarios fue la Escala Brasilera de Evaluación de la Satisfacción (SATIS-BR), de 1 a 5 puntos. Para evaluación cualitativa fueron realizados cinco estudios de caso, utilizándose metodología de evaluación de cuarta generación. Los datos fueron obtenidos en observación de campo y entrevistas (entre diez y 13 usuarios en cada campo, totalizando 57 usuarios) y presentados a los usuarios en oficinas de validación y negociación.
RESULTADOS: La escala SATIS-BR indicó evaluación positiva de todos los ítems, con promedio de 4,4 (desviación-estándar, DP= 0,4). La comunicación y la relación con el equipo presentaron promedio 4,5 (DP= 0,5): el acceso a informaciones a partir del equipo presentó promedio 4,8. La satisfacción con el servicio presentó el menor promedio, con 4,1; las condiciones generales de instalación de servicio presentaron promedio 3,9. En el estudio cualitativo, el tratamiento en el servicio fue bueno y el resultado satisfactorio. La reubicación del usuario contribuyó para satisfacción, así como el acceso a la atención, la ruptura con el aislamiento físico y social, el acogimiento de sus demandas y ayuda para organizar la vida.
CONCLUSIONES: Los resultados complementarios de ambas estrategias de estudio muestran satisfacción de los usuarios con la atención en los centros estudiados.

Descriptores: Servicios de Salud Mental. Satisfacción del Paciente. Evaluación de Servicios de Salud. Estudios de Casos. Estudios Transversales.

INTRODUCTION

Mental and behavioral disorders cause a great deal of personal and social distress. They affect 450 million people worldwide and the World Health Organization (WHO) has estimated that nearly 25% of people develop at least one mental disorder during their lifetime. In addition to these conditions' economic and social cost, affected individuals are vulnerable to violations of their human rights, and social isolation, and have low quality of life and increased risk of death. The disability-adjusted life years (DALY) for neuropsychiatric disorders is reported as 13.8%. When all mental and behavioral disorders and related factors are combined they are as high as 33%, comprising four out of six main causes of disability.ª

In Brazil, despite its tradition of institutionalizing patients that has contributed to stigmatization and social isolation of those suffering from mental conditions, starting from 1970s the movement for psychiatric reform gained momentum advocating comprehensive mental health care with an empowering approach. The psychiatric reform has been brought about by several different movements and conceptual lines.

The guiding principles of deinstitutionalization are reflected in a set of regulations that has been introduced by the Ministério da Saúde (Brazilian Ministry of Health) since the early 1990s and has had a great impact on the implementation of public policies in mental health.b

A new challenge has been set. The Ministério da Saúde has clearly defined mental health policies focused on initiatives and public authorities have been working to assure the delivery of mental health care in accordance with the proposed guidelines. A new scenario for mental health research has been created prioritizing studies based on a systematic scientific approach to evaluate services established during the psychiatric reform process.

The objective of the present study was to evaluate users' satisfaction with attention provided at psychosocial care services.

METHODS

The present study was based on data from a larger study evaluating psychosocial care services (CAPS) in southern Brazil. There were evaluated CAPS facilities in the states of Paraná, Santa Catarina, and Rio Grande do Sul in 2006. The study comprised both quantitative and qualitative approaches.

The quantitative evaluation of quality of care can help guiding the formulation of more effective new strategies for service reestructuring.7-9 This study consisted of three complementary sub studies: description of infrastructure of care facilities; description of the working process and organization of care in CAPS in the southern region; and evaluation of outcomes in mental health care.

The sample studied included coordinators, health workers, family members, and users from 30 facilities (CAPS I and CAPS II) from three different states based on the proportion of services per state.

Three cities were selected to participate in the study in the state of Paraná: Cianorte (CAPS I), and Curitiba and Francisco Beltrão (both CAPS II). Nine cities were selected in Santa Catharina: Xaxim, Timbó, Rio do Sul, Orleans, and Içara (CAPS I), and Caçador, Joinville, Florianópolis, and Criciúma (CAPS II). Nine CAPS I facilities were selected in the state of Rio Grande do Sul: Santo Ângelo, Panambi, Santiago, Bento Gonçalves, Triunfo, Parobé, São Sepé, Santana do Livramento, Capão do Leão; and nine CAPS II facilities: Carazinho, Passo Fundo, Alegrete, Bagé, Santa Maria, Santa Cruz do Sul, Rio Grande, Esteio, and Porto Alegre.

Users and family members were drawn and those included in the sample were interviewed at home and at CAPS facilities.

From the perspective of a potential effect of care model change, the estimate of the sample size was based on the following outcomes: user satisfaction; health status and autonomy, social inclusion and empowerment. The sample size was estimated for an alpha error of 5% and 95% power using Epi Info 6.04. Given the scarcity of studies from this mental health perspective and the fact that the literature has shown different measures and indicators of variability, there were used, values with the smallest likelihood of error. To the highest size value estimated was added 30% for controlling for confounders and losses. Differences in the level of complexity among the three models of care found in CAPS were taken into consideration in the sample. A total of 1,200 users and family members were included in the study.

Given the different proportion of CAPS by state, and due to logistic issues, there was established that 40 users and 40 family members would be interviewed at each facility for a total of 30 CAPS.

The group of 40 users selected was proportionally categorized by complexity of care into intensive (12 subjects), non-intensive (12), and semi-intensive (16).

The study questionnaires were developed and validated according to specific mental care regulations. The instruments were pre-tested in services that did not participate in the study. User satisfaction was evaluated using the Brazilian version of the Users' Satisfaction Scale (SATIS-BR). This scale was developed by the WHO Division of Mental Health and validated in Brazil. It consists of a 5-point Likert scale, where 5 means the greatest satisfaction and 1 means the lowest satisfation.3-5

Data were collected by 14 previously trained interviewers and their work was supervised by two supervisors.

After data collection, the questionnaires were then coded and doubled entered independently into a database using Epi Info. Both entries were compared and corrected for major errors and inconsistencies. All analyses were carried out using Stata program.

Quality was controlled during the application of each data collection instrument. Data was checked by all interviewers at the end of the interview, and questionnaires were reviewed by the supervisors. In addition, 5% of interviews were replicated.

The qualitative study comprised a fourth generation evaluation (FGE), which was constructivist, and responsive technique with a hermeneutic dialectic approach. The FGE, developed by Guba & Lincoln10-12 and adapted by Wetzel (2005),c guided the theoretical-methodological approach of the study. The instruments of data collection were interviews with health workers, users, and family members (defined as the stakeholding groups to form the hermeneutic dialectic circle) and field observations (prior ethnography).

Five CAPS were selected by intention based on data obtained during the quantitative evaluation and their adequacy to the regulations of Decree No. 336/2002.b There were also taken into consideration length of time in operation of a service, length of service of health workers, and availability of stakeholders to be involved.

The qualitative step of the study was conducted as a case study18 including field observations (between 282 and 650 hours in each field) and interviews.

There were interviewed between ten and 13 users in each field (total of 57 users), between ten and 14 family members in each field (total of 60 family members), and between ten and 26 health workers in each field (total of 88 health workers).

After data was collected in the interviews, an interim analysis was conducted to develop workshops for information validation. The analyzed data was presented to the applicable groups so they could have access to all information and to offer them an opportunity for making changes or confirming it as reliable. Negotiation was carried out in a group and all interviewees of a given circle were invited to participate.

Based on units of information extracted from discourses, markers for evaluation were defined to guide analyses. A marker is a given category abstracted from empirical data, which has an explanatory feature indicating an evaluation factor. For example, the marker for outcome indicating qualitative aspects to be evaluated was the mental health outcome, which enabled to understand subjects' satisfaction with service and their new status to respond to daily demands. Markers followed the conceptual principles used in the quantitative step of evaluation focusing on facility infrastructure, process and results.7-9 From this wider perspective, the following markers were subdivided: facility infrastructure - environment; process - activities such as treatment support, care team, characteristics and organization of work process, service flexibility, family involvement; result - psychosocial care outcome.

The study was approved by the Research Ethics Committee of Faculdade de Medicina da Universidade Federal de Pelotas (Protocol 074/05 of 11 November 2005). All subjects signed a free informed consent agreeing to participate in the study.

RESULTS

Quantitative study

There were 38 losses of a total of 1.200 users selected to the sample. Of those users included in the sample, 61,4% attended CAPS in Rio Grande do Sul, which is consistent with greater proportion of services in that state.

Most users were females (63.9%) with mean age of 42 years (standard deviation [SD]=12.3) and 70.9% self-referred as white skin color. Mean income in the previous month was R$338.02 while per capita family income was R$ 264.16.

The medical diagnosis was reported by 97% of users; 34.3% reported major depressive disorder, 28.3% psychosis, and 8.9% schizophrenia.

User satisfaction was positively evaluated for all items, with mean score of 4.4 (SD= 0.4). The results for each item of the scale are shown in Table 1.

Satisfaction with service had the lowest mean score (4.1); general facility infrastructure had mean score of 3.9, and "communication and relationship with the health team" had mean score of 4.5.

The highest means were seen for access to information and respect and dignity (mean scores of 4.8). Items with the lowest means were general facility infrastructure (mean score of 3.9); appearance and comfort, and users' involvement in treatment decision making (mean scores of 4.1).

The positive evaluation supported the decision to return to the service to seek help for 95.9% of the users studied and the recommendation of the service to a friend or family for 96.9% of them. Reasons for recommending the service were good quality of attention, support, and help provided for 36.7% of the users; good relationship with providers and best treatment option for 23.6%; and treatment and medication for 21.2%.

Although users have positively evaluated services, 58.5% believed that CAPS required improvement.

The most common factors that required improvement were: facility infrastructure (21.8%) users mentioned the need for more physical space (care rooms, workshop rooms, yard, sport court), renovations, better hygiene conditions (especially in the restrooms), aesthetically customized spaces (distinctive pleasant and colorful rooms); providers (19%) - better staffing and qualification; activities (15.4%) - greater variety of workshops, tea groups, physical activities, family groups, lounges for meeting people, opportunities for working and selling workshop products; attention (14.6%) - users referred to the need for reducing provider turnover, increasing the number of visits, having medication available at the facilities, treating substance abuse, providing care to more people and having more discussion groups; resources (10.5%) - lack of funds, supplies, and support from local administrations; meeting (8.1%) - increasing interaction between staff and users; medication (5%) - drug dispensing at CAPS and regular provision of drugs.

Qualitative study

According to users, treatment provided at CAPS was good and the outcome was satisfactory (Table 2). As a result, they gained more autonomy, had less crises (reduced frequency and intensity), had more opportunities of social interaction, socialization, and of enriching their daily life which is not restricted to the experience of the mental disorder.

Users reported that attention provided at CAPS facilities was qualitatively better than that provided by the Sistema Único de Saúde (SUS - Brazilian National Health Service). Another factor contributing to their satisfaction was their new status, as they had an opportunity to acquire new skills during workshops, which made them gain more autonomy and be pro-active, teaching the new skills learned to other users. Several factors that qualitatively contributed to their satisfaction were: ending of their social and physical isolation, attending a service where staff are familiar with them (they know their name, family, area), having their demands met (there is a professional who can be contacted even during weekends and off-hours) and having support to reorganize their life (dealing with internal conflicts, family members, community and simple tasks such as managing their benefits over one-month period).

Although users were qualitatively satisfied with the service provided, they referred to the existing poor conditions such lack of materials for workshops, irregular provision of drugs (except for Case 2), limited physical space and activities, understaffing, low qualification and lack of support to CAPS health workers, few activity options, little involvement in management decisions and difficult referral to other local mental health services (especially for admission and basic care).

DISCUSSION

The higher concentration of users and services in the state of Rio Grande do Sul is consistent with data from the Ministério da Saúde that showed that this state has 0.7 CAPS facilities per 100,000 inhabitants, while the state of Santa Catarina and Paraná have 0.6 and 0.4, respectively. According to the Ministério da Saúde criteria, good to fairly good coverage is seen with higher than 0.5 CAPS facilities per 100,000 inhabitants and very low coverage is seen when it is below 0.2.d

Most CAPS users are women, which corroborates that seen in the sample used for SATIS-BR scale validation where females accounted for 70%.4 Similarly, other studies investigating health services reported higher proportion of women among users.16

Users were on average older than the general population by about ten years6 and younger also by ten years to those subjects of a study including users of home care usually provided to patients discharged after prolonged psychiatric admissions.13 Mean income found in the sample shows the social vulnerability of the population studied and the exclusion of those suffering from mental conditions.14,17 On the other hand, the medical diagnoses reported indicate that CAPS generally provide care to those suffering from more severe conditions, which is consistent with the policy guidelines of the Brazilian psychiatric reform.b

The SATIS-BR scale revealed users' high satisfaction with CAPS in southern Brazil. Another study using the same scale to evaluate home care found mean score of 4.52 (SD= 0.48).13

The findings regarding users' relationship with health workers are similar to those reported in another study.16 The major weakness of services is regarding the infrastructure of facilities, as pointed out in other studies.3,15,16

In the present study, there may be evaluation bias resulting from the users' difficulty of expressing their dissatisfaction or not expressing their gratitude.2 This is true especially for public services in developing countries. In spite of that, given the approach used, the results indicate that CAPS have been a positive reference for mental health care. The services evaluated represent a valuable substitute to the reasoning of exclusion of psychiatric hospitals.

The psychosocial care approach aims at helping patients to acquire a new status. Rather than suffering the effects of conflicts and psychic contradictions, it allows patients to identify themselves as an involved agent and to act as a change agent.6 The evaluation of the present study required a qualitative approach and each one of the 57 users studied was empowered over the course of the evaluation process, showing aspects related to psychosocial care outcome and indicating what could be improved in CAPS. To bring users to the evaluation scenario is challenging and essential to emancipative constructivist processes as proposed in our study. Users' validation and negotiation, as part of the FGE, reinforced the importance of evaluations that create an opportunity for users to express themselves freely, acquire new status, and be empowered.

Freedom, before being an element of thought or will, was the element that enabled human beings to act and go out and meet people, exercising their power of speech and action. In addition, freedom required a public setting where people could meet.1 The present study prioritized in its construction process the creation of public settings where each individual involved in mental health care at CAPS could be included through words and actions in the evaluation. It is expected that the present study can be a reference to be applied in different scenarios such as academic settings, public policies and daily operation of CAPS.

By approaching quantitative and qualitative techniques, there were found convergent and divergent aspects, particularly in regard to users' satisfaction with CAPS. There was an agreement between factors with the highest scores in the SATIS-BR scale and themes that described reasons for satisfaction in the qualitative study. Despite users' significant positive evaluation, they pointed out to aspects that required improvement and adjustments.

As pointed out by users, we emphasize the need for improving facility infrastructure, and providing more supplies and medications as well as better training and qualification to health workers and CAPS staffing. Users also pointed to the need of offering a wider variety of activities and care at CAPS. Both studies also revealed issues related to management, especially concerning financing, user involvement in management decisions as well as service organization and referrals.

REFERENCES

1. Arendt H. Entre o passado e o futuro. São Paulo: Perspectiva; 1979.

2. Avis M, Bond M, Arthur A. Questioning patient satisfaction: an empirical investigation in two outpatient clinics. J Adv Nurs. 1997;44(1):85-92.

3. Bandeira M, Pitta AMF, Mercier C. Escalas da OMS de avaliação da satisfação e da sobrecarga em serviços de saúde mental: qualidades psicométricas da versão brasileira. J Bras Psiquitr.1999;48(6):233-44.

4. Bandeira M, Pitta AMF, Mercier C. Escala de avaliação da satisfação de familiares com os serviços de saúde mental: SATIS-BR. J Bras Psiquiatr. 2000;49(8):293-300.

5. Bandeira M, Mercier C, Perreault M, Libério MMA, Pitta AMF. Escala de avaliação da satisfação de familiares com os serviços de saúde mental: SATIS-BR. J Bras Psiquiatr. 2002;51(3):153-66.

6. Costa-Rosa A. O modo psicossocial: um paradigma das práticas substitutivas ao modo asilar. In: Amarante P, organizador. Ensaios: subjetividade, saúde mental, sociedade. Rio e Janeiro: Editora Fiocruz; 2000.

7. Donabedian A. The definition of quality and approaches to its assessment. Explorations in quality assessment and monitoring. Ann Arbor: Health Administration Press; 1980. v.1.

8. Donabedian A. La calidad de la atención medica: definición y métodos de evaluación. Ediciones científicas. México: Prensa Medica Mexicana; 1984.

9. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743-8. DOI: 10.1001/jama.260.12.1743

10. Guba E, Lincoln Y. Effective Evaluation. Improving the usefulness of evaluation results throug responsive naturalistic approaches. San Francisco: Jossey Bass Publishers;1985.

11. Guba E, Lincoln Y. Effective evaluation. San Francisco: Jossey Bass Publishers;1998.

12. Guba E, Lincoln Y. Fourth generation evaluation. Newbury Park: Sage Publications; 1989.

13. Jaegger RC, Guitton AP, Lyrio JM, Santos MM, Freitas RC, Gonçalves SR, et al. A experiência de morar fora: avaliação da satisfação de usuários em um serviço de saúde mental. Cad Saude Coletiva. 2004;12(1):27-39.

14. Ludermir AB, Melo Filho DA. Condições de vida e estrutura ocupacional associadas a transtornos mentais comuns. Rev Saude Publica. 2002;36(2):213-21. DOI: 10.1590/S0034-89102002000200014

15. Pitta AMF, Silva Filho JF, Souza GW, Lancman S, Kinoshita RT, Cavalcanti MT, et al. Determinantes da qualidade de serviços de saúde mental em municípios brasileiros: estudo da satisfação com os resultados das atividades desenvolvidas por pacientes, familiares e trabalhadores dos serviços. J. Bras Psiquiatr. 1995;44(9):441-52.

16. Silva Filho JF, Cavalcanti MT, Cadilhe GM, Vilaça AT, Avancini E, Machado HO, et al. Avaliação da qualidade de serviços de saúde mental no município de Niterói - RJ. A satisfação dos usuários como critério avaliador. J Bras Psiquiatr.1996;45(7):393-402.

17. Wohlfarth T. Socioeconomic inequality and psychopathology: are socioeconomic status and social class interchangeable. Soc Sci Med. 1997;45(3):399-410. DOI: 10.1016/S0277-9536(96)00355-3

18. Yin RK. Estudo de caso: planejamento e métodos. Trad. de D Grassi. 3. ed. Porto Alegre: Bookman; 2005.

Correspondence:
Av. Duque de Caxias 250 Fragata
96030-002 Pelotas, RS, Brasil
E-mail: kantorski@uol.com.br

Revised: 05/10/2009
Approved: 06/04/2009
Research funded by the Ministério da Saúde (Brazilian Ministry of Health) and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq - Proc. nº 554537/2005-2).

This article underwent the peer review process adopted for any other manuscript submitted to this journal, with anonymity guaranteed for both authors and reviewers. Editors and reviewers declare that there are no conflicts of interest that could affect their judgment with respect to this article.
The authors declare that there are no conflicts of interest.
a Organización Mundial de la Salud. Atlas: Salud mental en el mundo 2001. Ginebra; 2001.
b Ministério da Saúde. Executive Division. Department of Health Care. Legislação em saúde mental: 1990-2004 . 5. ed. Brasília; 2004.
c Wetzel C. Avaliação de serviços de saúde mental: a construção de um processo participativo. [doctorate thesis]. Ribeirão Preto: Escola de Enfermagem de Ribeirão Preto da USP; 2005.
d Ministério da Saúde. Department of Health Care. Saúde Mental no SUS: acesso ao tratamento e mudança do modelo de atenção. Relatório de Gestão 2003-2006. Brasília (DF); 2007.

Faculdade de Saúde Pública da Universidade de São Paulo São Paulo - SP - Brazil
E-mail: revsp@org.usp.br