Brazilian Scale for Evaluation of Mental Health Care Needs: Additional Evidence

Joana Moscoso Teixeira de Mendonça Flavio Rebustini Ana Alice Freire de Sousa Ilana Eshriqui Daiana Bonfim Leticia Yamawaka de Almeida About the authors

ABSTRACT

OBJECTIVE

To investigate validity evidence of the Brazilian Scale for Evaluation of Mental Health Care Needs (CuidaSM).

METHODS

This is a psychometric study, which seeks additional evidence of internal structure. Data collection was carried out in 11 Primary Health Care (PHC) services , which implement the Health Care Planning (HCP) methodology, distributed across the five Brazilian regions. The preliminary version of CuidaSM, containing a block self-referred by the user and another block evaluated by PHC professionals, was applied to users aged 18 or over who attended the PHC services for consultation with a higher education professional. The techniques of confirmatory factor analysis and network analysis were used to investigate validity evidence. For the primary data of the confirmatory factor analysis, the factorial loads and the item’s predictive power (R2) were used. Six model adjustment indices were adopted and reliability was measured by three indicators using Bayesian estimation.

RESULTS

A total of 879 users participated in the study. By confirmatory factor analysis, factorial loads ranged from 0.43 to 0.99 and R2 from 0.19 to 0.98. Both the primary indicators and the model adequacy indices were established at satisfactory and consistent levels. The network analysis showed that the items were appropriately associated with their peers, respecting the established dimensions, which again indicates the sustainability and stability of the proposed model.

CONCLUSIONS

The study findings confirm a consistent and reliable model of the instrument, through a combination of techniques. Considering the importance of using solid instruments in clinical practice, CuidaSM is a promising tool for population-based management and network care organization, aligned with HCP proposals.

Mental Health; Primary Health Care; Population Health Management

INTRODUCTION

Health Care Planning (HCP) is a methodology that seeks to provide health professionals with the development of skills aimed at planning and organizing health care, considering the needs of users in their territory. In general terms, the proposal aims to integrate the Health Care Network, using strategies to improve the organization of macro and micro processes of primary health care (PHC) and specialized outpatient care 11. Conselho Nacional de Secretários de Saúde. Planificação da Atenção à Saúde: um instrumento de gestão e organização da Atenção Primária à Saúde e da Atenção Ambulatorial Especializada nas Redes de Atenção à Saúde. Brasília, DF: Conselho Nacional de Secretários de Saúde; 2018 [cited 2023 Jan 10]. Available from: https://www.conass.org.br/biblioteca/caderno-conass-documenta-n-31/
https://www.conass.org.br/biblioteca/cad...
.

In the context of PHC, the metaphor of building a house is used to illustrate the theoretical operational model of its social construction, in which the basic macro and micro processes constitute the foundation and provide support for meeting the demands of the target population11. Conselho Nacional de Secretários de Saúde. Planificação da Atenção à Saúde: um instrumento de gestão e organização da Atenção Primária à Saúde e da Atenção Ambulatorial Especializada nas Redes de Atenção à Saúde. Brasília, DF: Conselho Nacional de Secretários de Saúde; 2018 [cited 2023 Jan 10]. Available from: https://www.conass.org.br/biblioteca/caderno-conass-documenta-n-31/
https://www.conass.org.br/biblioteca/cad...
. From this perspective, the risk stratification of chronic conditions, based on the Chronic Conditions Care Model , emerges as a fundamental element for planning and implementing actions in services, in accordance with population-based management11. Conselho Nacional de Secretários de Saúde. Planificação da Atenção à Saúde: um instrumento de gestão e organização da Atenção Primária à Saúde e da Atenção Ambulatorial Especializada nas Redes de Atenção à Saúde. Brasília, DF: Conselho Nacional de Secretários de Saúde; 2018 [cited 2023 Jan 10]. Available from: https://www.conass.org.br/biblioteca/caderno-conass-documenta-n-31/
https://www.conass.org.br/biblioteca/cad...
,22. Mendes EV. As redes de atenção à saúde. Brasília, DF: Organização Pan-Americana da Saúde; 2011 [cited 2023 Jan 10]. Available from: https://www.paho.org/bra/dmdocuments/Redes_Atencao_Saude_Eugenio_2ed.PDF
https://www.paho.org/bra/dmdocuments/Red...
.

Notably, the conduction of this process, by enabling the identification of users’ needs and differentiating them by groups/strata, directs the professional’s decision-making in relation to the type of care, resources and technologies specific to clinical management22. Mendes EV. As redes de atenção à saúde. Brasília, DF: Organização Pan-Americana da Saúde; 2011 [cited 2023 Jan 10]. Available from: https://www.paho.org/bra/dmdocuments/Redes_Atencao_Saude_Eugenio_2ed.PDF
https://www.paho.org/bra/dmdocuments/Red...
and organization of assistance in different lines of care.

In the context of mental health (MH) care, in the face of a global scenario of scarce resources and the therapeutic gap for mental disorders33. Alonso J, Liu Z, Evans-Lacko S, Sadikova E, Sampson N, Chatterji S, et al. Treatment gap for anxiety disorders is global: results of the World Mental Health surveys in 21 countries. Depress Anxiety. 2018 Ma;35(3):195-208. https://doi.org/10.1002/da.22711.
https://doi.org/10.1002/da.22711...
,44. Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry. 2017 Feb;210(2):119-24. https://doi.org/10.1192/bjp.bp.116.188078
https://doi.org/10.1192/bjp.bp.116.18807...
, PHC has emerged as a strategic component for care55. Barros S, Nóbrega MD, Santos JC, Fonseca LM, Floriano LS. Mental health in primary health care: health-disease according to health professionals. Rev Bras Enferm. 2019 Oct;72(6):1609-17. https://doi.org/10.1590/0034-7167-2018-0743
https://doi.org/10.1590/0034-7167-2018-0...
,66. Ayano G. Significance of mental health legislation for successful primary care for mental health and community mental health servisse: a review. Afr J Prim Health Care Fam Med. 2018 Mar;10(1):e1-4. https://doi.org/10.4102/phcfm.v10i1.1429
https://doi.org/10.4102/phcfm.v10i1.1429...
. However, the use of tools to quantify and organize this demand for services has been considered a challenge among professionals77. Gama CA, Lourenço RF, Coelho VA, Campos CG, Guimarães DA. Os profissionais da Atenção Primária à Saúde diante das demandas de Saúde Mental: perspectivas e desafios. Interface (Botucatu). 2021;25. https://doi.org/10.1590/interface.200438
https://doi.org/10.1590/interface.200438...
.

The MH care lines developed by the Ministry of Health88. Ministério da Saúde (BR). Linhas de cuidado. Brasilia, DF: Secretaria de Atenção Primária à Saúde; 2023 [cited 2023 Jan 10]. Available from: https://linhasdecuidado.saude.gov.br/portal/todas-linhas
https://linhasdecuidado.saude.gov.br/por...
recommend the use of instruments that, in general, aim to facilitate the detection of specific conditions, such as depression, anxiety, and problems related to alcohol use. However, there is a lack of instruments that support professionals in stepped care model sin the territory.

In this sense, the Brazilian Scale for Evaluation of Mental Health Care Needs (known as CuidaSM)99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
was developed, aligned with the concept of population-based management, to facilitate the recognition of specific needs for mental health care, configuring itself as a potential tool to collaborate with PHC teams in operationalizing the basic macro-process of stratification and, consequently, in the provision and organization of care.

A previous study demonstrated evidence of validity of CuidaSM, via exploratory factor analysis, thus, this instrument is composed of 31 dichotomous items, covering five dimensions in a self-referred block (Social Relationship, Functionality, Autonomy, Impulsiveness and Aggressiveness, Spirituality) and three dimensions in a block applied by a higher-level PHC professional (Violence, Self-aggression and Suicidal Behavior, Care Plan)99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
. From its application, it is possible to classify people who use PHC into four strata of mental health care needs (MHCN) (low, moderate, high, and very high)99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
. In this context, CuidaSM supports the process of stepped care based on objective data on MHCN. Furthermore, it has the potential to contribute to effective communication between different points of care in the network, with decision-making regarding the right therapy, at the right time, for the right user, and, therefore, with the more rational use of technical and human resources, assisting in care programming.

Considering that to date there are no other validated instruments to measure MHCN in the Brazilian population, as well as the relevance of using, in clinical practice, solid instruments in terms of psychometric properties and in line with current recommendations regarding multiple tests to adjust an instrument1010. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Joint Committee on Standards for Educational and Psychological Testing (U.S.). The standards for educational and psychological testing. Washington, DC: American Psychological Association; 2014., this study aimed to investigate evidence of the validity of CuidaSM.

METHODS

Design

This is a study of additional evidence of the internal structure of CuidaSM, carried out through confirmatory factor analysis (CFA) and network analysis, according to the current recommendation of the American Educational Research Association, American Psychological Association and the National Council on Measurement in Education1010. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Joint Committee on Standards for Educational and Psychological Testing (U.S.). The standards for educational and psychological testing. Washington, DC: American Psychological Association; 2014.. The study was approved by the Research Ethics Committee of Hospital Israelita Albert Einstein (CAAE: 12395919.0.0000.0071).

Data collection

The preliminary version of CuidaSM, consisting of 43 items (23 in a self-referred user block and 20 in a clinical assessment block carried out by a PHC professional), as described in a previous study99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
, was applied to users registered in 11 PHC services, totaling 27 family health teams. Collection took place between November 2021 and August 2022, and was conducted by a team of previously trained researchers.

The PHC services that made up the research field were defined based on their location, in municipalities that use the HCP methodology and the inclusion of at least one service in each geographic region of the country, including: one PHC service in the North region (Roraima); one PHC service in the Northeast region (Pernambuco); two PHC service in the Midwest region (Mato Grosso); five PHC service in the Southeast region (three in São Paulo and two in Minas Gerais); and two PHC service in the South region (Paraná).

After contacting the service management to agree on the activity schedule, the responsible researcher held virtual meetings with the units’ teams to present the study proposal. Furthermore, in these meetings, doubts were clarified and data collection was prepared, which included health professionals for the clinical block evaluation.

The operationalization of data collection in the PHC services was carried out in two moments. Initially, users were approached by researchers in the PHC services waiting rooms and invited to participate in the research, by reading and expressing acceptance of the Informed Consent Form. At this time, they answered a questionnaire composed of sociodemographic items, clinical profile, and the self-referred block of the CuidaSM scale.

Subsequently, during the user’s consultation with the PHC health professional, the clinical assessment of the CuidaSM scale was carried out. For this moment, a printed version of the instrument was used, which was collected by the researchers at the end of the day. Data recording was carried out in a tablet, through Research Electronic Data Capture (REDCap©).

The eligibility criteria for users were: being 18 years old or over and being at the PHC service for care in individual consultation with a doctor, a nurse or health professionals from the multidisciplinary team. Those who attended the service for emergency or dental care or procedures (such as vaccination, dressing and medication) or who did not complete the “health professional evaluation” block were excluded.

Data analysis

Confirmatory Factor Analysis (CFA)

For the primary data of the CFA, the factorial loads and the predictive power of the item (R2) were used22. Mendes EV. As redes de atenção à saúde. Brasília, DF: Organização Pan-Americana da Saúde; 2011 [cited 2023 Jan 10]. Available from: https://www.paho.org/bra/dmdocuments/Redes_Atencao_Saude_Eugenio_2ed.PDF
https://www.paho.org/bra/dmdocuments/Red...
. The model adjustment indices adopted were: χ2/df; Non- Normed Fit Index (NNFI ≥ 0.95); Comparative Fit Index (CFI ≥ 0.95); Goodness Fit Index (GFI ≥ 0.95); Root Mean Square Error of Approximation (RMSEA < 0.08) and Root Mean Square of Residuals (RMSR ≥ 0.8). The model tested in the EFA was the factorial solution found in the initial study99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
of the exploratory factor analysis.

Reliability was measured by three indicators: Cronbach’s alpha1414. Crobach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334. https://doi.org/10.1007/BF02310555
https://doi.org/10.1007/BF02310555...
; Greatest Lower Bound (GLB)1515. Woodhouse B, Jackson PH. Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: a search procedure to locate the greatest lower bound. Psychometrika. 1977;42(4):579-91. https://doi.org/10.1007/BF02295980
https://doi.org/10.1007/BF02295980...
; and Omega1616. McDonald RP. Test theory: a unified treatment. New York: Psychology Press; 2013., all of the three using Bayesian estimation1717. Pfadt JM, Bergh DV, Sijtsma K, Wagenmakers EJ. A tutorial on Bayesian single-test reliability analysis with JASP. Behav Res Methods. 2022;55(3):1069-78. https://doi.org/10.3758/s13428-021-01778-0
https://doi.org/10.3758/s13428-021-01778...
.

Network Analysis

In the last decade there has been an extension of the application of network analysis to different scenarios and applications, such as: symptom assessment1818. Mullarkey MC, Marchetti I, Beevers CG. Using network analysis to identify central symptoms of adolescent depression. J Clin Child Adolesc Psychol. 2019;48(4):656-68. https://doi.org/10.1080/15374416.2018.1437735
https://doi.org/10.1080/15374416.2018.14...
, psychological networks and psychopathologies1919. Epskamp S, Borsboom D, Fried EI. Estimating psychological networks and their accuracy: a tutorial paper. Behav Res. 2017;50(1):195-212. https://doi.org/10.3758/s13428-017-0862-1
https://doi.org/10.3758/s13428-017-0862-...
,2020. Contreras A, Nieto I, Valiente C, Espinosa R, Vazquez C. The study of psychopathology from the network analysis perspective: a systematic review. Psychother Psychosom. 2019;88(2):71-83. https://doi.org/10.1159/000497425
https://doi.org/10.1159/000497425...
, post-traumatic stress2121. Liang Y, Li F, Zhou Y, Liu Z. Evolution of the network pattern of posttraumatic stress symptoms among children and adolescents exposed to a disaster. J Anxiety Disord. 2021 Jan;77:102330. https://doi.org/10.1016/j.janxdis.2020.102330
https://doi.org/10.1016/j.janxdis.2020.1...
,2222. Yuan GF, Shi W, Elhai JD, Montag C, Chang K, Jackson T, et al. Gaming to cope: applying network analysis to understand the relationship between post-traumatic stress symptoms and internet gaming disorder symptoms among disaster-exposed Chinese young adults. Addict Behav. 2022;124:107096. https://doi.org/10.1016/j.addbeh.2021.107096
https://doi.org/10.1016/j.addbeh.2021.10...
, schizophrenia2323. Fonseca-Pedrero E, Ortuño J, Debbané M, Chan RC, Cicero D, Zhang LC, et al. The network structure of schizotypal personality traits. Schizophr Bull. 2018 Oct;44(2 suppl_2):S468-79. https://doi.org/10.1093/schbul/sby044
https://doi.org/10.1093/schbul/sby044...
, anxiety2424. Haws JK, Brockdorf AN, Gratz KL, Messman TL, Tull MT, DiLillo D. Examining the associations between PTSD symptoms and aspects of emotion dysregulation through network analysis. J Anxiety Disord. 2022 Mar;86:102536. https://doi.org/10.1016/j.janxdis.2022.102536
https://doi.org/10.1016/j.janxdis.2022.1...
and instrument development measuring2525. Borsboom D. Possible futures for network psychometrics. Psychometrika. 2022 Mar;87(1):253-65. https://doi.org/10.1007/s11336-022-09851-z
https://doi.org/10.1007/s11336-022-09851...
. Nevertheless, in Brazil, studies using network analysis are rare3030. Holst B, Lisboa CSM, Machado WL. Children and adolescents psychotherapy: Evidences of validity of the Brazilian Portuguese Youth Outcome Questionnaire 2.01. Psicol Clin. 2021;33(2):211-36. https://doi.org/10.33208/PC1980-5438v0033n02A01
https://doi.org/10.33208/PC1980-5438v003...
,3131. Wechsler SM, Benson N, Machado WL, Bachert CMD, Gums EF. Adult temperament styles: a network analysis of their relationships with the Big Five Personality Model. Eur J Psychol Educ. 2018;11(1):61-75. [cited 2023 Jan 10] Available from: https://psycnet.apa.org/record/2019-04358-005
https://psycnet.apa.org/record/2019-0435...
.

It is important to understand how it can be useful in the search for validity evidence. According to Newman3232. Newman ME. Networks: an introduction. Oxford: Oxford University Press; 2010.(2010), the analysis is composed of two stages: in the first, it estimates a statistical data model, from which some parameters can be represented as a weighted network between the observed variables; and in the second stage, the structure of the weighted network is analyzed, using measures taken from graph theory.

This study used the High-dimensional Undirected Graph Estimation (Huge)3333. Zhao T, Liu H, Roeder K, Lafferty J, Wasserman L. The huge package for high-dimensional undirected graph estimation in R. J Mach Learn Res. 2012 Apr;13:1059-62.and Extended Bayesian Information Criterion (Ebic) as criterion. Huge works with two estimation procedures: 1) the neighborhood search algorithm3434. Meinshausen N, Bühlmann P. High-dimensional graphs and variable selection with the Lasso. Ann Stat. 2006;34(3):1436-62. https://doi.org/10.1214/009053606000000281
https://doi.org/10.1214/0090536060000002...
and 2) the Lasso graph algorithm3535. Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics. 2008 Jul;9(3):432-41. https://doi.org/10.1093/biostatistics/kxm045
https://doi.org/10.1093/biostatistics/kx...
. The graph nodes were positioned using the Fruchterman and Reingold algorithm3636. Fruchterman TM, Reingold EM. Graph drawing by force-directed placement. Softw Pract Exper. 1991;21(11):1129-64. https://doi.org/10.1002/spe.4380211102
https://doi.org/10.1002/spe.4380211102...
, which is based on the strength and connectivity between the nodes. Each node represents an instrument item.

To evaluate the generated model, four indicators were adopted: Betweenness, which evaluates the efficiency with which a node connects to others; closeness, which evaluates the ease with which information reaches other nodes from a specific node; strength (or degree), which represents how much a node is connected to the rest of the network3737. Valente TW. Network interventions. Science. 2012 Jul;337(6090):49-53. https://doi.org/10.1126/science.1217330
https://doi.org/10.1126/science.1217330...
; and, finally, expected influence, which aims to assess the nature and strength of the cumulative influence of a node within the network2323. Fonseca-Pedrero E, Ortuño J, Debbané M, Chan RC, Cicero D, Zhang LC, et al. The network structure of schizotypal personality traits. Schizophr Bull. 2018 Oct;44(2 suppl_2):S468-79. https://doi.org/10.1093/schbul/sby044
https://doi.org/10.1093/schbul/sby044...
and, therefore, the role it can be expected to play in its activation, persistence and remission3838. Berger M, Fernando S, Churchill A, Cornish P, Henderson J, Shah J, et al. Scoping review of stepped care interventions for mental health and substance use service delivery to youth and young adults. Early Interv Psychiatry. 2022 Apr;16(4):327-41. https://doi.org/10.1111/eip.13180
https://doi.org/10.1111/eip.13180...
.

For both techniques, a bootstrap of 5,000 was applied. Analyses were performed using JASP 16.04.

RESULTS

1,219 people agreed to participate in the study, of which 879 had both CuidaSM blocks completed and were included in this study. Among these, the mean age was 45 years (SD = 16.7). The majority were female (74.3%), married or in a common-law marriage (46.5%), not beneficiaries of social programs (76.0%) and with nine or more years of schooling (54.2%). The participants lived mainly in the Southeast region (58.4%), followed by the South (11.1%), Midwest (11.0%), North (5.4%), and Northeast (4%).

Using exploratory factor analysis, factorial loads ranged from 0.43 to 0.99, with standard errors ranging from 0.004 to 0.018 and the item’s predictive capacity (R2) from 0.19 to 0.98 (Figure 1). All at satisfactory levels. In addition to the primary indicators, the model fit indices were established at: X2/df (245) = 1.64, p = 0.001; NNFI = 0.97, CFI = 0.98, GFI = 0.98, RMSEA = 0.06, and RMSR = 0.03.

Figure 1
Pathways diagram.

The reliability indices with Bayesian estimation were: Cronbach’s alpha = 0.81 (95%CI 0.79–0.82), McDonald’s omega = 0.78 (95%CI 0.76–0.81) and GLB = 0.93 (95%CI 0.92–0.93). All indicators at appropriate levels. In this way, both the primary indicators and the model adequacy indices were established at satisfactory and consistent levels.

The items were positioned within the network close to their peers in the domains. Figure 2 shows that the items were appropriately associated with their peers, respecting the established domains, which indicates the sustainability and stability of the proposed model.

Figure 2
Network analysis.

For the standardized centrality indicators, the items (Figure 3) that presented the most relevant results were: for betweenness, items 1 “Do you have friends?”, 3 “Are you able to maintain friendships?”, and 25 “Does the user think about self-aggression?”. For closeness, items 1 “Do you have friends?”, 3 “Are you able to maintain friendships?”, and 9 “Do you perform your daily hygiene by yourself?”. For the strength/degree index, items 1 “Do you have friends?”, 3 “Are you able to maintain friendships?”, and 8 “Are you able to bathe by yourself?”, 18 “Has the user witnessed violence?”, 23 “Does the user plan suicide?”, and 30 “Is the user unaware of their disease?”, indicating that these items are those that have the strongest connection with the network. Items 1, 3, 23, 18, and 30 are also those with the greatest expected influence and, therefore, those with the greatest cumulative influence on the configuration of the model.

Figure 3
Centrality indices of standardized items (z).

In short, both the CFA and network analysis results confirm a consistent and reliable model of the instrument, in accordance with the original model99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
obtained in the exploratory factor analysis.

DISCUSSION

This study sought to present additional evidence of the CuidaSM’s internal structure, through CFA and network analysis. The results demonstrated that the primary indicators and model adequacy indices were established at satisfactory and consistent levels. Furthermore, the items were appropriately associated with their peers, respecting the established domains and confirming the stability of the proposed model.

In this sense, these findings endorse the contribution regarding the information obtained from the combination of techniques. The literature has described the need and contribution of tested models integrating multiple techniques, as this combination seeks to improve the precision and quality of instruments1111. Hu Z, Lin L, Wang Y, Li J. The integration of classical testing theory and item response theory. Psychology (Irvine). 2021;12(9):1397-409. https://doi.org/10.4236/psych.2021.129088
https://doi.org/10.4236/psych.2021.12908...
. As Borsoom2525. Borsboom D. Possible futures for network psychometrics. Psychometrika. 2022 Mar;87(1):253-65. https://doi.org/10.1007/s11336-022-09851-z
https://doi.org/10.1007/s11336-022-09851...
points out, the use of network analysis or network psychometrics is part of a broader movement in psychology, which revolves around the analysis of human beings as complex systems, mainly because it allows the connection between components traditionally studied in isolation. Configuring itself as an emerging technique in psychometric studies.

This study, therefore, demonstrates additional validity evidence of CuidaSM, which presents eight dimensions distributed in 31 items, considering the user’s perception within the scope of their social relationships, functionality, autonomy, impulsiveness, aggressiveness and spirituality, in addition to counting on professional assessment regarding aspects related to violence, self-aggression, suicidal behavior and the care plan.

It is important to highlight that the development of the items was guided by a convergent understanding of the proposals of the psychosocial care model55. Barros S, Nóbrega MD, Santos JC, Fonseca LM, Floriano LS. Mental health in primary health care: health-disease according to health professionals. Rev Bras Enferm. 2019 Oct;72(6):1609-17. https://doi.org/10.1590/0034-7167-2018-0743
https://doi.org/10.1590/0034-7167-2018-0...
, covering aspects that go beyond addressing signs and symptoms. In addition, it sought to incorporate specific questions about the care plan which, in more complex cases, can alert to the involvement of actions shared with other members of the multi-professional team.

It is also noted that, in the context of HCP, the CuidaSM scale is characterized as a promising instrument, as it provides support for the articulation between the points of the Psychosocial Care Network, as it can contribute to the management function—offering objective parameters that enable the preparation of health services programming for the different strata of care needs in MH—and with an objective communicational function, by enabling PHC services to use the same stratification parameter22. Mendes EV. As redes de atenção à saúde. Brasília, DF: Organização Pan-Americana da Saúde; 2011 [cited 2023 Jan 10]. Available from: https://www.paho.org/bra/dmdocuments/Redes_Atencao_Saude_Eugenio_2ed.PDF
https://www.paho.org/bra/dmdocuments/Red...
.

In this way, according to the score established in the CuidaSM scale99. Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
https://doi.org/10.21203/rs.3.rs-2450855...
, users can be classified as having low, medium, high or very high MHCN. It is expected that, based on these strata, services can establish parameters to organize care among those who can benefit from supported self-care technologies focused on PHC or those who require significant professional attention, which may even involve co-participation of specialized care.

It is noteworthy that, at an international level, discussions have been carried out about the organization and planning of MH services in environments with limited resources, especially through the stepped care model, in which individuals access treatments in a sequential, self-correcting manner3838. Berger M, Fernando S, Churchill A, Cornish P, Henderson J, Shah J, et al. Scoping review of stepped care interventions for mental health and substance use service delivery to youth and young adults. Early Interv Psychiatry. 2022 Apr;16(4):327-41. https://doi.org/10.1111/eip.13180
https://doi.org/10.1111/eip.13180...
. Furthermore, recently, the adoption of a stratified approach has been incorporated into this debate3939. Delgadillo J, Ali S, Fleck K, Agnew C, Southgate A, Parkhouse L, et al. Stratified care vs stepped care for depression: a cluster randomized clinical trial. JAMA Psychiatry. 2022 Feb;79(2):101-8. https://doi.org/10.1001/jamapsychiatry.2021.3539
https://doi.org/10.1001/jamapsychiatry.2...
, strengthening the reflections triggered by the study.

The fact of adopting an intentional sample could represent a limitation for generalizing the results of this study. However, it is emphasized that numerous analyses and data processing were carried out to ensure that the instrument provided adequate, consistent evidence that legitimizes its application nationwide. Therefore, this study is distinguished by the fact that the analyses indicate that the items are valid for measuring the MHCN in federative units in Brazil’s five geographic regions.

Finally, it is recommended that future research investigate aspects of the implementation of the CuidaSM scale in the daily routine of PHC services and its applicability for decision-making in the planning of MH care, according to the needs identified in users.

CONCLUSION

The study’s findings confirm a consistent and reliable model for CuidaSM, using a combination of techniques. Thus, considering the importance of using solid instruments in clinical practice, CuidaSM is as a promising tool for population-based management and network care organization, in line with the proposals of the HCP.

REFERENCES

  • 1
    Conselho Nacional de Secretários de Saúde. Planificação da Atenção à Saúde: um instrumento de gestão e organização da Atenção Primária à Saúde e da Atenção Ambulatorial Especializada nas Redes de Atenção à Saúde. Brasília, DF: Conselho Nacional de Secretários de Saúde; 2018 [cited 2023 Jan 10]. Available from: https://www.conass.org.br/biblioteca/caderno-conass-documenta-n-31/
    » https://www.conass.org.br/biblioteca/caderno-conass-documenta-n-31/
  • 2
    Mendes EV. As redes de atenção à saúde. Brasília, DF: Organização Pan-Americana da Saúde; 2011 [cited 2023 Jan 10]. Available from: https://www.paho.org/bra/dmdocuments/Redes_Atencao_Saude_Eugenio_2ed.PDF
    » https://www.paho.org/bra/dmdocuments/Redes_Atencao_Saude_Eugenio_2ed.PDF
  • 3
    Alonso J, Liu Z, Evans-Lacko S, Sadikova E, Sampson N, Chatterji S, et al. Treatment gap for anxiety disorders is global: results of the World Mental Health surveys in 21 countries. Depress Anxiety. 2018 Ma;35(3):195-208. https://doi.org/10.1002/da.22711
    » https://doi.org/10.1002/da.22711
  • 4
    Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry. 2017 Feb;210(2):119-24. https://doi.org/10.1192/bjp.bp.116.188078
    » https://doi.org/10.1192/bjp.bp.116.188078
  • 5
    Barros S, Nóbrega MD, Santos JC, Fonseca LM, Floriano LS. Mental health in primary health care: health-disease according to health professionals. Rev Bras Enferm. 2019 Oct;72(6):1609-17. https://doi.org/10.1590/0034-7167-2018-0743
    » https://doi.org/10.1590/0034-7167-2018-0743
  • 6
    Ayano G. Significance of mental health legislation for successful primary care for mental health and community mental health servisse: a review. Afr J Prim Health Care Fam Med. 2018 Mar;10(1):e1-4. https://doi.org/10.4102/phcfm.v10i1.1429
    » https://doi.org/10.4102/phcfm.v10i1.1429
  • 7
    Gama CA, Lourenço RF, Coelho VA, Campos CG, Guimarães DA. Os profissionais da Atenção Primária à Saúde diante das demandas de Saúde Mental: perspectivas e desafios. Interface (Botucatu). 2021;25. https://doi.org/10.1590/interface.200438
    » https://doi.org/10.1590/interface.200438
  • 8
    Ministério da Saúde (BR). Linhas de cuidado. Brasilia, DF: Secretaria de Atenção Primária à Saúde; 2023 [cited 2023 Jan 10]. Available from: https://linhasdecuidado.saude.gov.br/portal/todas-linhas
    » https://linhasdecuidado.saude.gov.br/portal/todas-linhas
  • 9
    Mendonça JM, Sousa AA, Eshriqui I, Rebustini F, Bonfim D, Zorzi DS, et al. Brazilian Scale for Evaluation of Mental Health Care Needs: development and evidence of validity. Research Square. [citado 11Jan 2023]. Available from: https://doi.org/10.21203/rs.3.rs-2450855/v1
    » https://doi.org/10.21203/rs.3.rs-2450855/v1
  • 10
    American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Joint Committee on Standards for Educational and Psychological Testing (U.S.). The standards for educational and psychological testing. Washington, DC: American Psychological Association; 2014.
  • 11
    Hu Z, Lin L, Wang Y, Li J. The integration of classical testing theory and item response theory. Psychology (Irvine). 2021;12(9):1397-409. https://doi.org/10.4236/psych.2021.129088
    » https://doi.org/10.4236/psych.2021.129088
  • 12
    Schlechter P, Wilkinson PO, Knausenberger J, Wanninger K, Kamp S, Morina N, et al. Depressive and anxiety symptoms in refugees: insights from classical test theory, item response theory and network analysis. Clin Psychol Psychother. 2021 Jan;28(1):169-81. https://doi.org/10.1002/cpp.2499
    » https://doi.org/10.1002/cpp.2499
  • 13
    Raykov T, Marcoulides GA. On the relationship between classical test theory and item response theory. Educ Psychol Meas. 2015 Apr;76(2):325-38. https://doi.org/10.1177/0013164415576958
    » https://doi.org/10.1177/0013164415576958
  • 14
    Crobach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334. https://doi.org/10.1007/BF02310555
    » https://doi.org/10.1007/BF02310555
  • 15
    Woodhouse B, Jackson PH. Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: a search procedure to locate the greatest lower bound. Psychometrika. 1977;42(4):579-91. https://doi.org/10.1007/BF02295980
    » https://doi.org/10.1007/BF02295980
  • 16
    McDonald RP. Test theory: a unified treatment. New York: Psychology Press; 2013.
  • 17
    Pfadt JM, Bergh DV, Sijtsma K, Wagenmakers EJ. A tutorial on Bayesian single-test reliability analysis with JASP. Behav Res Methods. 2022;55(3):1069-78. https://doi.org/10.3758/s13428-021-01778-0
    » https://doi.org/10.3758/s13428-021-01778-0
  • 18
    Mullarkey MC, Marchetti I, Beevers CG. Using network analysis to identify central symptoms of adolescent depression. J Clin Child Adolesc Psychol. 2019;48(4):656-68. https://doi.org/10.1080/15374416.2018.1437735
    » https://doi.org/10.1080/15374416.2018.1437735
  • 19
    Epskamp S, Borsboom D, Fried EI. Estimating psychological networks and their accuracy: a tutorial paper. Behav Res. 2017;50(1):195-212. https://doi.org/10.3758/s13428-017-0862-1
    » https://doi.org/10.3758/s13428-017-0862-1
  • 20
    Contreras A, Nieto I, Valiente C, Espinosa R, Vazquez C. The study of psychopathology from the network analysis perspective: a systematic review. Psychother Psychosom. 2019;88(2):71-83. https://doi.org/10.1159/000497425
    » https://doi.org/10.1159/000497425
  • 21
    Liang Y, Li F, Zhou Y, Liu Z. Evolution of the network pattern of posttraumatic stress symptoms among children and adolescents exposed to a disaster. J Anxiety Disord. 2021 Jan;77:102330. https://doi.org/10.1016/j.janxdis.2020.102330
    » https://doi.org/10.1016/j.janxdis.2020.102330
  • 22
    Yuan GF, Shi W, Elhai JD, Montag C, Chang K, Jackson T, et al. Gaming to cope: applying network analysis to understand the relationship between post-traumatic stress symptoms and internet gaming disorder symptoms among disaster-exposed Chinese young adults. Addict Behav. 2022;124:107096. https://doi.org/10.1016/j.addbeh.2021.107096
    » https://doi.org/10.1016/j.addbeh.2021.107096
  • 23
    Fonseca-Pedrero E, Ortuño J, Debbané M, Chan RC, Cicero D, Zhang LC, et al. The network structure of schizotypal personality traits. Schizophr Bull. 2018 Oct;44(2 suppl_2):S468-79. https://doi.org/10.1093/schbul/sby044
    » https://doi.org/10.1093/schbul/sby044
  • 24
    Haws JK, Brockdorf AN, Gratz KL, Messman TL, Tull MT, DiLillo D. Examining the associations between PTSD symptoms and aspects of emotion dysregulation through network analysis. J Anxiety Disord. 2022 Mar;86:102536. https://doi.org/10.1016/j.janxdis.2022.102536
    » https://doi.org/10.1016/j.janxdis.2022.102536
  • 25
    Borsboom D. Possible futures for network psychometrics. Psychometrika. 2022 Mar;87(1):253-65. https://doi.org/10.1007/s11336-022-09851-z
    » https://doi.org/10.1007/s11336-022-09851-z
  • 26
    Marsman M, Rhemtulla M. Guest editors' Introduction to the special issue "network psychometrics in action": methodological innovations inspired by empirical problems. Psychometrika. 2022 Mar;87(1):1-11. https://doi.org/10.1007/s11336-022-09861-x
    » https://doi.org/10.1007/s11336-022-09861-x
  • 27
    Marsman M, Borsboom D, Kruis J, Epskamp S, Bork R, Waldorp LJ, et al. An introduction to network psychometrics: relating ising network models to item response theory models. Multivariate Behav Res. 2017 Nov 7;53(1):15-35. https://doi.org/10.1080/00273171.2017.1379379
    » https://doi.org/10.1080/00273171.2017.1379379
  • 28
    Costantini G, Richetin J, Preti E, Casini E, Epskamp S, Perugine M. Stability and variability of personality networks: a tutorial on recent developments in network psychometrics. Pers Individ Dif. 2019 Jan;136:68-78.
  • 29
    Daharnis D, Ardi Z, Alizamar A, Ifdil I, Rangka IB, Suranata K. Adaptation and validation of mathematics anxiety: rasch and network psychometrics analysis. J Phys Conf Ser. 2018 Nov;1114:012113. https://doi.org/10.1088/1742-6596/1114/1/012113
    » https://doi.org/10.1088/1742-6596/1114/1/012113
  • 30
    Holst B, Lisboa CSM, Machado WL. Children and adolescents psychotherapy: Evidences of validity of the Brazilian Portuguese Youth Outcome Questionnaire 2.01. Psicol Clin. 2021;33(2):211-36. https://doi.org/10.33208/PC1980-5438v0033n02A01
    » https://doi.org/10.33208/PC1980-5438v0033n02A01
  • 31
    Wechsler SM, Benson N, Machado WL, Bachert CMD, Gums EF. Adult temperament styles: a network analysis of their relationships with the Big Five Personality Model. Eur J Psychol Educ. 2018;11(1):61-75. [cited 2023 Jan 10] Available from: https://psycnet.apa.org/record/2019-04358-005
    » https://psycnet.apa.org/record/2019-04358-005
  • 32
    Newman ME. Networks: an introduction. Oxford: Oxford University Press; 2010.
  • 33
    Zhao T, Liu H, Roeder K, Lafferty J, Wasserman L. The huge package for high-dimensional undirected graph estimation in R. J Mach Learn Res. 2012 Apr;13:1059-62.
  • 34
    Meinshausen N, Bühlmann P. High-dimensional graphs and variable selection with the Lasso. Ann Stat. 2006;34(3):1436-62. https://doi.org/10.1214/009053606000000281
    » https://doi.org/10.1214/009053606000000281
  • 35
    Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics. 2008 Jul;9(3):432-41. https://doi.org/10.1093/biostatistics/kxm045
    » https://doi.org/10.1093/biostatistics/kxm045
  • 36
    Fruchterman TM, Reingold EM. Graph drawing by force-directed placement. Softw Pract Exper. 1991;21(11):1129-64. https://doi.org/10.1002/spe.4380211102
    » https://doi.org/10.1002/spe.4380211102
  • 37
    Valente TW. Network interventions. Science. 2012 Jul;337(6090):49-53. https://doi.org/10.1126/science.1217330
    » https://doi.org/10.1126/science.1217330
  • 38
    Berger M, Fernando S, Churchill A, Cornish P, Henderson J, Shah J, et al. Scoping review of stepped care interventions for mental health and substance use service delivery to youth and young adults. Early Interv Psychiatry. 2022 Apr;16(4):327-41. https://doi.org/10.1111/eip.13180
    » https://doi.org/10.1111/eip.13180
  • 39
    Delgadillo J, Ali S, Fleck K, Agnew C, Southgate A, Parkhouse L, et al. Stratified care vs stepped care for depression: a cluster randomized clinical trial. JAMA Psychiatry. 2022 Feb;79(2):101-8. https://doi.org/10.1001/jamapsychiatry.2021.3539
    » https://doi.org/10.1001/jamapsychiatry.2021.3539

  • Financing: Program of Support for the Institutional Development of the Unified Health System (Proadi -SUS - technical opinion no. 2/2021 – CGGAP/DESF/SAPS/MS [0019478128] and order SAPS/GAB/SAPS/MS [0019480381]).

Publication Dates

  • Publication in this collection
    15 Apr 2024
  • Date of issue
    2023

History

  • Received
    21 Jan 2023
  • Accepted
    03 Nov 2023
Faculdade de Saúde Pública da Universidade de São Paulo São Paulo - SP - Brazil
E-mail: revsp@org.usp.br