Exposure to psychosocial risk factors in the context of work: a systematic review

Cláudia Fernandes Anabela Pereira About the authors

ABSTRACT

OBJECTIVE

To analyze the scientific literature about the effects of exposure to psychosocial risk factors in work contexts.

METHODS

A systematic review was performed using the terms “psychosocial factors” AND “COPSOQ” in the databases PubMed, Medline, and Scopus. The period analyzed was from January 1, 2004 to June 30, 2012. We have included articles that used the Copenhagen Psychosocial Questionnaire (COPSOQ) as a measuring instrument of the psychosocial factors and the presentation of quantitative or qualitative results. German articles, psychometric studies or studies that did not analyze individual or work factors were excluded.

RESULTS

We included 22 articles in the analysis. Individual factors, such as gender, age, and socioeconomic status, were analyzed along with work-related factors such as labor demands, work organization and content, social relationships and leadership, work-individual interface, workplace values, justice and respect, personality, health and well-being, and offensive behaviors. We analyzed the sample type and the applied experimental designs. Some population groups, such as young people and migrants, are more vulnerable. The deteriorated working psychosocial environment is associated with physical health indicators and weak mental health. This environment is also a risk factor for the development of moderate to severe clinical conditions, predicting absenteeism or intention of leaving the job.

CONCLUSIONS

The literature shows the contribution of exposure to psychosocial risk factors in work environments and their impact on mental health and well-being of workers. It allows the design of practical interventions in the work context to be based on scientific evidences. Investigations in specific populations, such as industry, and studies with more robust designs are lacking.

Psychosocial Impact; Occupational Risks; Working Environment; Professional Autonomy; Job Satisfaction; Mental Health; Occupational Health

INTRODUCTION

The exposure to psychosocial risks in work contexts is one of the biggest challenges to occupational safety and health. This is due to the continuous change and evolution of nature and work organizationaaEuropean Agency for Safety and Health at Work. Research on the changing nature of work. Luxembourg: Office for Official Publications of the European Communities; 2002.,bbEuropean Agency for Safety and Health at Work. Expert forecast on the emerging psychosocial risks related to occupational safety and health. Luxembourg: Office for Official Publications of the European Communities; 2007. (European Risk Observatory Report, nº 50).,ccUS Department of Health and Human Sciences, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. The changing organization of work and the safety and health of working people: knowledge gaps and research directions. Cincinnati: NIOSH; 2002. and to their impact on people, organizations, societies1414. Kristensen TS, Jensen SM, Kreiner S, Mikkelsen S. Socioeconomic status and duration and pattern of sickness absence: a 1-year follow-up study of 2331 hospital employees. BMC Public Health. 2010;10:643. DOI:10.1186/1471-2458-10-643, and policies. The environment and nature of work influence the general health and welfare of the human being1717. Leka S, Jain A. Health impact of psychosocial hazards at work: an overview. Geneva: World Health Organization; 2010.,2222. Marmot M, Allen J, Bell R, Bloomer E, Goldblatt P. WHO European review of social determinants of health and the health divide. Lancet. 2012;380(9846):1011-29. DOI:10.1016/S0140-6736(12)61228-8. The International Labour Association defines psychosocial risk in terms of interaction between work content, work organization and management, other organizational and environmental conditions, and the skills and needs of the workerddInternational Labour Organization. Psychosocial factors at work: recognition and control. Geneva: International Labour Office; 1986. (Occupational and Health Series, 56).. These interactions proved that there are risks to the health of workers and differences in how they experience them. Literature shows a consensus about the nature and identification of psychosocial risks1616. Leka S, Cox T, editors. The European framework for psychosocial risk managemen: PRIMA - EF. Nottingham: Institute of Work, Health and Organization; 2008.,1717. Leka S, Jain A. Health impact of psychosocial hazards at work: an overview. Geneva: World Health Organization; 2010., such as labor demands, work organization and content, social relationships and leadership, work-individual interface, workplace values, justice and respect, health and well-being, and offensive behaviors. New or renewed forms of work, social contexts of interaction, demographic changes, migration flows, economic crisis on a global scale, new technologies, renewed business models, and management of business/logistics networks cause the appearance of new or different risks, which often take the form of emerging or unknown risks. These risks might not be represented in scientific publications66. Cox T. Stress research and stress management: putting theory to work. London: Health and Safety Executive; 1993. HSE Contract Research Report Nº 61/1993. because of the operational nature of the measurement of exposure to psychosocial risks in work context, but which may have a major impact at different levels.

The study on exposure to psychosocial factors and stress are interrelated1717. Leka S, Jain A. Health impact of psychosocial hazards at work: an overview. Geneva: World Health Organization; 2010.. To this end, many admeasurement tools and techniques are used in both constructs. Some of the most popular and used instruments are the Job Content Questionnaire (JCQ)1010. Karasek R, Brisson C, Kawakami N, Houtman I, Bongers P, Amick B. The Job Content Questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. J Occup Health Psychol. 1998;3(4):322-55. DOI:10.1037/1076-8998.3.4.322, the General Nordic Questionnaire for Psychosocial and Social Factors at Work (QPS)1919. Lindstrom K, Elo AL, Skogstad A, Dallner M, Gamberale F, Hottinen V, et al. User’s guide for the QPS Nordic: General Nordic Questionnaire for Psychosocial and Social Factors at Work. Copenhagen: Nordic Council of Ministers; 2000., the Effort-Reward Imbalance Questionnaire (ERI)3333. Siegrist J, Starke D, Chandola T, Godin I, Marmot M, Niedhammer I, et al. The measurement of effort-reward imbalance at work: European comparisons. Soc Sci Med. 2004;58(8):1483-99. DOI:10.1016/S0277-9536(03)00351-4, and the Copenhagen Psychosocial Questionnaire (COPSOQ)1414. Kristensen TS, Jensen SM, Kreiner S, Mikkelsen S. Socioeconomic status and duration and pattern of sickness absence: a 1-year follow-up study of 2331 hospital employees. BMC Public Health. 2010;10:643. DOI:10.1186/1471-2458-10-643. COPSOQ shows differentiating features from most of these instruments, in particular the JCQ and ERI, because it is not based only in a theoretical model explaining the relationship between psychosocial risks, work environment, and health55. Burr H, Albertsen K, Rugulies R, Hannerz H. Do dimensions from the Copenhagen Psychosocial Questionnaire predict vitality and mental health over and above the job strain and effort-reward imbalance models? Scand J Public Health. 2010;38(3 Suppl):59-68. DOI:10.1177/1403494809353436, but in a systemic approach. Because of its practicality, COPSOQ can include more relevant dimensions to the investigation, which might not be covered by previously validated models. It can an also include less studied factors. We chose studies that included this instrument because, besides its multifaceted approacheeSee the instrument validation manuals for different languages/countries, e.g., Silva CF. Copenhagen Psychosocial Questionnaire COPSOQ: Portugal e países africanos de língua oficial portuguesa. Lisboa: Fundação para a Ciência e a Tecnologia; 2000. Instituto Sindical de Trabajo y Salud, Environment). Manual del método CoPsoQ istas 21 para la evaluación y prevención de los riesgos psicosociales para empresas con 25 o más trabajadores y trabajadoras. Versión 1.5. Barcelona: Centro de Referencia de Organización del Trabajo y Salud; 2010., (i) it has at its base the epidemiological method, which defines units of analysis in three sections (improve, maintain, and promote) and allows a practical valuation measure and workstation intervention; (ii) it incorporates reference values for different countries, sectors, and particular occupations, allowing the definition of threshold levels for exposure to psychosocial risks (like the analysis of other types of risks, e.g., chemical contaminants); (iii) it is applicable to all types of companies/institutions, since it had in its genesis the analysis and prevention of occupational risks.

No systematic review of the literature about exposure to psychosocial risks in work contexts focusing on individual and work factors is reported. Besides, the systematic studies on instruments that assess these psychosocial factors are restricted. This study aimed to analyze the scientific literature about the effects of exposure to psychosocial risk factors in work contexts.

METHODS

Systematic review of the literature. The criteria of the Preferred reporting items for systematic reviews and meta-analyses (PRISMA)ffPRISMA Transparent Reporting of Systematic Reviews and Meta-Analyses. Ottawa: The PRISMA Statement; 2014 [cited 2014 Dec 30]. Available from: http://www.prisma-statement.org/ were applied. We identified scientific articles published in international journals using a systematic search in the databases in digital format: PubMed, Medline; in a second stage, at the Scopus database. We considered the studies published from January 1, 2004 to June 30, 2012. The research equation was [“PSYCHOSOCIAL FACTORS”], and refined to [“PSYCHOSOCIAL FACTORS” AND “COPSOQ”] because of the high number of correspondences. The study was restricted to articles published in English. The research using the full equation returned 79 abstracts with 30 correspondences in both databases (PubMed and Medline), resulting in 27 articles in full text. Of these 27, four psychometric studies and one study that did not incorporate individual or work factors were excluded, resulting in 22 articles. At the second stage of the research, on Scopus database, 13 articles were excluded because they were duplicated. The 22 articles in full text resulting from the structured research of the literature were analyzed in the review process of the literature. Cataloging and identification of repeated references were made by the software program of bibliographical referencing EndNote (Table 1; Figure).

Table 1
Criteria for inclusion and exclusion of studies about exposure to psychosocial risk factors in work contexts.

Figure
Flowchart – application of the inclusion and exclusion criteria to the researched studies.

Exposure to work-related psychosocial factors was codified according to the categories of COPSOQ: labor requirements, work organization and content, social relations and leadership, work-home interface, workplace values, personality, health and wellness, and offensive behaviors1313. Kristensen TS, Hannerz H, Hogh A, Borg V. The Copenhagen Psychosocial Questionnaire: a tool for the assessment and improvement of the psychosocial work environment. Scand J Work Environ Health. 2005;31(6):438-49. DOI:10.5271/sjweh.948. The individual factors (age, gender) were also coded.

Information on the type of study, sample, country, activity sector, main findings, and conclusions was collected, compiled and analyzed.

RESULTS ANALYSIS AND DISCUSSION

We included 22 articles according to the inclusion and exclusion criteria (Table 1). We covered 51,894 people, of nine different nationalities. In three studies, the nationality variable was not described. The most represented nationalities were Danish, mentioned in nine of the 22 studies, Swedish and Dutch, both present in four studies, followed by workers from Germany and Spain in three studies. Switzerland’s workers were present in two studies and those of Poland and China, in one. The working population in general (irrespective of sector or activity area) prevailed in the study (48.0%), followed by health care workers (33.0%), and the staff of prison services (14.0% of people covered in all the analyzed studies); civil servants, hoteliers, musicians, services, and farmers were also studied. It was used the correlational study methodology the most, and the longitudinal methodology was in the basis of the study, covering approximately 0.7% of all people studied. Four studies used methodology combined with the longitudinal design (average duration of 13 months), increasing the robustness of the research and of the obtained results. The most studied categories in the study of exposure to psychosocial factors in the work place were: labor demands (18 studies), organization of work and content (14), social relationships and leadership and health and well-being (both 13 categories), work-individual interface (12), offensive behavior (eight), and the categories values in the workplace, justice and respect, and personality (seven). Eight studies used a global index of exposure to psychosocial risk factors (Table 2).

Table 2
Summary of information about the 22 studies included regarding exposure to psychosocial risk factors in work contexts.

In general, mental health and psychosocial well-being of employees have deteriorated regardless of the psychosocial factor evaluated, with special focus on the emotional demands11. Albertsen K, Rugulies R, Garde AH, Burr H. The effect of the work environment and performance-based self-esteem on cognitive stress symptoms among Danish knowledge workers. Scand J Public Health. 2010;38(3 Suppl):81-9. DOI:10.1177/1403494809352104,22. Aust B, Rugulies R, Finken A, Jensen C. When workplace interventions lead to negative effects: learning from failures. Scand J Public Health. 2010;38(3 Suppl):106-19. DOI:10.1177/1403494809354362,44. Burke R. Corporate wellness programs: an overview. In: Burke RJ, Richardsen AM, editors. Corporate wellness programs: linking employee and organizational health. Cheltenham: Edward Elgar; 2014. p.13-27.,88. Ghaddar A, Ronda E, Nolasco A. Work ability, psychosocial hazards and work experience in prison environments. Occup Med (Lond). 2011;61(7):503-8. DOI:10.1093/occmed/kqr124,99. Holst GJ, Paarup HM, Baelum J. A cross-sectional study of psychosocial work environment and stress in the Danish symphony orchestras. Int Arch Occup Environ Health. 2012;85(6):639-49. DOI:10.1007/s00420-011-0710-z,1212. Kolstrup C, Lundqvist P, Pinzke S. Psychosocial work environment among employed Swedish dairy and pig farmworkers. J Agromed. 2008;13(1):23-36. DOI:10.1080/10599240801986157,1515. Larsman P, Sandsjö L, Klipstein A, Vollenbroek-Hutten MM, Christensen H. Perceived work demands, felt stress, and musculoskeletal neck/shoulder symptoms among elderly female computer users. The NEW study. Eur J Appl Physiol. 2006;96(2):127-35. DOI:10.1007/s00421-004-1263-6,1818. Li J, Fu H, Hu Y, Shang L, Wu Y, Kristensen TS, et al. Psychosocial work environment and intention to leave the nursing profession: results from the longitudinal Chinese NEXT study. Scand J Public Health. 2010;38(3 Suppl):69-80. DOI:10.1177/1403494809354361,2020. Llorens C, Alós R, Cano E, Font A, Jódar P, López V, et al. Psychosocial risk exposures and labour management practices: an exploratory approach. Scand J Public Health. 2010;38(3 Suppl):125-36. DOI:10.1177/1403494809354363,2121. Mache S, Vitzthum K, Nienhaus A, Klapp BF, Groneberg DA. Physicians’ working conditions and job satisfaction: does hospital ownership in Germany make a difference? BMC Health Serv Res. 2009;9:148. DOI:10.1186/1472-6963-9-148,2323. Moncada S, Pejtersen JH, Navarro A, Llorens C, Burr H, Hasle P, et al. Psychosocial work environment and its association with socioeconomic status: a comparison of Spain and Denmark. Scand J Public Health. 2010;38(3 Suppl):137-48. DOI:10.1177/1403494809353825

24. Nielsen K, Albertsen K, Brenner SO, Smith-Hansen L, Roepsdorff C. Comparing working conditions and physical and psychological health complaints in four occupational groups working in female-dominated workplaces. Int Arch Occup Environ Health. 2009;82(10):1229-39. DOI:10.1007/s00420-009-0464-z
-2525. Nübling M, Vomstein M, Schmidt SG, Gregersen S, Dulon M, Nienhaus A. Psychosocial work load and stress in the geriatric care. BMC Public Health. 2010;10:428.DOI:10.1186/1471-2458-10-428
https://doi.org/10.1186/1471-2458-10-428...
,2929. Rugulies R, Aust B, Pejtersen JH. Do psychosocial work environment factors measured with scales from the Copenhagen Psychosocial Questionnaire predict register-based sickness absence of 3 weeks or more in Denmark? Scand J Public Health. 2010;38(3 Suppl):42-50. DOI:10.1177/1403494809346873

30. Sandsjö L, Larsman P, Vollenbroek-Hutten MMR, Läubli T, Juul-Kristensen B, Klipstein A, et al. Comparative assessment of study groups of elderly female computer users from four European countries: questionnaires used in the NEW study. Eur J Appl Physiol. 2006;96(2):122-6. DOI:10.1007/s00421-005-0041-4

31. Schenk P, Läubli T, Hodler J, Klipstein A. Symptomatology of recurrent low back pain in nursing and administrative professions. Eur Spine J. 2007;16(11):1789-98. DOI:10.1007/s00586-007-0346-6
-3232. Sharipova M, Hogh A, Borg V. Individual and organizational risk factors of work-related violence in the Danish elder care. Scand J Caring Sci. 2010;24(2):332-40. DOI:10.1111/j.1471-6712.2009.00724.x. This deterioration manifests itself in increased levels of physiological stress and cognitive stress11. Albertsen K, Rugulies R, Garde AH, Burr H. The effect of the work environment and performance-based self-esteem on cognitive stress symptoms among Danish knowledge workers. Scand J Public Health. 2010;38(3 Suppl):81-9. DOI:10.1177/1403494809352104,99. Holst GJ, Paarup HM, Baelum J. A cross-sectional study of psychosocial work environment and stress in the Danish symphony orchestras. Int Arch Occup Environ Health. 2012;85(6):639-49. DOI:10.1007/s00420-011-0710-z,1111. Kiss P, De Meester M, Kruse A, Chavée B, Braeckman L. Comparison between the first and second versions of the Copenhagen Psychosocial Questionnaire: psychosocial risk factors for a high need for recovery after work. Int Arch Occup Environ Health. 2013;86(1):17-24. DOI:10.1007/s00420-012-0741-0,1515. Larsman P, Sandsjö L, Klipstein A, Vollenbroek-Hutten MM, Christensen H. Perceived work demands, felt stress, and musculoskeletal neck/shoulder symptoms among elderly female computer users. The NEW study. Eur J Appl Physiol. 2006;96(2):127-35. DOI:10.1007/s00421-004-1263-6 on workers. This increased stress makes the workers express their answers: i) physiologically, with neuroendocrine and immune reactions; ii) emotionally, with manifestation of feelings of anxiety, depression or depressive symptoms, alienation, apathy, among others; iii) cognitively, with restriction of perception, alteration of concentration capacity and creativity, difficulty in decision-making; iv) behaviorally, e.g., with substance abuse – alcohol, tobacco, drugs, violence.

These symptoms are usually seen as resulting from stress and are associated with clinical conditions with inconstant intensity, frequency, and durationgg European Commission Directorate-General for Employment, Industrial Relations and Social Affairs. Guidance on work related stress: spice of life or kiss of death? Luxembourg: Office for Official Publications of the European Communities; 2000., e.g., back pain, shoulder pain, headaches, or more complex conditions and with greater severity such as gastrointestinal disorders, ischemic heart disease, type II diabetes, mental illness and suicides associated with the workstationhhInternational Labour Organization. List of occupational diseases: revised 2010. Geneva; 2010.,iiInternational Labour Organization. Mental health in the workplace. Geneva; 2000.. The worker perception of a deteriorated psychosocial environment appears associated with higher levels of absenteeism in the workplace, because of sick leaves, for instance.

Bad management practices and destructive leadership on the part of the managers showed harmful effects on the psychosocial organizational environment and on the individual well-being of workers2020. Llorens C, Alós R, Cano E, Font A, Jódar P, López V, et al. Psychosocial risk exposures and labour management practices: an exploratory approach. Scand J Public Health. 2010;38(3 Suppl):125-36. DOI:10.1177/1403494809354363,2626. Nyberg A, Holmberg I, Bernin P, Alderling M, Akerblom S, Widerszal-Bazyl M, et al. Destructive managerial leadership and psychological well-being among employees in Swedish, Polish, and Italian hotels. Work. 2011;39(3):267-81. DOI:10.3233/WOR-2011-1175, such as: timetables above 40 hours per week, shift work, overtime unpaid, working methods not reported, little autonomy, impossibility of career development, excessive workloads. The number of hours worked per week influenced the way workers perceived the work environment in general. These two variables were negatively correlated2525. Nübling M, Vomstein M, Schmidt SG, Gregersen S, Dulon M, Nienhaus A. Psychosocial work load and stress in the geriatric care. BMC Public Health. 2010;10:428.DOI:10.1186/1471-2458-10-428
https://doi.org/10.1186/1471-2458-10-428...
. Extended working hours, even if paid, as the hours worked in period of “prevention”, negatively influenced how the employees evaluated the psychosocial factors associated with the daily laboring. The good management practices related to effective management of working time and the stability of timetables and working contracts, associated with the cordiality and non-aggressive behavior by managers were favorable to the development and perception by workers of healthier psychosocial environments.

The relationship between work psychosocial environment and the prevalence of musculoskeletal injuries (MI) varied according to the population under study and the type of function performed1212. Kolstrup C, Lundqvist P, Pinzke S. Psychosocial work environment among employed Swedish dairy and pig farmworkers. J Agromed. 2008;13(1):23-36. DOI:10.1080/10599240801986157,1515. Larsman P, Sandsjö L, Klipstein A, Vollenbroek-Hutten MM, Christensen H. Perceived work demands, felt stress, and musculoskeletal neck/shoulder symptoms among elderly female computer users. The NEW study. Eur J Appl Physiol. 2006;96(2):127-35. DOI:10.1007/s00421-004-1263-6,3030. Sandsjö L, Larsman P, Vollenbroek-Hutten MMR, Läubli T, Juul-Kristensen B, Klipstein A, et al. Comparative assessment of study groups of elderly female computer users from four European countries: questionnaires used in the NEW study. Eur J Appl Physiol. 2006;96(2):122-6. DOI:10.1007/s00421-005-0041-4,3131. Schenk P, Läubli T, Hodler J, Klipstein A. Symptomatology of recurrent low back pain in nursing and administrative professions. Eur Spine J. 2007;16(11):1789-98. DOI:10.1007/s00586-007-0346-6. With increasing age, also increases the likelihood of MI occurrence. Demanding work environments have led to a higher incidence of MI in blue-collar professions, because of its mediating effect on stress mechanisms. Labor demands, in particular the quantitative ones and imposed work rate presented themselves as factors relevant to the development of MI in the areas of the neck, shoulders, and lower back. These types of injuries also correlated with repetitive functions or in which the employee remained too long in the same position, constantly requesting the same muscle groups or functions that would require lifts and frequent manipulations. Worker complaints about the prevalence of MI were corroborated by medical examinations or medical diagnosis.

The interference of work on family life was an important factor for understanding the work-home conflict. The work and family demands have increased the need for effective management of time by the worker, and may lead to time conflict and impossibility to reconcile these demands. This interference was more pronounced in females. This factor presented itself as an increased risk in the event of bad interpersonal relationships with coworkers and lack of organization55. Burr H, Albertsen K, Rugulies R, Hannerz H. Do dimensions from the Copenhagen Psychosocial Questionnaire predict vitality and mental health over and above the job strain and effort-reward imbalance models? Scand J Public Health. 2010;38(3 Suppl):59-68. DOI:10.1177/1403494809353436,77. Fuss I, Nübling M, Hasselhorn HM, Schwappach D, Rieger MA. Working conditions and Work-Family Conflict in German hospital physicians: psychosocial and organisational predictors and consequences. BMC Public Health. 2008;8:353. DOI:10.1186/1471-2458-8-353,88. Ghaddar A, Ronda E, Nolasco A. Work ability, psychosocial hazards and work experience in prison environments. Occup Med (Lond). 2011;61(7):503-8. DOI:10.1093/occmed/kqr124,1111. Kiss P, De Meester M, Kruse A, Chavée B, Braeckman L. Comparison between the first and second versions of the Copenhagen Psychosocial Questionnaire: psychosocial risk factors for a high need for recovery after work. Int Arch Occup Environ Health. 2013;86(1):17-24. DOI:10.1007/s00420-012-0741-0,1212. Kolstrup C, Lundqvist P, Pinzke S. Psychosocial work environment among employed Swedish dairy and pig farmworkers. J Agromed. 2008;13(1):23-36. DOI:10.1080/10599240801986157,1818. Li J, Fu H, Hu Y, Shang L, Wu Y, Kristensen TS, et al. Psychosocial work environment and intention to leave the nursing profession: results from the longitudinal Chinese NEXT study. Scand J Public Health. 2010;38(3 Suppl):69-80. DOI:10.1177/1403494809354361,2020. Llorens C, Alós R, Cano E, Font A, Jódar P, López V, et al. Psychosocial risk exposures and labour management practices: an exploratory approach. Scand J Public Health. 2010;38(3 Suppl):125-36. DOI:10.1177/1403494809354363,2121. Mache S, Vitzthum K, Nienhaus A, Klapp BF, Groneberg DA. Physicians’ working conditions and job satisfaction: does hospital ownership in Germany make a difference? BMC Health Serv Res. 2009;9:148. DOI:10.1186/1472-6963-9-148,2323. Moncada S, Pejtersen JH, Navarro A, Llorens C, Burr H, Hasle P, et al. Psychosocial work environment and its association with socioeconomic status: a comparison of Spain and Denmark. Scand J Public Health. 2010;38(3 Suppl):137-48. DOI:10.1177/1403494809353825,2424. Nielsen K, Albertsen K, Brenner SO, Smith-Hansen L, Roepsdorff C. Comparing working conditions and physical and psychological health complaints in four occupational groups working in female-dominated workplaces. Int Arch Occup Environ Health. 2009;82(10):1229-39. DOI:10.1007/s00420-009-0464-z,2929. Rugulies R, Aust B, Pejtersen JH. Do psychosocial work environment factors measured with scales from the Copenhagen Psychosocial Questionnaire predict register-based sickness absence of 3 weeks or more in Denmark? Scand J Public Health. 2010;38(3 Suppl):42-50. DOI:10.1177/1403494809346873

30. Sandsjö L, Larsman P, Vollenbroek-Hutten MMR, Läubli T, Juul-Kristensen B, Klipstein A, et al. Comparative assessment of study groups of elderly female computer users from four European countries: questionnaires used in the NEW study. Eur J Appl Physiol. 2006;96(2):122-6. DOI:10.1007/s00421-005-0041-4
-3131. Schenk P, Läubli T, Hodler J, Klipstein A. Symptomatology of recurrent low back pain in nursing and administrative professions. Eur Spine J. 2007;16(11):1789-98. DOI:10.1007/s00586-007-0346-6. We considered the free time as period of rest and replacement of physical and mental energies. However, workplaces with high demands and insecurity in the workstation led to weak recovery after work1111. Kiss P, De Meester M, Kruse A, Chavée B, Braeckman L. Comparison between the first and second versions of the Copenhagen Psychosocial Questionnaire: psychosocial risk factors for a high need for recovery after work. Int Arch Occup Environ Health. 2013;86(1):17-24. DOI:10.1007/s00420-012-0741-0, contributing to the deterioration of health in general and to the low productivity of workers.

The socioeconomic status and the nationality of the workers were two variables that influenced how the work psychosocial environment is perceived2323. Moncada S, Pejtersen JH, Navarro A, Llorens C, Burr H, Hasle P, et al. Psychosocial work environment and its association with socioeconomic status: a comparison of Spain and Denmark. Scand J Public Health. 2010;38(3 Suppl):137-48. DOI:10.1177/1403494809353825, and were two important factors for the reduction of social inequalities when there is the intention to intervene in the workplace. Geographic variability and individual cultural specificities were evident as the influence of individual and work factors on the perception of exposure to psychosocial risks, with the most vulnerable groups such as migrants and the non-resident population2727. Olesen K, Carneiro IG, Jorgensen MB, Flyvholm M-A, Rugulies R, Rasmussen CDN, et al. Psychosocial work environment among immigrant and Danish cleaners. Int Arch Occup Environ Health. 2012;85(1):89-95. DOI:10.1007/s00420-011-0642-7. The groups of workers displaced from residence show more risks to physical and mental health, especially to hypertension and to the perception of work meaning2727. Olesen K, Carneiro IG, Jorgensen MB, Flyvholm M-A, Rugulies R, Rasmussen CDN, et al. Psychosocial work environment among immigrant and Danish cleaners. Int Arch Occup Environ Health. 2012;85(1):89-95. DOI:10.1007/s00420-011-0642-7,2828. Olesen K, Carneiro IG, Jorgensen MB, Rugulies R, Rasmussen CDN, Sogaard K, et al. Associations between psychosocial work environment and hypertension among non-Western immigrant and Danish cleaners. Int Arch Occup Environ Health. 2012;85(7):829-35. DOI:10.1007/s00420-011-0728-2. Gender differences were evident in the perception of stress symptoms: women reported greater labor demands and higher levels of stress symptoms when compared with men with analogous functions and tasks1818. Li J, Fu H, Hu Y, Shang L, Wu Y, Kristensen TS, et al. Psychosocial work environment and intention to leave the nursing profession: results from the longitudinal Chinese NEXT study. Scand J Public Health. 2010;38(3 Suppl):69-80. DOI:10.1177/1403494809354361.

Older workers showed coping strategies that allowed them to deal more adequately with job demands. The work experience was a mediator factor between age and capability for the work88. Ghaddar A, Ronda E, Nolasco A. Work ability, psychosocial hazards and work experience in prison environments. Occup Med (Lond). 2011;61(7):503-8. DOI:10.1093/occmed/kqr124. The professional experience and the physical and cognitive skills also presented themselves as mediators of productivityjj Hunkler c. Age and productivity: speech at the European Summit on Innovation for Active and Healthy Ageing; 2015 Mar 9-10; Brussels, Belgium. Munich: Max Planck Institute for Social Law and Social Policy; 2015 [cited 2015 Apr 14]. Available from: http://ec.europa.eu/research/innovation-union/pdf/active-healthy-ageing/hunkler.pdf#view=fit&pagemode=none/ . Younger employees were the group with higher risk to exposure to psychosocial risk factors, especially of exposure to violent behavior related to work3232. Sharipova M, Hogh A, Borg V. Individual and organizational risk factors of work-related violence in the Danish elder care. Scand J Caring Sci. 2010;24(2):332-40. DOI:10.1111/j.1471-6712.2009.00724.x. Age was a risk factor for the performance of tasks with heavier physical loads1515. Larsman P, Sandsjö L, Klipstein A, Vollenbroek-Hutten MM, Christensen H. Perceived work demands, felt stress, and musculoskeletal neck/shoulder symptoms among elderly female computer users. The NEW study. Eur J Appl Physiol. 2006;96(2):127-35. DOI:10.1007/s00421-004-1263-6 and also a resistance and increased resilience factor in the workplace to the psychosocial level88. Ghaddar A, Ronda E, Nolasco A. Work ability, psychosocial hazards and work experience in prison environments. Occup Med (Lond). 2011;61(7):503-8. DOI:10.1093/occmed/kqr124,1515. Larsman P, Sandsjö L, Klipstein A, Vollenbroek-Hutten MM, Christensen H. Perceived work demands, felt stress, and musculoskeletal neck/shoulder symptoms among elderly female computer users. The NEW study. Eur J Appl Physiol. 2006;96(2):127-35. DOI:10.1007/s00421-004-1263-6.

Deteriorated work psychosocial environment was associated to weak physical and mental health indicators such as hypertension, MI, stress, low self-esteem, burnout, and health in general11. Albertsen K, Rugulies R, Garde AH, Burr H. The effect of the work environment and performance-based self-esteem on cognitive stress symptoms among Danish knowledge workers. Scand J Public Health. 2010;38(3 Suppl):81-9. DOI:10.1177/1403494809352104,55. Burr H, Albertsen K, Rugulies R, Hannerz H. Do dimensions from the Copenhagen Psychosocial Questionnaire predict vitality and mental health over and above the job strain and effort-reward imbalance models? Scand J Public Health. 2010;38(3 Suppl):59-68. DOI:10.1177/1403494809353436,77. Fuss I, Nübling M, Hasselhorn HM, Schwappach D, Rieger MA. Working conditions and Work-Family Conflict in German hospital physicians: psychosocial and organisational predictors and consequences. BMC Public Health. 2008;8:353. DOI:10.1186/1471-2458-8-353,1414. Kristensen TS, Jensen SM, Kreiner S, Mikkelsen S. Socioeconomic status and duration and pattern of sickness absence: a 1-year follow-up study of 2331 hospital employees. BMC Public Health. 2010;10:643. DOI:10.1186/1471-2458-10-643,2424. Nielsen K, Albertsen K, Brenner SO, Smith-Hansen L, Roepsdorff C. Comparing working conditions and physical and psychological health complaints in four occupational groups working in female-dominated workplaces. Int Arch Occup Environ Health. 2009;82(10):1229-39. DOI:10.1007/s00420-009-0464-z,2727. Olesen K, Carneiro IG, Jorgensen MB, Flyvholm M-A, Rugulies R, Rasmussen CDN, et al. Psychosocial work environment among immigrant and Danish cleaners. Int Arch Occup Environ Health. 2012;85(1):89-95. DOI:10.1007/s00420-011-0642-7. A mediocre psychosocial environment was a risk factor for the development of clinical conditions from moderate to severe gravity, causing the absence of the workers in their work, especially for sick leave2929. Rugulies R, Aust B, Pejtersen JH. Do psychosocial work environment factors measured with scales from the Copenhagen Psychosocial Questionnaire predict register-based sickness absence of 3 weeks or more in Denmark? Scand J Public Health. 2010;38(3 Suppl):42-50. DOI:10.1177/1403494809346873 or the intention to leave work permanently1818. Li J, Fu H, Hu Y, Shang L, Wu Y, Kristensen TS, et al. Psychosocial work environment and intention to leave the nursing profession: results from the longitudinal Chinese NEXT study. Scand J Public Health. 2010;38(3 Suppl):69-80. DOI:10.1177/1403494809354361.

Synthesis of methodological limitations

The methodology selected for inclusion and exclusion of studies limits the obtained results. It excludes studies with valid data, especially in an area considered to be emerging, as the study of psychosocial factors. This causes relevant factors to the understanding of this subject to be excluded, although little studied.

The selection of studies that use only the COPSOQ measuring instrument of psychosocial factors reduces the studies included in the review, although the methodology allows easier comparison of the identified variables and the nature of the instrument allows covering a greater number of factors.

A quality study on the articles resulting from the application of the methodology was not carried out. This option joined the multidimensional and multidetermined nature of the concept, thus allowing a wider analysis and discussion on the topic.

There is a risk of not including all articles with the defined criteria, using the structured search in databases. To try to mitigate this effect, a third database, Scopus, was included, though it resulted in a full matching of results that did not lead to the inclusion of any study.

There is not enough geographical dispersion. The results may represent exposure to psychosocial risk factors of some geographical areas (e.g., Nordic countries) and with different cultures. One should consider cultural relativity patterns of the populations under study. The same comment applies to the sector of activity. For example, data on the industry area are not reported, and there is evidence in other areas and sectors, such as services or health, in which studies are more plentiful.

Most of the included studies followed a correlational methodological design, and this is not the most appropriate design if we want to understand causality. From the included articles, the more robust experimental designs were little used.

Implications for practice

An increasing number of organizations is interested in programs promoting the well-being of its employees and management of psychosocial risks, despite the fact that the interventions are commonly focused on a single behavioral factor (e.g., smoking) or on groups of factors (e.g., smoking, diet, exercise). Most programs offer health education, but a small percentage of institutions really changes organizational policies or their own work environment44. Burke R. Corporate wellness programs: an overview. In: Burke RJ, Richardsen AM, editors. Corporate wellness programs: linking employee and organizational health. Cheltenham: Edward Elgar; 2014. p.13-27..

This literature review presents important information to be considered in the design of plans to promote health and well-being in the workplace, in particular in the management programs of psychosocial risks. A company can organize itself to promote healthy work environments based on psychosocial risks management, adopting some measures in the following areas:

1. Work schedules – to allow harmonious articulation of the demands and responsibilities of work function along with demands of family life and that of outside of work. This allows workers to better reconcile the work-home interface. Shift work must be ideally fixed. The rotating shifts must be stable and predictive, ranging towards morning, afternoon and evening. The management of time and monitoring of the worker must be especially careful in cases in which the contract of employment predicts “periods of prevention”.

2. Psychological requirements – reduction in psychological requirements of work.

3. Participation/control – to increase the level of control over working hours, holidays, breaks, among others. To allow, as far as possible, workers to participate in decisions related to the workstation and work distribution.

4. Workload – to provide training directed to the handling of loads and correct postures. To ensure that tasks are compatible with the skills, resources and expertise of the worker. To provide breaks and time off on especially arduous tasks, physically or mentally.

5. Work content – to design tasks that are meaningful to workers and encourage them. To provide opportunities for workers to put knowledge into practice. To clarify the importance of the task to the goal of the company, society, among others.

6. Clarity and definition of role – to encourage organizational clarity and transparency, setting jobs, assigned functions, margin of autonomy, responsibilities, among others.

7. Social responsibility – to promote socially responsible environments that promote the social and emotional support and mutual aid between coworkers, the company/organization, and the surrounding society. To promote respect and fair treatment. To eliminate discrimination by gender, age, ethnicity, or those of any other nature.

8. Security – to promote stability and safety in the workplace, the possibility of career development, and access to training and development programs, avoiding the perceptions of ambiguity and instability. To promote lifelong learning and the promotion of employability.

9. Leisure time – to maximize leisure time to restore the physical and mental balance adaptively.

The management of employees’ expectations must consider organizational psychosocial diagnostic processes and the design and implementation of programs of promotion/maintenance of health and well-being. Assessment tools, such as COPSOQ, will early identify psychosocial risk factors and facilitate a more targeted intervention, specific and directed to the said factors concerning the well-being of the individual and the best ability to work. The results obtained with an intervention in the workplace may not be always positive or desired. The negative effect of interventions33. Briner RB, Reynolds S. The costs, benefits, and limitations of organizational level stress interventions. J Organ Behav. 1999;20(5):647-64. DOI:10.1002/(SICI)1099-1379(199909)20:5<647::AID-JOB919>3.0.CO;2-1
https://doi.org/10.1002/(SICI)1099-1379(...
may arise, in which the optimism about organizational interventions acts as a barrier to development and elaboration of differentiated forms of intervention22. Aust B, Rugulies R, Finken A, Jensen C. When workplace interventions lead to negative effects: learning from failures. Scand J Public Health. 2010;38(3 Suppl):106-19. DOI:10.1177/1403494809354362.

CONCLUSIONS

We presented the relevance and surplus value of the use of a multidimensional instrument, when there is the intention to intervene in organizational context. The nature of the factors under study – psychosocial risk factors – and the multiplicity of individual and labor factors may influence the relationship. Appealing to validated instruments with the possibility to adapt to the reality and context of each company, such as the COPSOQ, is a surplus value. The resulting data of this evaluation instrument of psychosocial risks can be compared with international reference values, which facilitates signaling deviations related to reference points in the population, facilitating the intervention. Therefore, what matters is to study and understand as many variables as possible to support an “emerging” field of study and the practical intervention.

We observed interaction between individual and work-related factors, and psychosocial risk factors. The exposure of workers to poor psychosocial environments influences various levels, from the physical and mental health of employees to the general work environment of institutions and the quality of leisure and rest time. It is necessary to understand the extent of these factors in population groups little studied, such as the industry, and to use more robust experimental plans.

Acknowledgments

To Prof. Carlos F. Silva, PhD, for the comments and suggestions to improve the quality of the manuscript.

REFERENCES

  • 1
    Albertsen K, Rugulies R, Garde AH, Burr H. The effect of the work environment and performance-based self-esteem on cognitive stress symptoms among Danish knowledge workers. Scand J Public Health 2010;38(3 Suppl):81-9. DOI:10.1177/1403494809352104
  • 2
    Aust B, Rugulies R, Finken A, Jensen C. When workplace interventions lead to negative effects: learning from failures. Scand J Public Health 2010;38(3 Suppl):106-19. DOI:10.1177/1403494809354362
  • 3
    Briner RB, Reynolds S. The costs, benefits, and limitations of organizational level stress interventions. J Organ Behav 1999;20(5):647-64. DOI:10.1002/(SICI)1099-1379(199909)20:5<647::AID-JOB919>3.0.CO;2-1
  • 4
    Burke R. Corporate wellness programs: an overview. In: Burke RJ, Richardsen AM, editors. Corporate wellness programs: linking employee and organizational health. Cheltenham: Edward Elgar; 2014. p.13-27.
  • 5
    Burr H, Albertsen K, Rugulies R, Hannerz H. Do dimensions from the Copenhagen Psychosocial Questionnaire predict vitality and mental health over and above the job strain and effort-reward imbalance models? Scand J Public Health 2010;38(3 Suppl):59-68. DOI:10.1177/1403494809353436
  • 6
    Cox T. Stress research and stress management: putting theory to work. London: Health and Safety Executive; 1993. HSE Contract Research Report Nº 61/1993.
  • 7
    Fuss I, Nübling M, Hasselhorn HM, Schwappach D, Rieger MA. Working conditions and Work-Family Conflict in German hospital physicians: psychosocial and organisational predictors and consequences. BMC Public Health 2008;8:353. DOI:10.1186/1471-2458-8-353
  • 8
    Ghaddar A, Ronda E, Nolasco A. Work ability, psychosocial hazards and work experience in prison environments. Occup Med (Lond) 2011;61(7):503-8. DOI:10.1093/occmed/kqr124
  • 9
    Holst GJ, Paarup HM, Baelum J. A cross-sectional study of psychosocial work environment and stress in the Danish symphony orchestras. Int Arch Occup Environ Health 2012;85(6):639-49. DOI:10.1007/s00420-011-0710-z
  • 10
    Karasek R, Brisson C, Kawakami N, Houtman I, Bongers P, Amick B. The Job Content Questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. J Occup Health Psychol 1998;3(4):322-55. DOI:10.1037/1076-8998.3.4.322
  • 11
    Kiss P, De Meester M, Kruse A, Chavée B, Braeckman L. Comparison between the first and second versions of the Copenhagen Psychosocial Questionnaire: psychosocial risk factors for a high need for recovery after work. Int Arch Occup Environ Health 2013;86(1):17-24. DOI:10.1007/s00420-012-0741-0
  • 12
    Kolstrup C, Lundqvist P, Pinzke S. Psychosocial work environment among employed Swedish dairy and pig farmworkers. J Agromed 2008;13(1):23-36. DOI:10.1080/10599240801986157
  • 13
    Kristensen TS, Hannerz H, Hogh A, Borg V. The Copenhagen Psychosocial Questionnaire: a tool for the assessment and improvement of the psychosocial work environment. Scand J Work Environ Health 2005;31(6):438-49. DOI:10.5271/sjweh.948
  • 14
    Kristensen TS, Jensen SM, Kreiner S, Mikkelsen S. Socioeconomic status and duration and pattern of sickness absence: a 1-year follow-up study of 2331 hospital employees. BMC Public Health 2010;10:643. DOI:10.1186/1471-2458-10-643
  • 15
    Larsman P, Sandsjö L, Klipstein A, Vollenbroek-Hutten MM, Christensen H. Perceived work demands, felt stress, and musculoskeletal neck/shoulder symptoms among elderly female computer users. The NEW study. Eur J Appl Physiol 2006;96(2):127-35. DOI:10.1007/s00421-004-1263-6
  • 16
    Leka S, Cox T, editors. The European framework for psychosocial risk managemen: PRIMA - EF. Nottingham: Institute of Work, Health and Organization; 2008.
  • 17
    Leka S, Jain A. Health impact of psychosocial hazards at work: an overview. Geneva: World Health Organization; 2010.
  • 18
    Li J, Fu H, Hu Y, Shang L, Wu Y, Kristensen TS, et al. Psychosocial work environment and intention to leave the nursing profession: results from the longitudinal Chinese NEXT study. Scand J Public Health 2010;38(3 Suppl):69-80. DOI:10.1177/1403494809354361
  • 19
    Lindstrom K, Elo AL, Skogstad A, Dallner M, Gamberale F, Hottinen V, et al. User’s guide for the QPS Nordic: General Nordic Questionnaire for Psychosocial and Social Factors at Work. Copenhagen: Nordic Council of Ministers; 2000.
  • 20
    Llorens C, Alós R, Cano E, Font A, Jódar P, López V, et al. Psychosocial risk exposures and labour management practices: an exploratory approach. Scand J Public Health 2010;38(3 Suppl):125-36. DOI:10.1177/1403494809354363
  • 21
    Mache S, Vitzthum K, Nienhaus A, Klapp BF, Groneberg DA. Physicians’ working conditions and job satisfaction: does hospital ownership in Germany make a difference? BMC Health Serv Res 2009;9:148. DOI:10.1186/1472-6963-9-148
  • 22
    Marmot M, Allen J, Bell R, Bloomer E, Goldblatt P. WHO European review of social determinants of health and the health divide. Lancet 2012;380(9846):1011-29. DOI:10.1016/S0140-6736(12)61228-8
  • 23
    Moncada S, Pejtersen JH, Navarro A, Llorens C, Burr H, Hasle P, et al. Psychosocial work environment and its association with socioeconomic status: a comparison of Spain and Denmark. Scand J Public Health 2010;38(3 Suppl):137-48. DOI:10.1177/1403494809353825
  • 24
    Nielsen K, Albertsen K, Brenner SO, Smith-Hansen L, Roepsdorff C. Comparing working conditions and physical and psychological health complaints in four occupational groups working in female-dominated workplaces. Int Arch Occup Environ Health 2009;82(10):1229-39. DOI:10.1007/s00420-009-0464-z
  • 25
    Nübling M, Vomstein M, Schmidt SG, Gregersen S, Dulon M, Nienhaus A. Psychosocial work load and stress in the geriatric care. BMC Public Health 2010;10:428.DOI:10.1186/1471-2458-10-428
    » https://doi.org/10.1186/1471-2458-10-428
  • 26
    Nyberg A, Holmberg I, Bernin P, Alderling M, Akerblom S, Widerszal-Bazyl M, et al. Destructive managerial leadership and psychological well-being among employees in Swedish, Polish, and Italian hotels. Work 2011;39(3):267-81. DOI:10.3233/WOR-2011-1175
  • 27
    Olesen K, Carneiro IG, Jorgensen MB, Flyvholm M-A, Rugulies R, Rasmussen CDN, et al. Psychosocial work environment among immigrant and Danish cleaners. Int Arch Occup Environ Health 2012;85(1):89-95. DOI:10.1007/s00420-011-0642-7
  • 28
    Olesen K, Carneiro IG, Jorgensen MB, Rugulies R, Rasmussen CDN, Sogaard K, et al. Associations between psychosocial work environment and hypertension among non-Western immigrant and Danish cleaners. Int Arch Occup Environ Health 2012;85(7):829-35. DOI:10.1007/s00420-011-0728-2
  • 29
    Rugulies R, Aust B, Pejtersen JH. Do psychosocial work environment factors measured with scales from the Copenhagen Psychosocial Questionnaire predict register-based sickness absence of 3 weeks or more in Denmark? Scand J Public Health 2010;38(3 Suppl):42-50. DOI:10.1177/1403494809346873
  • 30
    Sandsjö L, Larsman P, Vollenbroek-Hutten MMR, Läubli T, Juul-Kristensen B, Klipstein A, et al. Comparative assessment of study groups of elderly female computer users from four European countries: questionnaires used in the NEW study. Eur J Appl Physiol 2006;96(2):122-6. DOI:10.1007/s00421-005-0041-4
  • 31
    Schenk P, Läubli T, Hodler J, Klipstein A. Symptomatology of recurrent low back pain in nursing and administrative professions. Eur Spine J 2007;16(11):1789-98. DOI:10.1007/s00586-007-0346-6
  • 32
    Sharipova M, Hogh A, Borg V. Individual and organizational risk factors of work-related violence in the Danish elder care. Scand J Caring Sci 2010;24(2):332-40. DOI:10.1111/j.1471-6712.2009.00724.x
  • 33
    Siegrist J, Starke D, Chandola T, Godin I, Marmot M, Niedhammer I, et al. The measurement of effort-reward imbalance at work: European comparisons. Soc Sci Med 2004;58(8):1483-99. DOI:10.1016/S0277-9536(03)00351-4

  • a
    European Agency for Safety and Health at Work. Research on the changing nature of work. Luxembourg: Office for Official Publications of the European Communities; 2002.
  • b
    European Agency for Safety and Health at Work. Expert forecast on the emerging psychosocial risks related to occupational safety and health. Luxembourg: Office for Official Publications of the European Communities; 2007. (European Risk Observatory Report, nº 50).
  • c
    US Department of Health and Human Sciences, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. The changing organization of work and the safety and health of working people: knowledge gaps and research directions. Cincinnati: NIOSH; 2002.
  • d
    International Labour Organization. Psychosocial factors at work: recognition and control. Geneva: International Labour Office; 1986. (Occupational and Health Series, 56).
  • e
    See the instrument validation manuals for different languages/countries, e.g., Silva CF. Copenhagen Psychosocial Questionnaire COPSOQ: Portugal e países africanos de língua oficial portuguesa. Lisboa: Fundação para a Ciência e a Tecnologia; 2000. Instituto Sindical de Trabajo y Salud, Environment). Manual del método CoPsoQ istas 21 para la evaluación y prevención de los riesgos psicosociales para empresas con 25 o más trabajadores y trabajadoras. Versión 1.5. Barcelona: Centro de Referencia de Organización del Trabajo y Salud; 2010.
  • f
    PRISMA Transparent Reporting of Systematic Reviews and Meta-Analyses. Ottawa: The PRISMA Statement; 2014 [cited 2014 Dec 30]. Available from: http://www.prisma-statement.org/
  • g
    European Commission Directorate-General for Employment, Industrial Relations and Social Affairs. Guidance on work related stress: spice of life or kiss of death? Luxembourg: Office for Official Publications of the European Communities; 2000.
  • h
    International Labour Organization. List of occupational diseases: revised 2010. Geneva; 2010.
  • i
    International Labour Organization. Mental health in the workplace. Geneva; 2000.
  • j
    Hunkler c. Age and productivity: speech at the European Summit on Innovation for Active and Healthy Ageing; 2015 Mar 9-10; Brussels, Belgium. Munich: Max Planck Institute for Social Law and Social Policy; 2015 [cited 2015 Apr 14]. Available from: http://ec.europa.eu/research/innovation-union/pdf/active-healthy-ageing/hunkler.pdf#view=fit&pagemode=none/

Publication Dates

  • Publication in this collection
    24 May 2016

History

  • Received
    19 Jan 2015
  • Accepted
    24 Apr 2015
Faculdade de Saúde Pública da Universidade de São Paulo São Paulo - SP - Brazil
E-mail: revsp@org.usp.br