Secciones
Referencias
Resumen
Servicios
Descargas
HTML
ePub
PDF
Buscar
Fuente


Design and validity of a questionnaire for measuring psychosocial risk factors and burnout
Diseño y validación de un instrumento de medición de factores psicosociales de riesgo asociados al desgaste laboral
DYNA, vol. 87, no. 214, pp. 66-74, 2020
Universidad Nacional de Colombia

Articles


Received: 18 February 2020

Revised document received: 22 April 2020

Accepted: 12 April 2020

DOI: https://doi.org/10.15446/dyna.v87n214.85187

Abstract: In recent years, burnout has emerged as one of the most important work risks. Continuous chronical emotional and interpersonal stressors at the workplace can lead to burnout. Among those, psychosocial risk factors are things, events, or circumstances that may affect workers' health due to their work and workplace conditions and may result in burnout. The purpose of this research was to design and validate a scale to evaluate the relationship between psychosocial risk factors and burnout within the aerospace manufacturing industry. A reliable, valid scale was obtained, which can be applied to operative personnel.

Keywords: burnout, psychosocial risk factors, scale.

Resumen: El síndrome de desgaste laboral se reconoce como uno de los riesgos emergentes más importantes en los sitios de trabajo. Dentro de los factores que contribuyen a su desarrollo, están los psicosociales, que son hechos, acontecimientos, situaciones o estados que derivan de la organización del trabajo y tienen una alta probabilidad de afectar a la salud del empleado con consecuencias destacables. Esta investigación tuvo como propósito diseñar un instrumento que permitiera medir la relación que existe entre los factores de riesgo psicosocial y el desgaste laboral, en el entorno de la industria aeroespacial de manufactura. Se obtuvo un instrumento de medición fiable y válido que puede ser aplicado a trabajadores de nivel operativo.

Palabras clave: desgaste laboral, factores de riesgo psicosocial, instrumento de medición.

1. Introduction

To this day, one of the most studied phenomena in companies is the way in which workers experience and respond to labor demands; particularly, in the last 30 years research has focused on work burnout [1]. The term “burnout” was first used in 1974, by psychiatrists Herbert Freudenberger and Christina Maslach who began to spread the concept at the annual congress of the American Psychologists Association in 1977 [2]. In recent years, it has been classified as one of the most significant emerging hazards in the workplace, in the category of psychosocial risks, along with work stress, psychological harassment (mobbing) and other forms of violence, which are known as global issues affecting all countries, professions and workers because of their impact on workers' health, performance and attendance [3].

1.1. Work related burnout

Burnout, also known as professional, occupational or emotional exhaustion syndrome [2], is not just a situation or event that occurs within the company or workplace, as for example, violence or harassment, but the result of a process of chronic labor and organizational stress that ends in a state of emotional distress and demotivating fatigue for the performance of tasks [4]. It is a response to constant emotional and interpersonal stressors that occur at work, and are maintained over time, and it is defined from three dimensions: exhaustion, cynicism and inefficiency [5]. The three components of the syndrome are seen as a line of consequences that start with exhaustion [4]: at first, the workers’ performance surges as work demands, pressures and load start to increase. Which may be positive up to a certain point; however, when the peak passes, the pressure becomes negative or stressful, performance declines and the long-term result is burnout [3].

1.2. Psychosocial risk factors

Some of the main issues that contribute to its development are psychosocial risk factors, which are facts, events, situations or states that arise from the work organization and may potentially affect the worker’s health with major consequences [4]. Seen from a broader perspective, psychosocial risks refer to the potential of psychosocial threats of causing harm as well as major economic implications at the social level affecting companies of all types and sizes [6]. Unlike physical hazards, which are directly observable and measurable, psychosocial risks are the product of social interactions, and cannot be understood outside their context [7]. Some of the psychosocial factors associated with work burnout are: new hiring schemes and job insecurity, aging of the workforce, task intensification, high emotional demands at work, poor work-life balance, long shifts, higher workloads, poor salary and promotion prospects, ambiguous roles, and time and cost pressures that increase the risk of errors or compromise ethical and quality standards [8,9].

1.3. Measuring work burnout

There are several instruments to measure work burnout that have been widely used by companies and with results reported by literature; Table 1 shows three of the most used [10]. However, none of these scales concurrently measure work burnout and the psychosocial risk factors associated with it, so there is a need to develop such a survey; particularly, one that focus on operative level personnel in the aerospace manufacturing industry.

Table 1
Work burnout measurement instruments.

Source: [10]

The purpose of this work is to show the design and validation of a scale that measures the perception of operational level workers in the aerospace manufacturing industry regarding psychosocial risk factors linked to work burnout.

2. Materials and method

2.1. Instrument design

The instrument design was conducted through a literature review from the research databases where psychosocial factors identified risks affecting workers’ occupational health and quality of life. For the search, two tools were used: Google Scholar and the Meta-searcher of the Library at the Universidad Autónoma de Baja California. Key words were health hazard, job hazard, work hazard, psychosocial factor/risk, psychological factor/risk, social factor/risk, human factor, occupational hazard, manufacturing environment. In the case of Google Scholar, the key words concatenated in phrases such as: "Psychological factors in manufacturing", "psychosocial factors and health hazards in manufacturing", "work hazards in manufacturing". In the case of the Metasearch engine, logical search connectors were used, allowing to join several terms by conjunction (AND), disjunction (OR) and exclusion (NOT). The articles were limited to the period from 2012 to 2017 and articles from arbitrated academic publications took preference. The pre-selection was made through the title and summary of the article. In the first instance, medical articles related to symptoms and palliative or corrective treatment, health conditions not associated with the work environment such as HIV/AIDS and those dealing with psychosocial risk factors in environments other than the workplace were discarded, such as those focused on teenagers or infants. From this first selection, 189 articles were obtained. In this group of studies, a detailed review was conducted to the introduction section of each article and the results reported by the authors. Articles that showed links between working conditions and effects on workers' health were selected, an those making a differentiation by sex were discarded, given that the approach is of general interest and not specific. Once the process was completed, 70 studies were obtained, with specific information on 55 factors that impact health and working conditions. Next, these factors were grouped by their definition, resulting in 10 categories that are shown in Table 2. The table also shows the number of articles that referred to each category of identified psychosocial factor, as well as the cumulative percentage. As shown, workload imbalance had the highest number of mentions in literature, with 63 (19%); the second place, with 57 (17%) references, corresponded to lack of empowerment and the third was poor interpersonal relationships, with 53 (16%).

Table 2
Psychosocial factor categories reported by literature.

Source: Statistics based on the literature review made by the authors

Categories that represented 80% of the mentions in articles were selected. As shown in Table 2, these include: workload imbalance, lack of empowerment, poor interpersonal relationships, insufficient rewards, role conflict and ambiguity, and uncertainty about new labor schemes; leaving out: lack of work-life balance, unfitting work environment, discrimination and physical hazard.

2.2. Conceptual definition of constructs

In order to form a theoretic basis for the operationalization, conceptual definitions for each of the constructs were developed. These are shown in Table 3.

Table 3
Conceptual definitio n for constructs of interest.

Source: The Authors

2.3. Operationalization of the instrument

For the operationalization process, we identified 6 other instruments associated with psychosocial factors of interest: the one prepared by the Mexican Ministry of Labor and Social Welfare (STPS) described in NOM-STPS-035-2018 [17]; the FPSICO, by the National Institute of Occupational Health and Safety of Spain (INSHT) [18]; the instrument by the British Health and Safety Executive (HSE) [19]; the Copenhagen Psychosocial Questionnaire (COPSOQ) medium version [20]; and the Karasek demand-control model questionnaire (JCQ) [21]. For measuring work burnout, the Maslach Burnout Inventory [MBI] was used [5]. From all these instruments, 80 items were adapted and subsequently validated. Table 4 shows the operationalization for each item and its associated variable. The design of the measuring instrument proposed, for its empirical application, the evaluation of the items of each construct from a Likert type scale, ranging from totally disagree (1), to totally agree (5).

2.4 Validating the measurement instrument

The validation process occured in two stages: validating the content through the method proposed by Lawshe [22] and standardized by Tristán-López [23]; and determination of reliability by empiricaly testing the survey on a pilot sample and using Chronbach’s Alfa calculation on the results obtained.

2.4.1. Content validation

To verify if the items were suitable to measure the subject matter, the method proposed by Lawshe [22] was used. This methodology consists in gathering the opinions of an expert panel about the variables to be validated. Tristán-López [23] standardized this technique allowing the validation to be independent of the number of judges. The questionnaire was emailed to 18 potential judges during the month of June 2018 and 8 responses were obtained by August of that same year. The expert’s qualifications are shown in Table 5, which also shows that 62.5% of respondents were women, the most common level of education was a doctorate (62.5%) and the main areas of expertise were Psychology and Health and Safety (37.5% in both cases).

Table 4
Operationalization for each item

Source: The Authors

Table 5
Descriptive statistics of the experts validating the instrument.

Source: The Authors

Each expert evaluated the items in three categories: essential (1), useful but not essential (2) and not necessary (3). The Content Validity Ratio (CVR) was calculated from the experts’ answers. This indicator is determined dividing the number of agreements on the essential category, by the number of participating judges. Results must provide at least 58.23% (0.5823) of agreements to be acceptable; eq. (1) demonstrates the calculation [23].

(1)

Where:

ne = number of experts that agree on the “essential” category

N = total number of experts

According to the criteria above, 73 items had a value greater than the minimum cut-off point and 7 had to be eliminated for not reaching this value. The list of eliminated items by construct is shown in Table 6.

Table 6
List of eliminated items by construct

Source: The Authors

Then, the Content Validity Index (CVI) was calculated. The CVR values of the acceptable items were added, and the result was divided by the total number of acceptable items, as shown in eq. (2). The lower threshold for the CVI, similar to the CVR’s, is 0.5823 [23]; the index calculated for the instrument was 0.79, which is greater than the threshold. Therefore, it was determined that the instrument was within the acceptable range.

(2)

Where:

CVRi = Content Validity Ratio of acceptable items

M = Total acceptable items

2.4.2. Instrument reliability

Measuring internal consistency is the most common way to estimate the reliability of a test or scale, when sets of items are expected to measure the same attribute or content; the most used coefficient is Cronbach's Alpha, which measures the test reliability based on the length and covariance between items [23]. Therefore, when the expert validation concluded, the reliability of the instrument was evaluated through a pilot study in which the survey was administered to a sample of 30 operators working in an aerospace company in Mexicali, Baja California, where access and empirical administration were allowed. Cronbach’s Alpha was then calculated with the survey results, to verify their reliability. The software used was SPSS version 22. The individual constructs, as well as the whole measuring instrument, obtained values greater than 0.70, which is considered adequate [25]. Therefore, the instrument was considered reliable and valid to be administered to the population of interest. The final instrument then consists of 73 items that evaluate six different constructs of psychosocial risk and one more that measures work burnout.

2.4.3. Evaluation of the scale through empirical validation

In order to validate the scale, it was distributed to a sample consisting of 394 aerospace industry operators in Mexicali, Baja California. There were 367 (93.1%) valid responses that were used to perform the calculations. On a first run, several of the constructs were identified as non-unidimensional. A main component factor analysis was performed to determine the factor structure of each of the problematic constructs, using the Kaiser-Meyer-Olkin (KMO) coefficient and the Bartlett Sphericity tests. Data was analyzed using SPSS 22.0. The resulting two-dimensional structure is shown on Table 7. Further analysis resulted on the elimination of 8 items: 7 due to low cross-loading values (C5, C8, EMP1, EMP5, NVO6. REL9, and REL10), and 1 due to co-linearity (NVO3). The statistical analysis for the resulting model was performed with Partial Least Squares Structural Equation Modeling (PLSSEM) technique using the SmartPLS 3.0 software.

Table 7
Two-dimensional structure and number of items per construct.

Source: The Authors

3 Results

3.1. Internal Consistency Reliability and Convergent Validity

The traditional criterion for internal consistency is Cronbach’s alpha, which provides an estimate of the reliability based on the intercorrelations of the observed indicator variables. However, Cronbach’s alpha has some limitations: first, it assumes that all indicators are equally reliable; moreover, this criterion is sensitive to the number of items in the scale and generally tends to underestimate the internal consistency reliability. As such, it may be used as a more conservative measure of internal consistency reliability. For a more technically appropriate evaluation it is recommended to apply a different measure of internal consistency reliability, which is referred to as composite reliability. This measure of reliability takes into account the different outer loadings of the indicator variables [26]. Table 7 shows the calculated values of Cronbach’s alpha, and composite reliability. It can be seen that two values are below 0.600 for Cronbach’s alpha: “High demand” (0.599) and “Involvement” (0.402). However, all values are greater than 0.700 for composite reliability.

Convergent validity signifies that a set of indicators represents one and the same underlying construct, which can be demonstrated through their unidimensionality [27]. To evaluate it, researchers consider the outer loadings of the indicators and the average variance extracted (AVE) [26]. The results in Table 8, show values that vary from 0.501 (High demand) to 0.789 (Efficacy). All calculations are greater than 0.500, the minimum value accepted.

Table 8
Internal consistency reliability and convergent validity by latent variable

Source: The Authors

3.2. Discriminant Validity

According to Hair et al. [26], discriminant validity is the extent to which a construct is truly distinct from other constructs by empirical standards. Thus, establishing

discriminant validity implies that a construct is unique and captures phenomena not represented by other constructs in the model. Two criteria were calculated to test the discriminant validity: the Fornell-Larcker criterion and the heterotrait-monotrait ratio (HTMT).

The Fornell-Larcker criterion postulates that a latent variable shares more variance with its assigned indicators than with any other latent variable. In statistical terms, the AVE of each latent variable should be greater than the latent variable’s highest squared correlation with any other latent variable [27]. This is shown on the main diagonal of Table 9, where it can be seen that, for instance, for “Uncertainty”, the calculated square root of its AVE is 0.816, which is the largest number when compared to the values calculated for other latent variables along its column (underneath: 0.018, 0.210, 0.109, 0.385, 0.144, 0.357, 0.281, 0.316) and its row (to the left: 0.342, 0.094, -0.008, 0.375, 0.362, 0.391). The same is true for the rest of the latent variables.

Table 9
Fornell-Larcker criterion for latent variables.

Source: The Authors

The heterotrait-monotrait ratio (HTMT) of the correlations, is an estimate of what the true correlation between two constructs would be, if they were perfectly measured [26]. The maximum threshold level of the HTMT is debatable. Some authors suggest a threshold of 0.85, whereas others propose a value of 0.90 [28]. In this case, as shown in Table 10, all the calculated values for the latent variables are lower than the strictest threshold of 0.85, since the larger values are for the “Development”-“Change” (0.825) and the “Involvement”-“Change” (0.822) ratios.

Table 10
Heterotrait-monotrait ratio for latent variables

Source: The Authors

4. Discussion and conclusion

Work burnout syndrome is considered an occupational hazard [26], therefore, screenings at the workplace and identifying psychosocial risk factors related to burnout are crucial to ensure a healthy work environment. Per the above, the aim of this study was to design and validate a useful measuring instrument to evaluate those psychosocial risk factors and work burnout, within a manufacturing work environment in the aerospace industry. The tests showed a scale that is reliable and valid to measure work burnout plus six psychosocial risk factors: workload imbalance, poor interpersonal relationships, lack of empowerment, insufficient rewards, role conflict and ambiguity, and uncertainty about new labor schemes. The existence of such measurement instruments like this one, is essential not just to evaluate the presence of the syndrome, but also to identify its potential causes and work on appropriate solutions.

Previous studies indicate that this syndrome is not a problem created by employees, but one that arises in the workplace when the company does not recognize the human side of work, or demands an excessive work effort, which results on people feeling overwhelmed, frustrated and worn out [27]. Also, the implications on workers wellbeing may be many and varied: from headaches, insomnia, sleeping and eating disorders, fatigue, irritability, emotional instability and problematic social relationships, up to alcoholism and health problems like high blood pressure and heart failure [10].

Like any other study, this effort has several limitations. First, only the six risk factors are included and life-work imbalance, inappropriate work environment, discrimination and physical risk were not researched in depth. Also, work burnout is the only job hazard being evaluated, while others such as stress, harassment or work violence are not being considered. Moreover, it should be noted that the scale is designed for the operational level personnel of the aerospace industry and it must be adapted in the case of other work environments. Finally, in regards to the tests, even though the composite reliability and AVE values for “Involvement” are greater than the minimum accepted, the Cronbach’s alpha is very low, which requires to revisit the construct and try to improve the indicator’s reliability by adding more items, since currently it only has two.

References

Pines, A.M. and Keinan, G., Stress and burnout: the significant difference. Personality and individual differences, 39(3), pp. 625-635, 2005. DOI: 10.1016/j.paid.2005.02.009

Neffa, J.C., Los riesgos psicosociales en el trabajo: contribución a su estudio. CEIL-CONICET, Buenos Aires, Argentina, 2015.

ILO, SOLVE: Integrando la promoción de la salud a las políticas de SST en el lugar de trabajo - Guía del formador, Organización Internacional del Trabajo, Ginebra, 2012, 348 P.

Moreno-Jiménez, B. y Báez-León, C., Factores y riesgos psicosociales, formas, consecuencias, medidas y buenas prácticas, Universidad Autónoma de Madrid, España, 2010, 189 P.

Maslach, C., Schaufeli, W.B. and Leiter, M.P., Job burnout. Annual Review of Psychology, 52(1), pp. 397-422, 2001. DOI: 10.1146/annurev.psych.52.1.397

BSI, PAS 1010:2011 Guidance on the management of psychosocial risks in the workplace. British Standards Institution, England, 2011, 48 P.

Jespersen, A.H., Hohnen, P. and Hasle, P., Internal audits of psychosocial risks at workplaces with certified OHS management systems. Safety Science, 84, pp. 201-209, 2016. DOI: 10.1016/j.ssci.2015.12.013

Brun, E., Milczarek, M., Roskams, N., De Beeck, R.O., Pahkin, K., Berthet, M. and Oncins-de Frutos, M., Expert forecast on emerging psychosocial risks related to occupational safety and health, European Agency for Safety and Health at Work, Luxemburgo, 2007, 126 P.

Rožman, M., Treven, S. and Cingula, M., The impact of behavioral symptoms of burnout on work engagement of older employees: the case of Slovenian companies. Naše gospodarstvo/Our economy, 64(3), pp. 3-11, 2018. DOI: 10.2478/ngoe-2018-0013

ILO, Estrés en el trabajo: un reto colectivo, Organización Internacional del Trabajo, Ginebra, 2016, 68 P.

Kristensen, T.S., Borritz, M., Villadsen, E. and Christensen, K.B., The Copenhagen burnout inventory: a new tool for the assessment of burnout. Work & Stress, 19(3), pp. 192-207, 2005. DOI: 10.1080/02678370500297720

Bell, C., Johnston, D., Allan, J., Pollard, B. and Johnston, M., What do demand-control and effort-reward work stress questionnaires really measure?. A discriminant content validity study of relevance and representativeness of measures. British Journal of Health Psychology, 22(2), pp. 295-329, 2017. DOI: 10.1111/bjhp.12232

Torrano, F., Aja, M. y Soria, M., Métodos de evaluación psicosocial: análisis comparativo FPSICO-COPSOQ, Seguridad y Salud en el Trabajo, 89, pp. 42-55, 2016.

Cooper, C.L., Coping with stress in organizations: the role of management, in: Kalimo, R., EI-Batawi, M.A. and Cooper, C.L., eds., Psychosocial factors at work and their relation to health, World Health Organization, England, 1987, pp. 185-205.

Kompier, M.A.J., Job design and well-being, in: Schabracq, M.J., Winnubst, J.A.M. and Cooper, C.L., Eds., The Handbook of work and health psychology, Wiley Eds., Chichester, U.K., 2002, pp. 429-454.

Derycke, H., Vlerick, P., Van de Ven, B., Rots, I. and Clays, E., The impact of effort-reward imbalance and learning motivation on teachers' sickness absence. Stress & Health: Journal of the International Society for the Investigation of Stress, 29(1), pp. 14-21, 2013. DOI: 10.1002/smi.2416

STPS. Norma Oficial Mexicana NOM 035 STPS 2018, Factores de riesgo psicosocial. Identificación y prevención, Diario Oficial de la Federación, 2ª. Sección, México, 2018, pp. 85-128.

INSHT. Cuestionario de evaluación de riesgos psicosociales, Instituto Nacional de Seguridad e Higiene en el Trabajo, España, 2012, 9 P.

HSE. How to tackle work-related stress: a guide for employers on making the management standards work. Health Safety Executive, UK, 2010, 9 P.

COPSOQ. Identificación y evaluación de riesgos psicosociales. Manual de la versión media del método PSQ CAT21 COPSOQ (Cuestionario psicosocial de Copenhague) adaptado, Departamento de Trabajo, Barcelona, 2000, pp. 155-202.

Karasek, R., Brisson, Q., Kawakami, N., Houtman, I., Bongers, P. and Amick, B., The job content questionnaire (JCQ): an instrument for internationally comparative assessments of psychological job characteristics. Journal of Occupational Health Psychology, 3(4), pp. 322-355, 1998.

Lawshe, C.H., A quantitative approach to content validity. Personnel Psychology, 28(4), pp. 563-575, 1975. DOI: 10.1111/j.1744-6570.1975.tb01393.x

Tristán-López, A., Modificación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un instrumento objetivo. Avances en Medición, 6, pp. 37-48, 2008.

Ledesma, R., Molina, G. y Valero, P., Análisis de consistencia interna mediante Alfa de Cronbach: un programa basado en gráficos dinámicos. PsicoUSF, 7(2), pp. 143-152, 2002. DOI: 10.1590/S1413-82712002000200003

Bland, J.M. and Douglas, G.A. Statistics notes: Cronbach's Alpha. BMJ: British Medical Journal, 314(7080), pp. 572, 1997. DOI: 10.1136/bmj.314.7080.572

Hair, J.F., Hult, G.T.M., Ringle, C. and Sarstedt, M., A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM). SAGE, Los Angeles, USA, 2017.

Henseler, J., Ringle, C.M. and Sinkovics, R.R., The use of partial least squares path modeling in international marketing, in: New challenges to international marketing, Emerald Group Publishing Limited, London, UK, 2009, pp. 277-319.

Henseler, J., Ringle, C. and Sarstedt, M., A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), pp. 115-135, 2015. DOI: 10.1007/s11747-014-0403-8

Maslach, C. and Goldberg, J., Prevention of burnout: new perspectives. Applied and Preventive Psychology, 7(1), pp. 63-74, 1998. DOI: 10.1016/S0962-1849(98)80022-X

Maslach, C. and Leiter, M.P., Take this job and … love it! Psychology today, 32(5), pp. 50-55, 1999.

Notes

How to cite: Jacobo-Galicia, G. and Máynez-Guaderrama, A.I, Design and validity of a questionnaire for measuring psychosocial risk factors and burnout. DYNA, 87(214), pp. 66-74, July- September, 2020.

Author notes

G. Jacobo-Galicia, received the BSc. Eng. in Electronics and Cybernetic Engineering from the Centro de Enseñanza Técnica y Superior, campus Mexicali, BC. Has a MSc. from Instituto Tecnológico de Monterrey, campus Monterrey, NL. Has worked as process engineer in the electronics manufacturing sector for over 10 years. Currently, she works as full time professor-researcher at the Industrial Engineering Department of Universidad Autónoma de Baja California, Mexico.ORCID: 0000-0001-8390-300X
A.I. Máynez-Gaderrama, recived the PhD in Strategic Planning and Technology Direction from Universidad Popular Autónoma del Estado de Puebla, Mexico. Currently, she works as professor-researcher at Instituto de Ingeniería y Tecnología, Universidad Autónoma de Ciudad Juárez, Mexico. She is a member of the National Research System (SNI), level I. She has twice received the National Award on Financial Research IMEF-EY. Her research interests include: organizational factors associated with a sustainable competitive advantage.ORCID: 0000-0001-8174-3807


Buscar:
Ir a la Página
IR
Scientific article viewer generated from XML JATS4R by