Construct Validity of the Two Safety Culture Questionnaires
Authors: Fontenele Lima de Carvalho RE*, Paz de Oliveira SK, Araújo Girão AL, Fontenele Lima de Carvalho RE*, Paz de Oliveira SK, Araújo Girão AL
Journal: Nursing & Healthcare International Journal. Article type: Research Article. Published: 2019-01-30. DOI: https://doi.org/10.23880/nhij-16000172
Abstract
Objective: The aim of the study was to validate the construct of two culture assessment instruments adapted and validated for the Portuguese language, the Safety Attitudes Questionnaire (SAQ) and the Hospital on Patient Safety Culture (HSOPSC). Material and Methods: Cross-sectional, exploratory-descriptive study with quantitative approach. The two instruments were applied at the same time to all professionals working in intensive care units that met the inclusion criteria: working at least 20 hours per week and at least one month in the sector. Results: 170 questionnaires were distributed and 127 were considered valid for the research. The professionals that participated in the study, 60 (75%) were female, most had between 1 and 5 years of time in unit 40 (50%), with institution 39 (49%) and workload between 20 and 39 hours per week 32 (40%). Still, 69 (86.3%) reported having direct contact with the patient. Regarding the validation results, we observed positive and statistically significant correlations between SAQ domains with four HSOPSC domains. In the regression analysis, all domains of the SAQ, except for Stress Perception, explained all the variables resulting from HSOPSC. Conclusion: Based on this study it was concluded that both instruments are valid and reliable for the evaluation of safety culture. However, it is worth emphasizing that the two instruments present factors that facilitate and hinder the application of the same.
Keywords: Nursing, Intraclass, Safety
Introduction
Safety culture is increasingly recognized as an important strategy and precursor needed to improve patient safety. According to the United States Institute of Medicine, safety culture can be defined as an individual and organizational behavior that continually seeks to establish a commitment to the safety and quality of services provided [1]. The first step in implementing an institution's safety culture is to evaluate it. Assessing safety culture in health facilities can reveal conditions in a work environment that lead to adverse events. This evaluation can be performed soon after interventions aimed at patient safety and as a method of monitoring the effectiveness of these interventions, in addition to increasing the awareness of professionals and patients about safety issues.
In Brazil, there are two safety culture assessment instruments translated and adapted for the Portuguese language, the Safety Attitude Questionnaire (SAQ) and the Hospital on Patient Safety Culture (HSOPSC). These two instruments are the most applied in the world for this purpose, as well as being the only safety culture evaluation instruments translated and adapted to the Portuguese language [2-4]. SAQ was built in the United States by researchers at the University of Texas, Center of Excellence for Patient Safety and has been applied in more than 500 hospitals in the United States, United Kingdom and New Zealand. This instrument has 41 items distributed on a single page and measures the perception of health professionals through six domains: Team Work Climate, Safety Climate, Job Satisfaction, Stress Perception, Management Perception, Working Conditions [2]. HSOPSC was developed and made available by the US Agency for Research and Quality of Health (AHRQ) in 2004 [5] and translated and validated for Brazil in 2013 [4], and organizational norms, reporting of adverse events, communication, supervision and management. The HSOPSC contains 50 items distributed in 12 points in item scale [5]. Since the items were written in both directions as positive and negative, the written items were partially reversed. The validation process of a validation instrument during cross-cultural adaptation concluded that, unlike the validation process, the evidence was repeated at various times, evidencing the importance of a continuous evaluation of the instruments, since the two are reference tools for assessing safety culture. Construct validity is based on what is really thought about the median measures. The necessary evidence for this type of evaluation can be obtained from the internal analysis of the instrument, through confirmatory factor analysis, internal consistency and its correlations with other testicles, by the preference that they are accepted as construct measures. The higher the correlations are, the more the test will be the measurement sense of the construct in question [6]. Given the availability of two instruments in the Portuguese language that evaluates the same construct, but with different characteristics, which seeks to answer as the main research questions: Is the SAQ domain convergent to the same constructs and evaluated by the HSOPSC domains? As predictive variables (domains) they explained as a result. In this sense, the Safety Study Questionnaire (SAQ) and the Hospital on Patient Safety Culture (HSOPSC).
Materials and Methods
Cross-sectional, exploratory-descriptive study with quantitative measures in intensive care units of two hospitals in Brazil. The hospitals that participated in the study were part of a larger study evaluating the culture of Brazilian hospitals [7]. The construct validation handles, the two instruments were distributed to all individuals working in the intensive care units of the hospitals participating in the study, corresponding to a sample of 170 professionals. However, 127 returned the completed instruments. The study included nurses, physicians, nursing assistants, physiotherapists, nutritionists, psychologists, pharmacists and social workers who met the inclusion criteria: less than 20 hours a week and less than a month in the sector. Data were collected through the Safety Attitude Questionnaire (SAQ) and the Hospital on Patient Safety Culture (HSOPSC), both translated and validated for Brazil, as well as a sociodemographic characterization form. One answer to each question is a 5-point scale, including the final variation of 100, with a cut-off point of 752. The HSOPSC covers 12 dimensions of the safety culture in various sizes. It contains 50 items in total; 44 are related to safety culture and 6 items are related to personal information [4]. The study participants signed and received the Informed Consent Term. The instruments were filled in the working environment available on paper. The study was approved by the Ethics Committee of the institutions and has protocol number 985.564. All participants in the study signed the informed consent form and were guaranteed anonymity. The collected data were inserted into a database and then processed in the R software. Each questionnaire had its results converted into scores according to the value established for each option. To make this comparison possible, the HSOPSC scores were normalized according to the SAQ score. The calculation of the Intraclass Correlation Coefficient (ICC) was applied to evaluate the correlation between the domains, and the mixed two way model was chosen, with a 95% confidence interval. The interpretation of ICC values was based on the scale of values suggested by Menz, et al. [8,9], which are: greater than 0.75 are considered excellent; values between 0.40-0.75 are moderate and / or satisfactory and values below 0.40 are unsatisfactory. The reliability of the instruments was measured by Cronbach's alpha and was considered 0.69, the lower limit generally accepted in exploratory research. To understand how much each predictive variable (domains) explained the outcome variables, the Regression Analysis was performed. The following variables were considered: Number of events reported in the last 12 months (Section D), Patient Safety Grade (Section E) and Overall Perceptions of Patient Safety (A10, A15, A17 e A18).
Results
For the validation of construct, the two instruments were applied at the same time, being answered by 127 professionals. Of these, 68 (75%) professionals were female, with 1 to 5 years of time in unit 59 (46.5%) and with a workload of 20 to 60 hours per week 99 (60%). Still, 115 (90.6%) professionals reported having direct contact with the patient (Table 1).
Regarding the Infraclass Correlation Coefficient, SAQ presented significant ICC 0.63 for all domains, considering moderate to excellent reproducibility. The HSOPSC showed ICC ranging from 0.33 to 0.91 significant for six domains and reproducibility ranging from poor to excellent (Table 3).
significant association, considered excellent reproducibility. The domain Teamwork Climate (SAQ) was significantly correlated with five HSOPSC domains, namely: Learning, Teamwork of the unit, Feedback, Degree of safety and Frequency of reported events. This correlation makes sense since good quality of the relationship and collaboration among team members occurs when there are positive changes and learning from the reported events, when team members support each other, treat each other with respect and work together as a team. The other HSOPSC domains had positive correlations for one or two SAQ domains. Similar results were identified in the study by Etchegaray and Thomas [25] that demonstrated the predictive reliability and validity in both SAQ and HSOPSC instruments. The regression analysis allowed us to understand how much each predictive variable (domains) explains the outcome variables. The three HSOPSC variables explained all SAQ domains, with emphasis on the domains of Safety, Teamwork Climate, Work Satisfaction and Hospital Management Perception that were explained by all outcome variables. While two variables result General Degree of Security and General Perception of Security, they explained HSOPSC domains. The evaluation of the reliability and validity of the two instruments was consistent with previous studies that evaluated the psychometric characteristics of the two instruments [3,4,25]. However, this was the first construct validation survey with the two brazilian instruments that evaluate safety culture, so it is emphasized the need for more research with additional safety culture results to elucidate the strengths and weaknesses of the two instruments. It was observed that almost all domains of SAQ are correlated with some domains of HSOPSC and all variable outcomes. In addition, all variables resulting from the HSOPSC explain the domains of the SAQ.
Conclusion
It was concluded that both instruments are valid and reliable for evaluating the safety culture. It is worth emphasizing that the two instruments present factors that facilitate and hinder their application. The SAQ has a single page and has been evaluated by health professionals as a quick and practical tool to fill out, but does not have outcome evaluation items and suggestions space. The HSOPSC is a long six-page instrument, a characteristic identified as an impeding factor by many professionals, reflecting the low adherence of professionals in this study. It should be highlighted that this instrument presents variables that allow the evaluation of results, indicators that can complement and confirm the evaluated constructs. In addition to possessing a space for the writing of suggestions. The two culture evaluation questionnaires available in the Portuguese language can be useful tools in the initial diagnosis of the institution's safety culture and assist managers in decision making.
References
1. Kohn LT, Corrigan JM, Donaldson MC (2000) To Err is Human: building a safer health system. Committee on Quality of Health Care, Institute of Medicine, National Academy Press Washington (DC).
2. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella
K, et al. (2006) The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 6: 44.
3. Carvalho REFL, Cassiani SHB (2012) Cross-cultural
adaptation of the Safety Attitudes Questionnaire- Short Form 2006 for Brazil. Revista Latino-Americana de Enfermagem 20(3): 575-582.
4. Reis CT, Martins M, Laguardia J (2012) Transcultural
adaptation of the Brazilian version of the Hospital Survey on Patient Safety Culture: initial stage. Cad Saúde Pública, Rio de Janeiro 28(11): 2199-2210.
5. Agency for Healthcare Research and Quality (2015)
Hospital Survey on Patient Safety Culture: 2004. User comparative database Report, AHRQ.
6. Sternick C (2017) Technical characteristics of a measuring instrument. Arquivos Brasileiros de Psicologia Aplicada 28 (1): 48-67.
7. Carvalho RE, Arruda LP, Birth NK, Sampaio RL,
Cavalcante ML, et al. (2017 ) Assessment of the safety culture in public hospitals in Brazil. Rev Lat Am Nursing 25: e2849.
8. Menz HB, Latt MD, Tiedemann A, Mun San Kwan M,
Lord SR (2004) Reliability of the GAITRite walkway system for the quantification of temporal spatial parameters of gait in young and older people. Gait Posture 20(1): 20-25.
9. Hair JF, William B, Babin B, Anderson RE (2009)
Multivariate data analysis 7th (Edn.), Porto Alegre: Bookman, pp: 1-734.
10. Rigobello MCG, Carvalho REFL, Guerreiro JM, Motta
APG, Atila E, et al. (2017) The perception of the patient safety climate by professionals of the emergency department. Int Emerg Nurs 33: 1-6.
11. Zimmermann N, Küng K, Sereika SM, Engberg S,
Sexton B, et al. (2013) Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitals-a validation study. BMC Health Serv Res 13: 347.
12. Santiago TH, Turrini RN (2015) Organizational
culture and climate for patient safety in Intensive Care Units. Rev Esc Enferm USP 49: 123-130.
13. Najjar S, Hamdan M, Baillien E, Vleugels A, Euwema
M, et al. (2013) The Arabic version of the hospital survey on patient safety culture: a psychometric evaluation in a Palestinian sample. BMC Health Serv Res 13: 193.
14. Bodur S, Filiz E (2010) Validity and reliability of Turkish version of Hospital Survey on Patient Safety Culture and perception of patient safety in public hospitals in Turkey. BMC Health Services Research 10: 28.
15. Kings CT (2013) The safety culture of the patient:
validation of a measurement instrument for the Brazilian hospital context [thesis]. Rio de Janeiro (RJ): National School of Public Health.
16. Chen IC, Li HH (2010) Measuring patient safety culture in Taiwan using the Hospital Survey on Patient Safety Culture (HSOPSC). BMC Health Serv Res 10: 152.
17. Dolci GF, Suárez JR, Casanova HA, Fernández CR,
Torres FH, et al. (2010) Culture about patient safety in health professionals. Cir Cir 78(6): 527-532.
18. Brasaite I, Kaunonen M, Martinkenas A, Suominen T
(2016) Health care professionals' attitudes regarding patient safety: cross-sectional survey. BMC Research Notes 9: 177.
19. Nguyen G, Gambashidze N, Ilyas SA, Pascu D (2015)
Validation of the safety attitudes questionnaire (short form 2006) in Italian in hospitals in the northeast of Italy. BMC Health Services Research 15: 284.
20. Inoue T, Karima R, Harada K (2017) Bilateral effects of hospital patient-safety procedures on nurses' job satisfaction. Int Nurs Rev 64(3): 437-445.
21. Weng SJ, Kim SH, Wu CL (2017) Underlying influence of perception of management leadership on patient safety climate in healthcare organizations-A mediation analysis approach. Int J Qual Health Care 29(1): 111-116.
22. Carvalho PA, Göttems LB, Pires MR, de Oliveira ML
(2015) Safety culture in the operating room of a public hospital in the perception of healthcare professionals. Rev Lat Am Nursing 23(6): 1041-1048.
23. Aymen E, Akbarisari A, Rashidian A, Aljeesh Y,
Radwan M, et al. (2017) Psychometric Properties of an Arabic Safety Attitude Questionnaire. Oman Med J 32(2): 115-123.
24. Perneger TV, Staines A, Kundig F (2014) Internal
consistency, factor structure and construct validity of the French version of the Hospital Survey on Patient Safety Culture. BMJ Qual Saf 23(5): 389-397.
25. Etchegaray JM, Thomas E (2012) Comparing two safety culture surveys: Safety Attitudes Questionnaire and Hospital Survey on Patient Safety. BMJ Qual Saf 21(6): 490-498.
Figure 1
Characterization of the subjects of the research.
Presentation of mean scores and internal consistency of HSOPSC and SAQ domains.
Intraclass coefficient and ANOVA of the domains.
Correlation between SAQ and HSOP domains.
Regression analysis of predictive variables and outcome.
References
- Kohn LT, Corrigan JM, Donaldson MC (2000) To Err is Human: building a safer health system. Committee on Quality of Health Care, Institute of Medicine, National Academy Press Washington (DC).
- Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, et al. (2006) The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 6: 44.
- Carvalho REFL, Cassiani SHB (2012) Cross-cultural adaptation of the Safety Attitudes Questionnaire- Short Form 2006 for Brazil. Revista Latino-Americana de Enfermagem 20(3): 575-582.
- Reis CT, Martins M, Laguardia J (2012) Transcultural adaptation of the Brazilian version of the Hospital Survey on Patient Safety Culture: initial stage. Cad Saúde Pública, Rio de Janeiro 28(11): 2199-2210.
- Agency for Healthcare Research and Quality (2015) Hospital Survey on Patient Safety Culture: 2004. User comparative database Report, AHRQ.
- Sternick C (2017) Technical characteristics of a measuring instrument. Arquivos Brasileiros de Psicologia Aplicada 28 (1): 48-67.
- Carvalho RE, Arruda LP, Birth NK, Sampaio RL, Cavalcante ML, et al. (2017 ) Assessment of the safety culture in public hospitals in Brazil. Rev Lat Am Nursing 25: e2849.
- Menz HB, Latt MD, Tiedemann A, Mun San Kwan M, Lord SR (2004) Reliability of the GAITRite walkway system for the quantification of temporal spatial parameters of gait in young and older people. Gait Posture 20(1): 20-25.
- Hair JF, William B, Babin B, Anderson RE (2009) Multivariate data analysis 7th (Edn.), Porto Alegre: Bookman, pp: 1-734.
- Rigobello MCG, Carvalho REFL, Guerreiro JM, Motta APG, Atila E, et al. (2017) The perception of the patient safety climate by professionals of the emergency department. Int Emerg Nurs 33: 1-6.
- Zimmermann N, Küng K, Sereika SM, Engberg S, Sexton B, et al. (2013) Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitals-a validation study. BMC Health Serv Res 13: 347.
- Santiago TH, Turrini RN (2015) Organizational culture and climate for patient safety in Intensive Care Units. Rev Esc Enferm USP 49: 123-130.
- Najjar S, Hamdan M, Baillien E, Vleugels A, Euwema M, et al. (2013) The Arabic version of the hospital survey on patient safety culture: a psychometric evaluation in a Palestinian sample. BMC Health Serv Res 13: 193.
- Bodur S, Filiz E (2010) Validity and reliability of Turkish version of Hospital Survey on Patient Safety Culture and perception of patient safety in public hospitals in Turkey. BMC Health Services Research 10: 28.
- Kings CT (2013) The safety culture of the patient: validation of a measurement instrument for the Brazilian hospital context [thesis]. Rio de Janeiro (RJ): National School of Public Health.
- Chen IC, Li HH (2010) Measuring patient safety culture in Taiwan using the Hospital Survey on Patient Safety Culture (HSOPSC). BMC Health Serv Res 10: 152.
- Dolci GF, Suárez JR, Casanova HA, Fernández CR, Torres FH, et al. (2010) Culture about patient safety in health professionals. Cir Cir 78(6): 527-532.
- Brasaite I, Kaunonen M, Martinkenas A, Suominen T (2016) Health care professionals' attitudes regarding patient safety: cross-sectional survey. BMC Research Notes 9: 177.
- Nguyen G, Gambashidze N, Ilyas SA, Pascu D (2015) Validation of the safety attitudes questionnaire (short form 2006) in Italian in hospitals in the northeast of Italy. BMC Health Services Research 15: 284.
- Inoue T, Karima R, Harada K (2017) Bilateral effects of hospital patient-safety procedures on nurses' job satisfaction. Int Nurs Rev 64(3): 437-445.
- Weng SJ, Kim SH, Wu CL (2017) Underlying influence of perception of management leadership on patient safety climate in healthcare organizations-A mediation analysis approach. Int J Qual Health Care 29(1): 111-116.
- Carvalho PA, Göttems LB, Pires MR, de Oliveira ML (2015) Safety culture in the operating room of a public hospital in the perception of healthcare professionals. Rev Lat Am Nursing 23(6): 1041-1048.
- Aymen E, Akbarisari A, Rashidian A, Aljeesh Y, Radwan M, et al. (2017) Psychometric Properties of an Arabic Safety Attitude Questionnaire. Oman Med J 32(2): 115-123.
- Perneger TV, Staines A, Kundig F (2014) Internal consistency, factor structure and construct validity of the French version of the Hospital Survey on Patient Safety Culture. BMJ Qual Saf 23(5): 389-397.
- Etchegaray JM, Thomas E (2012) Comparing two safety culture surveys: Safety Attitudes Questionnaire and Hospital Survey on Patient Safety. BMJ Qual Saf 21(6): 490-498.