Abstract

Background: Ten years after implementation of WHO Surgical Safety Checklist (WHO SSC), used as a core item to assess hospital performance in China, an observational study was performed to evaluate level of clinical motivation for routine performance of the surgical safety checklist.

Methods: A survey was designed on the basis of three parts of the surgical safety checklist and forming 23 questions. It covers the three main processes of safety check: (1) Before anesthesia induction, (2) Before operation start and (3) Before patient leaving the operating room. From March to April 2019, the questionnaire was sent to the members of the Chinese-based online New Youth Anesthesia Forum through the WeChat® social media platform. Answers were completed by mobile phones or desktop computers. Each WeChat ID number allowed only one answer for each individual participant.

Results: A total of 3943 members read the questionnaire invitation, of which 2121 members participated to completion with an overall completion rate of 53.8%. 28% of participants were trainees (medical students and residents) while the remaining 72% were practicing physicians (40% attendings, 32% division chiefs). The vast majority (93%) of operative cases were preceded by WHO SSC prior to induction of anesthesia but had lower rate of use (78%) prior to start of surgery. There were also reported high rates of list item omissions at 36.4% during these steps. In particular, key information regarding the surgery communicated by the surgeon was routinely provided only 18% of the time and a similar 18% frequency by the anesthesiologist regarding patient-specific concerns prior to inducing anesthesia. Teams completed an omission-free end-of-case checklist with only 44% frequency. 56% of medical teams were described as careful when performing verification checks while the remaining 37% checked in hurry and 7% checked with insufficient understanding of patient’s condition. Even though respondents perceive surgeons as most familiar with the patient, surgeons led checklists at a frequency of 5.3%, deferring to anesthesiology 59% or OR nurse 36% of the time.

Conclusion: High rates of pre-anesthesia participation is followed by significant stepwise decrease in participation prior to start of surgery as well as prior to patient leaving the OR. High rates of list item omissions and lack of care during checklist performance was reported. This study identifies major gaps in proper implementation of the WHO SSC despite widespread adoption.

Key Points

Question: What is the general perception of WHO SSC performance in the OR after implementation according to the multidisciplinary OR staff?

Findings: Frequency of participation in WHO SSC is highest during the pre-anesthesia phase and a high rate of list item omissions were reported during all phases of SSC implementation.

Meaning: Major gaps in safe implementation of the WHO SSC persist despite widespread adoption due to lack of motivation by multidisciplinary staff which may be improved with increased SSC efficiency.

Glossary of Terms

WHO: World Health Organization; SSC: Surgical safety checklist

Introduction

The World Health Organization (WHO) initiated the second Global Patient Safety Challenge with the theme of Safe Surgery Saves lives in 2008, and launched the WHO Surgery Safety Checklist (WHO SSC) to the world [1,2]. We subsequently translated it into Chinese and published the full text in the Chinese Association of Anesthesiologists Newsletter (2008, Issue 12), and at the same time, we took the lead in launching the trial operation of the checklist in Peking Union Medical College Hospital [3]. Soon, WHO SSC received national attention. The Ministry of Health at the time officially issued a document in March 2010, requiring the promotion and implementation of the WHO SSC throughout the country [4]. In 2011, the WHO SSC was included in the Three-level General Hospital Evaluation Standards (2011 Edition) [5]. In November 2016, the Legal Affairs Department of the National Health and Family Planning Commission issued the Medical Quality Management Measures and once again listed the WHO SSC asone of the 18 core systems to ensure medical quality and patient safety [6].

So far, the WHO SSC has been introduced to China for 10 years, and it has been enforced as a core system. However, compulsory use does not mean correct use. Five years ago, we conducted an investigation on the implementation of surgical safety checklist at Peking Union Medical College Hospital, which was the first hospital to introduce and use SSC in China, and found that there were some problems with the use of the checklist, such as the incomplete verification, passing by, answering casually, and key surgeons not present for verification, etc. Some did not even implement verification, but just completed the checklist form [7,8]. Subsequently, the researchers extended the survey to many other domestic hospitals. The results showed thatthe average implementation rate of the three checks before the anesthesia, before the start of the surgery and before the patient leaves the room still has a lot of room for improvement; in some hospitals, the SSC process lacks a unified standard practice procedure, and there are phenomena of unclear responsibilities, formal verification, and ineffective daily supervision [9,10]. Therefore, our study intended to use the form of questionnaire survey to investigate the current status of the 10^th anniversary of the implementation of SSC in China.

Methods

A survey was designed to assess the quality of participation in routine SSC by members of the Chinese-based online New Youth Anesthesia Forum. The online survey was designed on the basis of three parts of the surgical safety checklist: (1) Before anesthesia induction, (2) Before skin incision and (3) Before patient leaves operating room. In aggregate, these sections comprised a 23 item single-answer multiple choice questionnaire without any open or free text answer options in order to reduce analysis time and maintain high response rates. From March to April 2019, the questionnaire was sent to the members of the Chinese-based online New Youth Anesthesia Forum through a proprietary social media platform (WeChat®, Tenecent Inc., Shenzhen, China) after the research protocol was approved by the New Youth Anesthesia Forum Research Committee and the requirement for written informed consent was waived by the committee. Answers were completed by either mobile phones or desktop computers. Each Wechat ID number allowed only one answer for each individual participant. The electronic method of survey distribution allowed for a larger target population, quick response time and data compilation as well as higher predicted response rate. No monetary or non-monetary incentives were used for participation. For data analysis, no tests of significance were required and all results were reported in a manner of distribution. Targeted survey population included respondents self-described as (1) Anesthesiologist, (2) Certified Registered Nurse Anesthetist (CRNA), (3) Surgeon or (4) Operation Room Nurse at various professional levels including medical students, residents, attending and chief-level participants. This report follows the Standards for Quality Improvement Reporting Excellence (SQUIRE) 2.0 guidelines revised September 15^th, 2015 [11].

All analyses were performed using Microsoft Excel (version 2019, Microsoft©). Categorical variables were given as a number and percentage. Categorical variables were compared using the chi square test. The differences were considered to be significant for values of P < 0.05.

Results

A total of 3943 members read the questionnaire invitation, 2121 of which participated in the survey to completion with an overall completion rate of 53.8%. 28.1% of participants marked themselves as trainees (either students or residents) while the remaining 71.9% were practicing physicians or nurses (39.6% attendings, 32.3% chiefs). Self-reported professional distributions of participants were as follows: 74.2% anesthesiologists, 16.7% OR nursing staff, 5.5% surgeons and 3.6% nurse anesthetists (Figure 1). Overall, 93.4% of reported cases were preceded by WHO-structured SSC prior to induction of anesthesia according to survey participants. During the pre-anesthesia phase, surgery team participation in SSC occurred at a frequency of 83%, which included various members of the surgery team including assistant surgeon or other surgeons not directly involved in the operation or patient’s care. The operating surgeon was reported to participate 10.7% of the time during this phase. When prompted, 81.5% of study participants stated the surgery team should be involved in the pre-anesthesia SSC because they know the patient best. Study participants also reported high rate of list item omissions during pre-anesthesia SSC implementation, reporting an omission frequency of 39.0% or absence of pre-anesthesia SSC phase altogether at 9.1% with remaining 51.9% of participants endorsing careful verification of SSC items (Figure 2).

Figure 1: Self-reported professional distributions of participants.

Figure 2: List item omissions during pre-anesthesia SSC implementation.

There was a significant difference between participation in the pre-operative SSC before incision and before anesthesia induction (P < 0.05) (Figure 3). Omissions of the checklist items during pre-operative SSC occurred at a frequency similar to the pre-anesthesia phase: 39.0% versus before incision 36.4%, which was not a significant difference.

Figure 3: Participation in the pre-operative SSC before incision and before anesthesia induction (P < 0.05).

Critical information was often omitted during both phases of SSC: Key information regarding the surgery communicated by the surgery team was routinely provided only 18.2% of the time during the surgical time out and a similar frequency of 18.8% by the anesthesia team regarding patient-specific concerns during the anesthesia time out was routinely performed. The survey also assessed for critical information regarding material preparation, disinfection and preparedness of instrumentation by the nursing staff which was routinely performed at a frequency of 64%. However, this step was not a part of hospital policy at 10.4% of participant home institutions. Identification of intraoperative antibiotics occurred with 73.5% frequency during SSC where 5.4% of hospitals did not institute this step suggesting that the remaining 21.1% of participants do not discuss intraoperative antibiotic use during SSC despite institutional policy.

Omission-free end-of-case checklists were performed with a frequency of 44%, if performed at all. This step was not part of institutional policy according to 18.9% of participants. The remaining 36.9% of participants reported routine performance of end-of-case SSC which included list-item omissions.

When assessing for level of motivation during overall checklist participation, 37.2% of participants describe OR team attitude as "checked in a hurry" and 6.6% "checked with insufficient understanding of patient’s condition". The remaining 56.2% of participants describe team participation as "careful when performing verification checks". Even though respondents perceive surgeons as most familiar with the patient, surgeons led checklists at a frequency of 5.3%, deferring to anesthesiology 59% or OR nurse 36% of the time. Participants claimed the main difficulty in performing SSC was due to time-consuming and laborious nature of task (46.4%). Other reasons included participants did not know the checklist steps (15.2%), checklist was not suitable for respective procedure (14.0%), verification perceived as having little significance (7.8%) or other unspecified reasons (16.8%) (Figure 4). Overall, the anesthesia team was reported to lead the SSC with highest frequency at 58.9% followed by the OR nurse staff at 35.7% and surgery team at 5.3% (Figure 5). 39.8% of respondents believed the surgery team would be the best leader during SSC implementation whereas 45.0% thought the anesthesia team is best leader for SSC implementation, followed by OR nursing team at 15.2%. When asked "if you or your family were going to undergo surgery, would you like your surgical team to perform SSC?" the overwhelming majority of respondents said yes (98.3%). 94.3% of participants believed the practice of SSC does reduce complications related to the surgery and administration of anesthesia and improves overall patient safety.

Figure 4: Assessing for level of motivation during overall checklist participation.

Figure 5: Frequency of the teams that lead the SSC.

Discussion

Despite a growing volume of evidence associating improved outcomes with WHO SSC implementation, paucity of data remains regarding the quality of SSC implementation and may present as a potential institutional quality improvement initiative. Our survey displayed the current status of the ten year SSC implementation in china and the primary descriptive data was already partially published [12]. After statistical analysis we identified a number of areas of potential improvement when performing the WHO SSC. We found that despite 93.4% pre-anesthesia SSC participation, participation significantly declined to 78.2% for the pre-operative SSC and 80.9% for end-of-case SSC. According to participants, the decreased participation in SSC was attributed to time consuming and laborious nature of the task, unfamiliarity with the task, checklist format being impractical for respective operations or verification perceived as having little significance. These perceptions differ from previously discussed barriers in the literature including unfamiliarity and embarrassment, hierarchy in the operating theatre creating difficulties for nursing staff when leading, timing of checks, duplication of checks and irrelevant checklist items [13] thereby identifying new pitfalls to modern SSC implementation that can be addressed at the institutional level.

Level of quality in SSC implementation remains a source of improvement. One multicenter prospective trial conducted at 5 English hospitals evaluated level of completeness when performing checklist items during "time out" and "sign out" tasks and found absent team members in 40% of cases and failure to focus on checks with > 70% frequency as well as complete omission of sign out step in 39% of cases. They also concluded checklist performance was best when led by the surgery team [14]. Another UK study identified barriers to optimal implementation: Unfamiliarity and embarrassment, hierarchy in the operating theatre creating difficulties for nursing staff when leading, timing of checks, duplication of checks and irrelevant checklist items [13]. Further investigations identified other barriers including participation of only a few members of the surgical team, restricted communication due to time constraints and hierarchical culture inhibiting open line of communication between surgeons and anesthetists and eventually OR staff normalized performance of abbreviated time out procedures [15]. A Swiss study demonstrated poor adherence to checklist items where only 13% of time outs and 3% of sign outs were properly checked [16].

Even when the SSC was performed, it was not without error. When performing the pre-anesthesia, pre-operative and end-of-case SSC, participants reported list item omission frequencies occurring at 36.4%, 39.0% and 36.9% respectively. Such errors included critical omissions such as key information regarding the surgery provided by the surgery team as well as key patient-specific concerns prior to induction of anesthesia in addition to information regarding material preparation, disinfection and instrument preparedness. This implies all three professions (surgery, anesthesia and nursing) contribute to lapses in SSC performance. Furthermore, excluding intraoperative antibiotics form the checklist occurred at a high frequency of 26.5%. This implies that antibiotics were administered at the risk of being incorrect dose, class, route or not administered at all for over 25% of all cases and may be a source of improving surgical site infection rates with improved SSC implementation. The OR team should be distinctly aware of the disadvantages of a poorly executed checklist. Misuse of checklist can lead to a false sense of security [17]. For example a non-participating surgeon may falsely assure the information is correct to OR staff and thus forego additional safety check measures in moments of uncertainty. Interestingly, survey participants felt the surgery team was most suitable for leading SSC as they were assumed to be most familiar with the patient, a sentiment corresponding with previous surveys [15]. Despite this sentiment, we have found that the level of leadership by the surgery team was limited, occurring at a frequency of 5.3% overall. This may present as a second major source of improved SSC performance as well as postoperative outcomes. However, this assumption is based on the subjective nature of perceived role of surgery team within the overall patient care process as they are intimately involved in the preoperative, intraoperative and postoperative phases of care.

Some authors have developed factors of successful checklist implementation: Adequate training and learning material, enthusiasm from leadership, cultivate local champions, clarify role of each professional group, enact regular internal audits, incorporate measurements of effectiveness and support local adaptations but discourage over simplification [15]. Combining the data from this survey, strengthening the education and training of surgical safety checklist, and increasing the understanding of the verification system, may help increase compliance with checklist implementation.

There are several limitations of the study. This sample population was selected with the goal of capturing experiences from each member of the multidisciplinary OR team in attempt to generalize SSC participation among institutional affiliates. By definition, this is a cross-sectional study as the samples were drawn from the relevant population and studied in a single unit of time for each participant. As such, this study describes only the characteristics of the cohort at one moment in time due to lack of ability to analyze beyond correlations. Moreover, because the survey targeted members of an anesthesia-specific social media platform, there is an unquantifiable degree of selection bias present in our results. Recall bias may also contribute to errors in our study. However, we assume the participants are frequently involved in OR SSC implementation such that the time interval between OR and survey participation remains small enough to mitigate degree of recall bias.

Conclusion

High rates of pre-anesthesia participation are followed by significant decrease in SSC participation during both pre-operative and end-of-case SSC. A high rate of list item omissions and lack of care during checklist performance was reported despite widespread adoption of the WHO SSC as standard of care among participants. This observational study identifies major gaps in proper implementation of the WHO SSC and has identified weaknesses which can be addressed at the intuitional level with the objective of continued improvement in perioperative patient safety.

Financial Disclosures

None.

Conflicts of Interest

None.

Author Contributions

Bin Zhu helped analyze the data, prepare the manuscript and revise the manuscript.

Benjamin A Eslahpazir helped analyze the data, prepare the manuscript and revise the manuscript. This author is co-First Author.

Huan Gao helped design the study and participate in data collection.

Xiangyong Zhou helped design the study and participate in data collection.

Yu Liu helped design the study and participate in data collection.

Yuguang Huang helped design the study and participate in data collection.

Jeffrey J. Huang helped design the study, supervise the data collection and revise the manuscript.

References

1. WHO Patient Safety & World Health Organization. (‎2009)‎. WHO guidelines for safe surgery: 2009 :safe surgery saves lives. World Health Organization. (https://apps.who.int/iris/handle/10665/44185).
2. Haynes A, Weiser T, Berry W, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360(5):491-499.
3. Zhu B, Huang Y. Implementation and application analysis of the surgical safety checklist. Chinese Journal of Hospital Management, 2012,32(4):34-35.
4. Notice of the General Office of the Ministry of Health on issuing the surgical safety checklist system. (http://www.nhc.gov.cn/wjw/gfxwj/201304/f95253fa25c14d339ed99ef75f5c2b17.shtml) [Accessed 27 December 2020].
5. Notice of the Ministry of Health on Printing and Distributing the Evaluation Standards for Tertiary General Hospitals (2011 Edition). (http://www.nhc.gov.cn/wjw/gfxwj/201304/b98329ec713a4e8d812b23a56d13f94f.shtml) [Accessed 27 December 2020].
6. Medical Quality Management Measures. (http://www.nhc.gov.cn/cms-search/xxgk/getManuscriptXxgk.htm?id=ae125f28eef24ca7aac57c8ec530c6d2) [Accessed 27 December 2020].
7. Ma S, Zhu B, Huang Y. Investigation and analysis of the surgical safety checklist in our hospital. Chinese Journal of Hospital Management, 2013,33(9): 43-44.
8. Ma S, Huang Y, Yu X, et al. Implementation and promotion of surgical safety checklist. Modern Hospital Management, 2015,13(1): 6-8.
9. Yu X, Huang Y, Guo Q, et al. Clinical motivation and the surgical safety checklist. BJS 2017; 104(1): 472-479.
10. Wang D, Li X, Zhang H, et al. Investigation on the current situation of operation safety inspection management in tertiary hospitals in Beijing. Chinese Journal of Hospital Management, 2017,37(7):43-44.
11. Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016, 25(12): 986-992.
12. Zhu B, Gao H, Zhou X, et al. A survey of the ten-year implementation of WHO surgical checklist in China. Chin J Anesthesiology, 2019，39(9):1041-1046.
13. Vats A, Vincent C, Nagpal K, et al. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ. 2010;340:b5433.
14. Russ S, Rout S, Caris J, et al. Measuring variation in use of the WHO surgical safety checklist in the operating room: a multicenter prospective cross-sectional study. J Am Coll Surg. 2015;220(1):1-11.
15. Braaf S, Manias E, Riley R. The 'time-out' procedure: an institutional ethnography of how it is conducted in actual clinical practice. BMJ Qual Saf. 2013;22(8):647-55.
16. Cullati S, Le Du S, Raë A, et al. Is the Surgical Safety Checklist successfully conducted? An observational study of social interactions in the operating rooms of a tertiary hospital. BMJ Qual Saf. 2013;22(8):639-46.
17. Degani A, Wiener E. Cockpit Checklists: Concepts, Design, and Use. Human Factors. 1993;35(2):345-359. (https://journals.sagepub.com/doi/pdf/10.1177/001872089303500209#articleCitationDownloadContainer)