The smaccGOLD debate ‘should real airway experts use checklists‘ in March 2014 between RFDS powerhouse Dr Minh le Cong (against) and myself (in favour) started some nine months prior to the actual event, with an almost daily exchange of trash talk on the topic via the wonderful medium of Twitter.
Twitter trash talk between Minh & myself got heated on occasions. Not quite Bruce Lee vs Chuck Norrris, but close…
You can download a PDF of my argument PRO checklists here.
It’s no secret that I am a fan of checklists and cognitive aids – not just for ‘occasional intubators’ but across the board, even for those with significant expertise. Minh on the other hand was determined to argue the counterpoint and expose some of the flaws and fallacies surrounding checklist use.
“quality care, out there”
The debate takes place at smaccGOLD – a critical care conference open to intensivists, emergency physicians, rural clinicians, paramedics, nurses, students – indeed, anyone interested in the delivery of “quality care, out there”. With this maxim in mind, we are all looking for things that will make a difference during critical procedures – whether this be relatively new concepts like use of apnoeic diffusion oxygenation in RSI, developing robust airway equipment and algorithms for airway management…I believe that use of checklists in a crisis helps deliver quality care.
There are arguments for and against checklists in medicine sensu lato, not just airway management. This post is a summary of checklists relevant to their use wherever airways are managed (hence not just Operating Theatre, but also to ICU, ED and prehospital). Checklists aren’t just for routine – they should be used in crisis management – whether this is emergency RSI, dealing with unexpected hypoxia, a rare emergency such as MH or similar.
The main benefits for use of a checklist regardless of experience include:
Reduced task fixation & improve situational awareness
Reduced cognitive overload
Reduced stress
Defined team member roles
Improved team function
Improved team communication
Team understanding of potential complications and what they will be expected to do
Use can make crisis management & high-stakes procedures such as RSI smoother and quicker(when well practiced at using the checklist)
The argument is that in sick patients, unfamiliar environments, etc we are very prone to task fixation and loss of situational awareness. The adrenaline rush pushes us to get on with it, when often time is not actually so critical that we can’t pause for 30-60 seconds and have a quick brief.
Minh however doggedly argues the opposite, pointing out that checklists per se do not bring about quality – it requires team buy in and awareness of their limitations. Using a checklist as a ‘tick and flick’ exercise, or inappropriate use of a checklist are all real caveats.
So what actually IS a checklist?
Clinicians have been using checklists since medical school – we are all familiar with the usual deluge of mnemonics , acronyms, algorithms and assorted prompts. Newer formats such as The Vortex are being used increasingly in crisis management, not as a ‘recipe’ but as cognitive aids in a crisis.
Some scorn checklists and other cognitive aids as ‘a crutch for the novice’, ‘cookbook medicine’ or as nothing other than ‘cheat sheets for novices’. Perjorative terms, often used by experts. After all, the expert doesn’t need help…or do they?
Humans make mistakes, especially when tired, when distracted or interrupted…or in a crisis when under cognitive load. Moreover, the expert does not function as an individual – he or she is part of a team, and members may have varying levels of expertise. We know this from the extensive human factors work, championed in healthcare (and anaesthesia has lead the way) by people such as David Gaba or Martin Bromiley.
In the true sense of the word, a checklist is exactly that – a list against which a check (or tick) is placed, to confirm an action has been completed. However it is not just learners who benefit – the true power of a checklist is to establish routine and so allow cognitive offloading by experts in an emergency, as well as to strengthen team performance and establish shared mental models.
“effective CLs are more about helping reduce cognitive overload and error reduction, not purely about the manual procedure”
@HawkMoonHEMS Dr Brian Burns, FACEM & Retrieval Physician
One of the best forms of checklist is a two-person challenge-response format. This obviates the temptation to ‘skim’ items on a checklist and places responsibility for reading the items with the assistant, not the operator. I think that this sort of challenge-response checklist is the most useful in crisis management – but in order to be effective, must be embedded into routine practice and adopted by teams with institutional support.
“a check of what has been DONE by experts, not a HOW TO for novices”
Delving a little deeper, the literature describes FOUR main types of checklist (Winters et al, 2009) which differ in terms of number of operators and the extent to which information is verified :
static parallel : usually completed by a single operator and executed in the form of a series of ‘read-do’ tasks. The anaesthetic machine check is a good example, as is restocking of equipment
static sequential with verification : this involves a challenge-response, with one operator reading a series of items for verification of completion or normality by another. The central line checklist in ICU is such an example as is an RSI challenge-response checklist
static sequential with verification and confirmation : these are used more in team-based settings, with sets of tasks completed by different team members. A designated person reads the items (challenge) and each responsible party verifies the completion of a specific task. The WHO Surgical Checklist is such an example, with separate activities for scrub, surgical and anaesthetic teams
dynamic : these typically use a flowchart to guide complex decision-making. There may be multiple options to choose from and the team must decide optimal course – in essence, an algorithm. The UK’s Difficult Airway Society algorithms & Australian Resuscitation Council ACLS algorithms are such examples.
Of course it’s not just in medicine that we use a checklist – we write a shopping list before visiting the supermarket and most of us will perform some sort of mental checklist before leaving for work (have I got my car keys? My wallet? My sunglasses? Is today even a work day?). Humans use cognitive aids most days. Why would we abandon them at work?
That’s not to say that one needs a checklist for everything. For some reason, Minh seemed obsessed with the idea of using a checklist before intercourse. Whilst this is clearly a ‘tightly coupled’ procedure, I think Minh was missing the point. To paraphrase Weingart, sex – like resus – is one of those times when “slow is smooth, and smooth is good”.
Well then, what is a REAL airway expert?
Yeah, yeah – you want me to say that it’s an anaesthetist. Except of course it’s not – airways are also managed outside of the elective operating theatre setting by non-anaesthetists. Real airway doctors can be found in the Emergency Department, in the Intensive Care Unit, in the Pre-Hospital & Retrieval environment – as well as in the operating theatre. It goes without saying that even an experienced anaesthetist, with decades of training and experience in routine anaesthesia, may struggle when faced with a soiled airway at the roadside, with unfamiliar kit and a lack of trained assistance. Expertise in one area, even airway management in the OT, may not translate into another arena such as the ED.
“Smoke me a kipper, I’ll be back by breakfast”
- the idea of the expert as an individual ‘ace’ is obsolete
So ‘real’ airway experts are drawn from the ranks of anaesthetists, emergency physicians, intensivists, & rural doctors. Heck, they may not be doctors – we have excellent intensive care paramedics working in various systems. Ambulance Victoria relies on MICA paramedics to deliver prehospital RSI; the South Australian medSTAR service has retrieval practitioners, drawn from ranks of doctors, nurses, paramedics. The skill set defines the job, not the initial qualification. And all are striving for mastery, whatever our heritage…
Medicine is truly an apprenticeship. We move through the classic stages of skills-acquisition as described by Dreyfus & Dreyfus (2005), each seeking to achieve mastery in our field.
Classic Model of Skills Acquisition (Dreyfus & Dreyfus, 2005)
Not everyone transitions to expert stage; many are doomed to confuse proficiency with expertise; few reach true mastery.
The concept of “it takes 10,000 hours of training to be an expert” from Ericsson is often banded about – with the generalisation that it takes 10 years or so to accrue this experience. Sadly many do not reach the desired level – many repeat one year of experience ten times, rather than deliberately train and gain mastery. Moreover, for the ‘expert’ in emergency medicine or intensive care, airway management makes up just one component of practice. Does expertise in one area of medicine necessarily translate to expertise in procedures such as intubation? Even once ‘expert’ status is achieved, there is a real risk of ‘skill fade’ – a recent observational study of endotracheal intubation in a tertiary ED suggested that emergency physicians may only perform this skill as little as three times per week, due to requirements to supervise trainees and allow them to acquire skills (Fogg et al 2012).
“Half of all experts are below average”
Even within the ranks of anaesthetists, performance will follow a Gaussian distribution, with half ‘below average’ when compared to peers. Expertise in the calm ordered environment of the Operating Theatre may not translate to the chaotic environment of an emergency airway in unfamiliar settings.
Truly expertise is a relative concept.
Hang on, this is all getting a bit Zen isn’t it?
Well, possibly. The path to mastery is something we all strive for – but as time goes on, many clinicians become comfortable at the ‘competent-proficient’ stage. They are considered technically capable, they are able to make a (good)-living doing what they have trained to do. To strive for expertise and mastery requires a lot more effort for seemingly less reward.
The danger is that once you are at the ‘proficient’ stage, you may feel that there is no need for cognitive aids. You are on top of things at work, you manage things adroitly…and then you are thrown a curve ball. An unexpected crisis. Something you have read about, perhaps even drilled for, in the comfort of a sim lab or one of the many continuing-professional development courses…but sadly your proficiency in everyday practice does not translate into expertise in a crisis. How can it? True crises are relatively infrequent. How many have managed malignant hyperthermia? A true CICO requiring surgical airway? Intraoperative anaesthetic machine failure?
The anaesthetic error in the sad case of Elaine Bromiley is but one example of this. Here experts in their field were overwhelmed by the situation (CICO) and failed to make appropriate decisions despite having the requisite skills. If you aren’t familiar with this story, take time now to review the video :
Martin Bromiley, Airline Pilot & Human Factors Expert
There are many more examples of error leading to catastrophe in airway management, despite the presence of experts. Hence there has been a huge emphasis on training in human factors for crisis management in medicine – mostly lead by anaesthetists but seeping across into other clinical disciplines as the years go on. We can all learn from this.
Error in Medicine
We need to be very clear – error in medicine occurs despite the presence of experts & their expertise.
If you read nothing else, read McIlvaine (2006) on ‘Human error and its impact on anesthesiology’ – a gentle introduction to understanding how we err and use of tools to mitigate against the inevitable.
Catastrophe can occur because of a major error – failing to check the correct site for surgery; accidentally injecting a cytotoxic intrathecally; awareness under anaesthesia due to forgetting to replenish the vaporiser; failure to monitor ETCO2 in the ICU to confirm ongoing ETT placement.
Harm can happen because of equipment failure or because of a seemingly trivial mistake – forgetting to have suction available during RSI, or not having a syringe available to inflate ETT pilot cuff. Not having drugs available in a crisis. Assuming that assistant is familiar with contents of difficult airway trolley.
These are all anathema to experts – who would be so foolish as to perform RSI without suction? Without checking the ET cuff? With an assistant who is not equally ‘expert’? Yet in analysis of medical procedures, mistakes & omissions are noted in a similar percentage as in aviation – 75%. Thankfully humans are adaptable and able to compensate; for most cases (eg: RSI in the OT for an elective, fasted patient) no harm will occur. Similarly minor mistakes may not amount to much, given levels of redundancy. But for the trauma patient with critical hypoxaemia, shock, MILS limiting optimal positioning and in the presence of a soiled airway, such omissions are less-forgiving.
“It may take many minor “holes in the swiss cheese” to line up to cause catastrophe. If only 9/10 holes line up, the operator will end the day in blissful ignorance of the near disaster as if 0/10 errors occurred. It’s the final hole that gets you”
But hang on – surely the expert is immune to error? I’ve already alluded to the problems of ‘expert’ status – it’s a relative concept, subject to problems of skill fade, of poor translation between areas of expertise or in dealing with relatively infrequent events…and of variation within a cohort.
OK, perhaps checklists are OK for routine procedures
- but surely there is no time to use one in a crisis?
The literature is fairly robust on use of checklists for routine procedures – the anaesthetic machine check, the WHO surgical checklist, central line checklists etc. But what about in a crisis? Surely we can rely upon the expert to manage the situation? Sadly not. Human factors research shows that humans can only manage to retrieve seven +/- 2 pieces of information from our memory with relative accuracy. During a complex procedure, with additional elements of stress or fatigue, memory becomes increasingly unreliable. Worse still, as the number of tasks we simultaneously manage exceeds three, we show significant decline in the accuracy and speed of handling problems. Even worse still, under stress we show increased skill degradation. Listen to Cliff Reid and Scott Weingart talk about the impact of stress and the concept of stress inoculation.
A checklist can help compensate for this, allowing “cognitive offloading” and empowering others (the non-expert team member) to ensure that procedures are followed.
Minimising error is especially important in ‘tightly-coupled’ processes – technical procedures where omission of a single step can lead to failure. Airway management is one example – one cannot afford to be distracted during RSI by another task, as the patient will be at risk. Conversely interruption during a pre-anaesthetic consultation is less serious – and may even be welcomed.
Certain procedures – such as RSI – are ‘tightly-coupled’ with little margin for error
Rather than expect perfection in the human operators, we need to make sure there are effective systems in place – acknowledging that error is inevitable in any system. A comparison is usually made between medicine and other fields – most notably aviation and the nuclear industry. Characteristics of these industries is the requirement for high degrees of safety despite the technically complex nature of tasks. Similar concepts abound in the manufacturing industry, where efforts are made to minimise variability and defects in products – the ‘Sigma Six’ process aims to shift defects to six standard deviations from mean – or 3.4 errors/million (99.99966% error free).
I don’t know if achieving a 0.00033% error rate in medicine is achievable – but there is no doubt that conscientious clinicians will strive for perfection – whilst acknowledging that no system involving humans is perfect. Most importantly, even experts will make mistakes – and expertise in one situation does not translate to another. Put simply – an experienced and expert anaesthetist will encounter catastrophe relatively infrequently – hence their expertise in ‘routine’ airway management may not translate to the same in a crisis. How could it? It is difficult for to gain expertise in infrequent events.
Anaesthesia & Aviation – caution with analogy
Human factors experts talk about creating opening up of communication, of team training and use of cognitive aids to minimise error. One of my bugbears is that anaesthesia is often compared to aviation. True, pilots are highly skilled professionals and use checklists bin acknowledgment of the fact that mistakes can happen even when performing routine tasks due to inherent complexity (read about the origin of checklists in aviation here). I think most of us would hesitate to get onboard an aircraft if we knew that the pilot had omitted to use a pre-flight checklist and instead relied upon ‘experiential expertise’ as sole safety check.
The standard argument is that ‘pilots use checklists’ and so should we
Some argue that in a crisis there is not time to use a checklist – the landing of US Flight 1549 on the Hudson River by Captain Cheslea ‘Sully’ Sullenberger after an unprecedented complete engine failure due to birdstrike immediately after takeoff is a case in point. Nevertheless, the pilots used their checklists even though they KNEW there was insufficient time to complete it – falling back on routine allowed them to rapidly confirm crucial procedures performed and come up with a novel solution to the problem. This is expertise – high-level problem solving in a crisis – and it relies upon the routine use of procedures such as checklists to help cognitively offload and so allow such improvisation.
“standardise until you absolutely have to improvise”
Dr Kevin Fong in “How to Avoid Mistakes in Surgery” – BBC Horizon
Of course anaesthesia is nothing like aviation. For a start, aircraft are designed to fly. And it’s not the norm for an aircraft to be on fire and hurtling towards the ground, whilst the Chief Engineer attempts to repair holes in the fuel pump – which is the analogy of a patient in haemorrhagic shock, needing an RSI for theatre as the surgeons attempt to ‘find the bleeding, stop the bleeding’.
There is a classic article written by Grant Hutchinson in 1998, originally published in ‘Today’s Anaesthetist’ and reproduced via this link. It is well-worth a read if you had any doubts about comparisons between aviation & anaesthesia.
As time moves on, lessons learned from aviation are being supplanted by those learned from the human factors industry and within anaesthesia itself. Training in “crisis resource management” has been practiced since 1990 (Gaba, 2010) – and it is not always appropriate to seek answers from the aviation field, when healthcare differs fundamentally.
If you really want to piss off a surgeon, remind them :
“anaesthetists fly the plane …
…whilst surgeons serve the drinks & do the in-flight entertainment“
Not that lessons from aviation are all bad – emphasis on human factors and team training is generally considered a good thing in clinical education via crisis simulation. But of course the clinical team is NOTHING like the highly-trained team on an aircraft or in a nuclear power plant. Rather the ‘team’ in an airway crisis in the ED is often comprised of disparate individuals who may have unknown skills mix and have probably never trained before (the exception of course being small rural EDs and of course highly-trained prehospital teams who ‘train hard, fight easy’). These are best considered as ‘flash teams’ – and it is obvious that although ostensibly a ‘team’ the rate of error in achieving a highly technical task amongst a ‘flash team’ is higher than in a team that has trained together.
Tannenbaum et al. describe the concept of “flash teams”
One of the ways to help mitigate against the vagaries of performance within a ‘flash team’ is to incorporate protocols that allow teams to form quickly – rapid identification of role clarity, use of a ‘shared mental model’ (the C-ABC approach). Checklists during critical procedures such as RSI can be considered as ‘join-in-progress’ protocols that allow individuals to function as a team – independent of the operator’s expertise.
Thus error is seen as inevitable in any system and may occur independently of expertise. Changes in systems, in addition to individual expertise, is the way forward to minimise error and reduce patient harm, particularly in technical & tightly-coupled processes. Cognitive aids (such as The Vortex) and use of checklists for critical procedures (surgery, RSI) have much to offer – even to experts.
Checklists in Medicine
Unless you have been living under a rock, it is highly likely that you have at least heard about Atul Gawande and his book ‘The Checklist Manifesto’. With any luck you will have read this and his other books – ‘Complications’ and ‘Better’. Gawande writes eloquently on the path to expertise, the need to see improvements in quality and the problems that clinicians face when things do not go wrong. He articulates clearly the difficulties of tightly-coupled processes in clinical practice, along with the inherent instability in dynamic systems such as the human body and disease.
The three books read as a series : ’Complications‘ (published 2002) describes the problems we face in medicine and difficulties to achieve excellence. ’Better‘ (published 2007) describes the qualities for success in medicine – diligence, doing the ‘right thing’ and ingenuity’. ’The Checklist Manifesto‘ (published 2009) describes the importance of pre-planning and how simple interventions at an organisational level may help overcome predictable error.
Gawande’s work has lead to the widespread implementation of the WHO Surgical Checklist, reported to reduce surgical error (such as wrong-site surgery or retained instruments) by as much as 30% – see safesurg.org for more details.
There have been criticisms of the WHO Checklist – there are reports in the UK’s NHS of the checklist being introduced ‘top down’ with no local change champion, resulting in lack of institutional ‘buy in’ from team members. I have certainly witnessed the checklist completed as an aside AFTER induction – or the checklist being performed by part of the team (typically scrub & scout nurse) without the participation of either surgeon or members of the anaesthetic team. Moreover they can fail – as discussed by active twitter-contributer Dr Helgi Johanssen (@traumagasdoc) in this podcast.
WHO Checklist MUST take place without interruption, involve all team members and take place before the procedure – not aside as ‘additional paperwork’
The WHO Surgical Checklist – widely adopted – is designed to be adapted to local circumstances – this is the Aus/NZ version
There have been other successes in checklist implementation – Provonost describes reductions in complications from central line insertion by introduction of a simple checklist to reduce central-line associated infection.
A recent (this week) paper in the NEJM describes NO improvement in outcomes when comparing checklist use before/after implementation (Urbach et al “Implementation of surgical checklists in Ontario, Canada” N Engl J Med 2014 370 : 1029-38). However, an accompanying editorial points out that the checklist was NOT USED in many cases, and cites other examples where checklist implementation takes time – it is probably that benefits will be proven with a longer period of study and ensuring that the checklist is used (Leape, L. 2014 The Checklist Conundrum NEJM 2014). As Gawande says “the checklist won’t work if you don’t use it”
The question is as to whether use of checklists is really needed by experts dealing with complex, non-routine events such as airway crisis management.
The smaccGOLD debate :
‘Real Airway Experts Use Checklists’
Given the proven value of checklists in routine and yet tightly-coupled procedures such as surgery and central line insertion, should their use become standard in airway management, even by experts?
Arguments in Favour
Expertise is a hard area to define. Whilst the assumption that Fellowship or Board Certification in Anaesthesia means that one is an ‘expert’, even anaesthesia is a broad church. The wise anaesthetist knows that he/she may have strengths as well as weaknesses – and whilst airway management is a core skill, during times of cognitive overload even the most seasoned veteran may struggle. Moreover, the management of an airway procedure (let’s say RSI) or of a crisis (let’s say intra-operative hypotension) requires more than just the individual’s skills – success or failure requires a complex interplay of expertise, coupled with teamwork and equipment. Even the best airway doctor can fail if his/her assistant is unable to assist or if a vital piece of equipment is lacking. These are not theoretical concerns. In one study, anaesthetists were reported to make mistakes in nearly 50 % of mistakes even during a ‘core skills’ such as RSI – thankfully suck mistakes are small and usually compensated for by redundancy in systems and the fact that error (eg: failing to check suction available) may be undetected in a routine RSI that goes well.
First up, who is an expert? Monash anaesthetist Nicholas Chrimes suggested via Twitter that we consider this to be “anyone qualified to independently perform advanced airway management, regardless of specialty” – this including anaesthetists, intensivists, emergency physicians, pre-hospitalists and paramedics – even those country doctors. At 3am in an ED, the only available ‘expert’ for a crisis is going to be one of the ED/ICU/anaes registrars…not the anaesthetic consultant. Ditto in a small rural town, the country doctor or paramedic will be the designated ‘expert’.
(i) being expert is not enough
Regardless of the ‘label’, assuming the operator is qualified to independently practice, I argue that expert status is insufficient. Understanding dual-process decision-making shows us that experts tend to practice via so-called “system I” – experience and intuition, whereas novices tend to use more “system II” – rule-based heuristics. This can lead to the phenomenon of ‘strong but wrong’ decisions…countered by introducing cognitive checkpoints – such as checklists.
Experts tend to operate in routine, borne by years of experience. I interviewed Dr Helgi Johannsen (@traumagasdoc) for a podcast on wrong-site surgery (an example where use of the WHO checklist failed) and this naturally lead to a discussion of checklists in crisis management and high-stakes procedures such as RSI. Helgi made it clear that he does NOT use a checklist – as he operates in a theatre environment with ‘one set way’. Even if moving into a different environment (such as the ED) for an RSI, Helgi admits he would bring his ODP (a trained theatre assistant) with him from the OT to that environment.
Yet not all of us have that luxury. Moving from a routine, practiced over years of experience, to an unfamiliar environment, with new personnel, probable changes in equipment and a dynamic situation (invariably a crisis – hence the need for expertise) places extra demands, such as stress & cognitive overload, unclear authority gradients, complex human factors interplay. As cognitive demand increases, performance decreases with concomitant loss of situational awareness, task-fixation etc – the perfect storm for error. Add in the common strains of working in a medical environment (often being hungry, angry, late or tired) and error is almost inevitable. That’s just part of being human…
Checklists can help – not as a recipe or cookbook for novices, but as an adjunct to the expert to ensure key stages or information have not been omitted. They function as a check of what has been done, not a ‘how to do it’.
Being expert is NOT enough! Using a checklist is an additional line of defence against latent error.
(ii) a “team of experts” is not an “expert team”
Even assuming our expert is in tip-top form, able to superhumanly overcome the demands of cognitive load under stress (perhaps through stress-inoculation training)…he or she is only as good as the team around them.
Checklists help to democratise knowledge and establish a shared mental model of what is going on. The pre-RSI checklist is a good example – taking 60 seconds to run through the checklist during preoxygenation not only ensures that all equipment and planning is in place, it also sets the scene for crisis management in case of difficulty. Again, checklists are there to help the team ‘check done’ not tell the expert ‘how to do’.
(iii) checklists are for crises as well as routine
Books, journal articles and lay media have popularised the value of checklists for routine procedures – the WHO Safe Surgery Checklist from Gawande and colleagues as well as the central-line checklist from Provonost are the usual cited examples. Critics will tell you that checklists are fine for routine – but not for crisis management.
This is, of course hocum. Since the 1990s, anaesthetists have been implementing training in crisis resource management, leading to development of crisis algorithms & checklists. More recently the NAP4 study from the UK suggested implementation of checklists into practice. Aids such as Borshoff’s Anaesthetic Crisis Manual and crisis cards such as those for malignant hyperthermia, local anaesthetic systemic toxicity, CICO, anaphylaxis etc are enthusiastically adopted in theatre by anaesthetists. Is there evidence? Well, like aviation, anaesthesia is prepared to implement safety measures borne through accumulated wisdom – lessons learned through tragic human loss. Anaesthetists accept the use of such cognitive aids in crisis management, despite ‘expert’ status. Is anyone going to do a randomised-controlled trial of checklist use vs non-use in a crisis? Probably not. Should we use them? Probably yes.
Critics also state that use of a checklist in a crisis will cause delay and that sometimes you ‘just have to act’. Indeed. But if you break this down, it should be clear by now that routine use of a checklist helps establish a team with shared mental model and ability to flatten authority gradients…and if a crisis evolves, all the necessary steps have been considered and are in place. Moreover, during a crisis the impact of stress and cognitive overload can lead to ‘strong but wrong’ decisions by experts using ‘system I’ decision-making – having a reader run through checklist can act as a cognitive waypoint, ensuring alternative options have been considered and acting as an adjunct to the expert.
Critics often point to examples from aviation – either QF32 or US Airways Flight 1549. In the former, QF32 (an Airbus A-380) was on climb-out from Singapore to Sydney, then suffered an “uncontained turbine failure” causes multiple other failures. Using their own human instincts and decisions allowed the crew to override the numerous automated error messages. Does this mean that ‘expertise’ wins out over checklists? Not at all – having a routine – whetehr this be SOPs or checklists – allowed the QF32 team to make decisions even under pressure. In aviation parlance, they were able to aviate-navigate-communicate. Ditto in the case of Capt Sully Sullenberger and crew – loss of both engines after a birdstrike from New York to Charlotte, NC allowed only 208 seconds from crisis to landing – in this case on the Hudson River. There is no checklist for loss of both engines on climb-out – nor was there time to execute the checklists for power loss at higher altitude – nevertheless, the First Officer commenced a checklist and the use of routine allowed the pilot and team to function.
As Sullenberger says :
“My pulse shot up. My blood pressure shot up. My perceptual field narrowed because of the stress. And I had to really actively compartmentalize and focus and force that distraction away, and just concentrate on the task at hand. So I forced calm on my myself and then I imposed order on the situation”
This illustrates the impact of stress on performance – moving into the zone of raised HR and impairment of cognitive skills, with risk of loss of situational awareness and task saturation. Compartmentalising the problem allowed a solution – in this case, I believe regular use of checklists (as well as team training and crisis management) allowed the expert pilot to realise standard options are exhausted and to improvise.
Regular training, standardisation and checklists to ensure task completion allow us to develop both individual and team cognitive resilience.
There is published evidence that use of a checklist for out-of-theatre RSI both reduces error and takes no longer than non-use (“Checklist for emergency induction of anaesthesia in critical care” Babolhavaeji, F. et al, Anaesthesia 2013 68 655-661)
(iv) There is evidence for beneficial use of checklists
This leads on from the above. Critics say that there is no evidence for checklists and that as such they should not be adopted. Most of the work comes from simulation…but as mentioned previously, there is no reason NOT to extrapolate from simulation, given the known problems of under-reporting of error in healthcare and the relative infrequency of crisis during training and subsequent professional career.
Stiegler et al (2012) highlight cognitive errors in as many as 50% of simulated anaesthetic crises – a sobering fact. Moving on from demonstrating error alone in simulated crisis management, Arriaga et al (2013) demonstrate an almost 75% reduction in error rate during simulated surgical crises when comparing management with checklists (6%) vs without (23%). 97% of clinicians involved in this trial expressed a preference for checklist use.
“If I were having an operation & experienced this intraoperative emergency, I would want the checklist to be used”
One of the key recommendations from the UK’s NAP4 study was for use of an intubation checklist to be used alongside capnography in all remote site emergency airway management situations involving critically ill patients, as they have been shown to significantly reduce complication rates (Wijesuriya & Brand, 2014). Learning from NAP4, the recent introduction of a package of RSI checklist in all out-of-theatre intubations in a UK NHS trust lead to improved confidence in managing such situations and has been enthusiastically adopted locally, with suggestions to implement across other institutions.
There is recent evidence for a 50% reduction in the risk of desaturation for children undergoing RSI in a paediatric emergency department. That is impressive. Here’s the checklist user by Kerrey et al in the USA to reduce error in paediatric RSI :
In Australia, the Royal North Shore Hospital ED published an important piece of work highlighting the performance of ED RSI in their institution. It makes for sobering reading, highlighting problems of skill fade, infrequent use of a bougie and complications in 29%. Kudos to the authors for publishing and more importantly, for using this audit to drive change in their institution and elsewhere. Fogg and colleagues have established an updated RSI checklist and are calling for other to join the audit of ED RSI via airwayregistry.org.au
Implementing change in medicine takes time. Yet high-performance teams such retrieval services are also enthusiastic early adopters of checklists – they understand the impact of human factors on crisis management, and even despite regular training will adopt cognitive aids and check tools into daily routine because of perceived benefits. And you think you are too cool or too good to need a checklist et these guys do? Come on!
Even Minh agrees – he is on the record talking at SMACC 2013 about “Airway Clean Kills” (slideset available here). Intubation offers you many ways to kill your patient. Checklists can help avoid some of them as usually Minh espouses…
Slide from “Airway Clean Kills” by Dr Minh Le Cong, SMACC2013
Anaesthetist Dr Mark Knights, whose team in Wales, UK demonstrated less error and no delay in use of an RSI checklist, puts this well :
Arguments Against Checklists
We have covered (and debunked) some of these above. Anaesthesia has been embracing lessons of crisis resource management for almost 25 years, with development of guidelines, algorithms, crisis action cards and checklists as a result. Whilst the weight of evidence for checklists has been demonstrated in routine procedures (elective surgery, central line insertion), there is emerging evidence that checklist use not only reduces error, but also does not cause delay in a crisis. Indeed, regular use may enhance performance in a crisis.
Much of this work has come from simulation – necessary given the problems of error reporting and relative infrequency of crises – but savvy experts who understand the limitations of crisis management even by experts are adopting checklists into their practice, guided by evidence from audit and the emerging evidence of benefit.
Of course checklists are not a panacea for all ill. Merely having a checklist won’t ward off disaster. Neither is a crisis the ideal time to implement a checklist – they need practice, buy-in from team members and regular rehearsal. They need to kept simple and only used for high-risk, tightly-coupled procedures where omission of steps can lead to disaster. Checklist fatigue is a real danger, as is the ‘tick and flick’ mentality of filling in boxes or ‘just another piece of paper’.
Ideal checklists should be kept as a laminated sheet for reference, not for filling in. The items are read as a challenge-response to enable the expert and team to avoid cognitive error in a crisis.
Design is key – a poorly designed or inappropriate checklist is worse than useless. They need to be relevant, simple and adapted to local use.
Have a look at some of the examples below and see if they fulfil the criteria.
Introducing The Vortex
The Vortex is a cognitive tool that we demonstrated in the Airway Workshop. I’ve seen it used effectively on the ETMcourse and seeing it in use by experts and novices alike is one of those ‘lightbulb’ moments of clarity. It neatly simplifies previously complex airway management algorithms into a “high stakes cognitive aid” – simple enough to be recalled in a crisis and flexible enough to be used in any context.
If you missed the workshop, check out The Vortex via vortexapproach.com. There is a neat video demonstrating the ‘read aloud’ manner of a sensible and well-designed checklist in a crisis.
@NicholasChrimes “#Vortex is a checklist. Prompts the enactment of training in crisis”
and
@NicholasChrimes “#Vortex - It’s a novel checklist, for broad strategies not exhaustive, designed for real time crisis Mx”
Nick was kind enough to email me some ‘ammo’ in the argument PRO checklists – particularly poignant as he knows how much Minh loves The Vortex approach (as do I) :
“the Vortex is essentially an innovative graphic representation of a simple checklist for 4 airway techniques & 5 categories of optimisation manoeuvre. The graphic representation allows it to also convey additional elements of urgency (in the funnel) & safety (in the green zone) as well as facilitating use by the whole team”
So – it’s a checklist & cognitive aid – one which Minh endorses. Game – Set – Match.
Here are more examples of checklists used by airway experts :
From Bangor, Wales (authors of study demonstrating use of checklist did not delay intubation and reduced error rates in out-of-theatre airway management). Click to read their presentation via slideshare.
Checklist from from the Royal North Shore Hospital ED via airwayregistry.org.au - see also commentary from Cliff Reid at resus.me
The above checklist is from George Douros of Austin Health (Victoria, Australia) and is discussed over in an article at lifeinthefastlane.com. It’s simple and well laid out.
This example is from Yen Chow in Thunder Bay (Ontario, Canada). He’s a co-author on Minh’s excellent prehospitalmed.com site and posts on PHARM about the benefits of checklists here. Check it out.
Remembering that a true checklist is a “check of what has been done” not a “how to do it read-do”, I use the above in my ED and prehospital bag – works really well as a challenge-response checklist in the final minute before intubation and has the added bonus of a ‘kit dump’ to help layout kit – especially useful if enlisting helpers who may be unfamiliar with the kit.
This has been modified from similar checklist mats used by retrieval services in Australia and the UK. This and a smaller ‘checklist action card’ is available for download and adaptation from the anaesthesia resources section of the ruraldoctors.net site, along with an ED/Theatre Crisis Checklists pdf for iPad.
The above modification of an RSI kit dump and checklist is courtesy of Dr John Hinds from Eire – the essential item is in bottom right corner, concerning cricoid pressure….
Scott Weingart has weighed in on checklists – his version is above and can be downloaded here. I think one has to weigh up the pros of being all inclusive vs ease-of use.
With this in mind, I reckon the BEST checklists are those that are simple, easy to follow and limited to just key items. They should read as checks, not ‘to do’, although the latter blurs the line between checklists proper and cognitive aids or more complex crisis algorithms.
The above ‘checklist for checklists‘ is from ProjectCheck.org – an excellent site which discusses checklist design and implementation.
Of course, checklists are not a panacea for all – they are but one extra layer of defence against error. Checklist implementation requires appropriate leadership, training and organisational change and must take place against a background of embracing human factors into a safety culture. This is discussed more in the book ‘Beyond the Checklist‘. In case you were dead set against checklists – don’t worry, the patients will be doing it for you – here is a checklist for patients to assess the teamwork & safety of their clinicians!
As EM IM Doc states in the “Ch…ch…ch…check it out; ED RSI Checklists” blog post – to create a practical checklist, consider the following:
Design each item to address a specific, actionable, critical safety step
Make it short and easily understandable
Utilize natural breaks in work-flow
Build it in to your process and delegate a team member
Review, Revise and Refine
The last word (for now) should go to Atul Gawande – writing in the Incidental Economist “When Checklists Work & When They Don’t“, he comments on checklists and their naysayers, saying “there is one thing we know for sure: if you don’t use it, it doesn’t work”
In Summary – Real Airway Experts use Checklists
Resources
Recommended Books
“An astronauts guide to life on earth” Chris Hadfield 2013 Macmillan
“Better – a surgeon’s notes on performance” Atul Gawande 2008 Profile Books
“Beyond the Checklist – what else healthcare can learn from aviation teamwork & safety” Suzanne Gordon, Patrick Mendenhall & Bonnie Blair O’Connor 2013 Cornell University Press
“Complications – a surgeon’s notes on an imperfect science” Atul Gawande 2002 Profile Books
“Crew resource management” barbara Kanki, Robert Helmreich & Jose Anca 2010 Academic Press
“The Checklist Manifesto – how to get things right” Atul Gawande 2010 Profile Books
“On Combat – the psychology & physiology of deadly conflict in war and in peace” Dave Grossman 2004 Warrior Science Publications
“Patient Safety” Charles Vincent 2010 BMJ Books
“Patient safety in emergency medicine” Pat Croskerry, Karen Cosby, Stephen Schenkel & Robert Wears 2009 Lippincott, Williams & Wilkins
Recomme