A comprehensive review of regulated en route patient care (Bridges, 2018) identified the need for research to evaluate and describe the current state of US Air Force (USAF) Aeromedical Evacuation (AE) crew en route care practices related to patient safety including patient handoffs, equipment events, and crew safety performance. En route care can range from combat injuries, cardiac, chronic and infectious diseases to mental health and preventive medicine (Allison & Trunkey, 2009). Patients are cleared by a validating flight surgeon and prepared for AE flight transport by En Route Patient Staging System (ERPSS) or Aeromedical Staging Facility (ASF) personnel.

Past AE research has focused on specialized physical and mental health patient treatment protocols to develop evidence-based AE care practices, which guide AE teams as well as Critical Care Air Transport Teams (CCATTs) once airborne. CCATTs consisting of physicians, critical care nurses, and respiratory therapists manage the most critically ill/injured patients. Little is known about how patient care occurs once the patient is handed off from the ERPSS/ASF personnel to the AE flight crews through the various phases of flight until they are transferred at the destination healthcare facility (Bridges, 2018).

Aeromedical transport reduces the time from injury to the highest level of required care to prevent further decline or death (Rowley & Bryant, 2017). Time pressures are constant within the AE care environment and crews routinely work long crew duty days resulting in fatigue and elevated stress (Pierce et al., 2018). Within this dynamic care environment, little is known about the day-to-day, mission-to-mission safety challenges AE crews face while caring for patients in this unique care setting.

The Line Operations Safety Audit (LOSA) safety management methodology has been used by military and commercial aviation to create a ‘snapshot’ view of work in real time (FAA, 2006). LOSA is a proven proactive safety management methodology to identify performance gaps, highlight best practices, and reveal previously latent critical system anomalies during routine flight operations. Data collected during a LOSA is analyzed, categorized, and prioritized by threats (events adding operational complexity outside the control of the crew), errors (crew deviations from expectations or intentions) or undesired states (errors that degrade safety margins or near-misses) (FAA, 2006) (Dalto, Weir, & Thomas, 2013).

The objective of this research was to analyze, compare, and report AE system threats, AE crew errors, and resultant AE undesired states using the LOSA methodology from a randomized sample of 60 live worldwide USAF AE patient care missions conducted from February-March 2020.

The results provide a baseline measure of the USAF AE Threat and Error Management (TEM) program by identifying opportunities for improving AE crew safety and patient safety and provide a clearer perspective to prioritize system enhancements in training, policies, technology, equipment, and other critical frontline resources (FAA, 2006, Hollnagel, Wears, & Braithwaite, 2015).

Methods to determine threats

An expert panel of 16 USAF AE personnel developed a tailored LOSA measurement tool known as the TEM Matrix, Specific AE threats, errors, and undesired states were initially organized by types, sub-types, and sub-codes, building on other LOSA aircrew TEM matrices. These parameters were modified, enhanced, and customized to capture the breadth of AE patient transport operations. Additional data fields including de-identified crew/flight demographics including experience, operational risk management scores, and patient census were collected to help prioritize future continuous performance improvement opportunities.

A de-identified individual crewmember survey included questions regarding inflight crew fatigue and the Aviation Safety Action Program (ASAP) safety reporting utilization – two areas of USAF safety management emphasis (Steckel, 2014). Lastly, a codebook of detailed threat, error, and undesired state types was developed.

AE LOSA observer training was conducted by experienced LOSA facilitators from AE and aircrew backgrounds. Inter-rater reliability exercises were conducted indicating good agreement among observers ( = 0.75). Sixteen observers (eight flight nurses and eight AE techs) were selected based on their experience in AE operations, instructor qualifications, and standardization/evaluation backgrounds. Observers conducted LOSA observations in teams of two.

An AE flight crew typically consists of two Flight Nurses (FNs) and three AE Technicians (AETs) although crew size can vary. AE missions include humanitarian, disaster relief, wartime, and conflicts other than war. AE missions may include more than one sortie to pick up or drop off patients (i.e., receiving patients from more than one forward operating base).

AE crewmembers reported fatigue levels at top of descent as alert (41%), neutral (20%), and tired (39%). Of the 60 observed missions, 73% exceeded a 12-hour duty day (range: 8-22 hours)

The 60 LOSA observations during AE missions transited 25 continental US airfields and 17 airfields outside the continental US including all AF major command areas of operations between February and March 2020. The number of LOSA observations (n=60) represented a convenience sample of non-CCATT, worldwide operational AE patient transport missions conducted between February and April 2020. Military cargo (C-17, C-130), aerial refueling (KC-135), and Learjet (C-21) aircraft were configured for the AE missions by adding patient litters and stanchions, supplemental oxygen, crew supplies, and medical equipment.

Once observations were submitted electronically, contractor data reviewers ensured submissions were complete and accurate. A data verification roundtable consisting of AE experts from the TEM matrix workshop and AE LOSA steering committee was conducted to review the 60 completed observations to validate performance in accordance with USAF policies, procedures, standards, and protocols. Next, the data was analyzed according to the LOSA TEM Matrix framework. Aggregated rates of threats, errors, and undesired states were calculated across the 60 missions to determine the overall threats/errors/undesired state per mission. Aggregated rates of threats, errors, and undesired states that were managed versus mismanaged were calculated with comparisons to other historical crewmember (such as pilot) management/mismanagement rates. Threat, error, and undesired state sub-type total prevalence was calculated using a sort and pivot table method. Prevalence was defined as at least one incidence (observed threat, error, or undesired state) occurring during each LOSA observation (n=60 missions).

Results of the observations

Sixty AE LOSA mission observations were collected. Sixteen LOSA observers were paired in two-person teams (FN/AET) who observed a total of 136 flight nurses and 195 AETs transporting a total of 844 patients during the 60 AE missions. Normally, LOSA observations consist of one observer for each observation in other crew settings but the size of AE teams and the number of patients dictated two observers per observation. The average AE crew component members included 2.27 flight nurses (M = 2.27, range: 1-3) and 3.25 AE techs per mission (M = 3.25, range: 1-4). The average number of patients per mission was 12.72 (range: 1-60). Average flight hour experience for observed flight nurses was 478 and average flight hour experience of AETs was 594 hours.).
Average observed AE threats per mission was 12.9 (M = 12.9, range: 3-28). The percentage of successfully managed threats by AE crews was 72%. Comparative managed threat average percentage for USAF loadmasters was 82% and 84% managed for cockpit crews. Loadmasters conduct loading/unloading operations for cargo and passengers. The four most prevalent observed AE threat sub-types were En Route Patient Staging System/Aeromedical Staging Facility (ERPSS/ASF) personnel (80%), Patient Condition (77%), Equipment/Supplies (72%), and Operational Pressure (72%).

The three most prevalent mismanaged AE threats included ERPSS/ASF (43%), Patient Condition (40%), and Equipment/Supplies (40%). The most frequently mismanaged AE threat event (sub-code) was patient handoff error by ERPSS/ASF sending or receiving team.

Average observed AE errors per mission was 9.67 (M = 9.67, range: 1-36). The percentage of successfully managed errors by AE crews was 69%. Comparative managed threat average total percentage for loadmasters was 74% managed and 79% managed for cockpit crews. The four most prevalent observed AE error sub-types overall were Checklists (72%), Equipment/Supplies (67%), Patient Care Continuum (55%), and External AE Crew Communication (55%). The three most prevalent mismanaged AE errors included Checklists (37%) Patient Care Continuum (37%), and Equipment/Supplies (37%). The most frequent mismanaged AE checklist error event (sub-code) was checklist not performed to completion.

Average observed AE undesired states per mission was 3.33 (M = 3.33, range: 1-11). The percentage of successfully managed undesired states by AE crews was 96%. Comparative managed/mismanaged threat average for loadmasters was 97% managed and 93% for cockpit crews. The four most prevalent observed AE undesired states sub-types overall were Equipment (68%), Patient (65%), AE Crew Member (45%), and Aircraft (23%). The level of prevalence of AE undesired states varied by phase of flight with patient loading through ascent (62%), cruise (52%), descent/offload/before leaving aircraft (52%), mission prep/pre-flight (35%), and post-mission (7%).

Combined AE crewmembers reported fatigue levels at top of descent as alert (41%), neutral (20%), and tired (39%). Of the 60 observed missions, 73% exceeded a 12-hour duty day (range: 8-22 hours).

Take aways of the research

Conducting AE LOSA observations during live patient missions provided essential operational system effectiveness and crew performance information that is critical to patient safety and quality of care continuous improvement. This baseline AE audit provided quantitative and qualitative data to assess the level of operational complexity outside the control of the AE crew (threats), the nature of AE crew errors, and the impact of threat and error management performance of AE crews to recognize, manage, and mitigate threats and errors to avoid serious safety events.

Aircrews managing a higher volume of threats especially those threats, which are unanticipated (‘pop-up’) threats, may become task saturated or distracted during critical operational activities. Reducing overall system threats reduces the system complexity that crews must manage. AE crews mismanaged threats at a higher rate than cockpit crewmembers (28% versus 16%); yet none of the mismanaged threats resulted in patient harm. Mismanaged threats often lead to deviations from expectations or errors. Increasing the ability of AE crews to better anticipate and plan for unexpected threats, especially those threats linked to AE crew errors, could reduce the overall mismanaged threat rate and subsequently the AE crew error rate.

As error rates increase, recovery from those errors becomes more difficult to manage especially when errors are not recognized in a timely manner. AE crews mismanaged errors at a higher rate than cockpit crewmembers (31% versus 21%).

There was no significant difference observed between AE crews and other aircrews undesired state management. Error management improvement opportunities should focus on the most prevalent errors to identify their root cause especially mismanaged errors that are linked to undesired states. System improvements should make errors more visible and recovery strategies more reliable (i.e., challenge/response checklists and standardized patient handoff protocols).

Most undesired states occur when errors (i.e., wrong decisions, communication errors of commission/omission, procedural errors, and workarounds) are mismanaged (FAA, 2006). Transporting patients, especially higher acuity patients, in an aircraft at higher cabin altitudes with inflight turbulence, noise, and vibration creates additional risks to safe patient care. Patient care in AE operations is dynamic presenting unique challenges to AE aircrew unlike cockpit or loadmaster crewmembers. Each undesired state creates a proactive opportunity to investigate root causes including the mismanaged errors and/or threats, the crew response (or lack of response), phase of flight factors, and the resulting outcomes (recovery actions).

As error rates increase, recovery from those errors becomes more difficult to manage especially when errors are not recognized in a timely manner. AE crews mismanaged errors at a higher rate than cockpit crewmembers (31% versus 21%)

Nearly 4 in 10 AE crewmembers reported being moderately tired, extremely tired, or completely exhausted nearing the end of each mission. While AE crew members self-evaluate fatigue states prior to each mission using an Operational Risk Management (ORM) methodology based on subjective fatigue ratings, AE crews do not observe formal inflight rest schedules, and do not have dedicated inflight crew rest facilities onboard the aircraft.

Reporting threats, errors, and undesired states using non-punitive reporting system provides opportunities to analyze, track, trend, investigate, improve, and learn from high-risk experiences, mistakes, and best practices before an accident or serious safety event occurs. Operational leaders should educate and encourage AE crews to use the non-punitive reporting systems as soon as possible to facilitate continuous improvement and promote a culture of safety.

Having two LOSA observers may have increased the average rate of threats, errors, and undesired states of AE crews compared to other crew settings with one observer. This AE LOSA demonstrates the ability to adapt the LOSA process to AE crewmembers while also demonstrating the importance of LOSA in a patient care setting to proactively assess operational risk and patient safety. The success of this initial AE LOSA adaptation is promising for spreading LOSA to CCATT teams, helicopter emergency medical services, other air medical transport operations, combat casualty settings, and high-risk hospital units to enhance proactive safety management programming. Further analysis of the AE LOSA such as threat, error, and undesired state management rates with increased patient loads, crew size, experience levels, and duty day duration could lead to additional system enhancements.

To ensure LOSA data is actionable, the Air Force Air Mobility Command (AMC) convenes a Safety Investigation Board (SIB) consisting of aviation domain experts to further investigate, deliberate, and make recommendations for system improvements. AMC senior leadership makes the final decision to accept or reject SIB recommendation and findings followed by the assignment of improvement responsibilities, allocation of resources, and milestones for completion.
Future LOSA research in other air medical transport settings could lead to a standardized approach for implementing the LOSA methodology to compare TEM safety performance, share TEM best practices, and improve aviation and patient safety outcomes across the air medical transport community.