TY - JOUR
T1 - Situation Awareness Assessment for Anesthesia Residents (SAAAR)
T2 - Development and Preliminary Evaluation of a Multimodal System
AU - Daza-Beltrán, Carolina
AU - Fajardo Escolar, Angélica Paola
AU - Caro, Martha
AU - Suárez, Daniel R.
N1 - Publisher Copyright:
© 2026 Human Factors and Ergonomics Society. This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
PY - 2026/1/2
Y1 - 2026/1/2
N2 - Objective: This study reports the development and preliminary evaluation of SAAAR, a multimodal system designed to assess and support the development of situation awareness (SA). Background: SA is critical in anesthesiology, yet existing assessment methods lack standardized tools tailored to its complexities of anesthetic practice. Systems developed in other domains have limited applicability, highlighting the need for a purpose-built approach for anesthesia residents. Method: The SAAAR comprises two components: a 16-item behavioral marker scale and a structured debriefing with eye-tracking. Thirteen anesthesiology faculty tested interrater and test-retest reliability, while five experts conducted content validation of the scale. Both components were implemented in a simulation-based training program for preliminary system evaluation. Results: The behavioral marker scale demonstrated moderate content validity and high reliability. Internal consistency was strong (McDonald’s Ω = 0.928), test-retest reliability high (Spearman’s ρ = 0.952), and interrater agreement moderate (Kendall’s W = 0.412). Faculty reported the scale to be clear, comprehensive, and easy to use. Pilot implementation showed significant improvements across domains (Wilcoxon signed-rank test), indicating the system’s potential to provide targeted feedback and guide educational interventions. Conclusions: Grounded in HFE principles, the SAAAR provides a structured approach to assessing SA in anesthesia residents and demonstrates preliminary potential to inform educational strategies. Further research is required to determine its impact on clinical performance. Application: The SAAAR offers residency programs and human factors experts a practical tool for assessing SA and designing targeted training. Its adaptable framework suggests potential applicability in other high-pressure medical contexts, pending further evaluation.
AB - Objective: This study reports the development and preliminary evaluation of SAAAR, a multimodal system designed to assess and support the development of situation awareness (SA). Background: SA is critical in anesthesiology, yet existing assessment methods lack standardized tools tailored to its complexities of anesthetic practice. Systems developed in other domains have limited applicability, highlighting the need for a purpose-built approach for anesthesia residents. Method: The SAAAR comprises two components: a 16-item behavioral marker scale and a structured debriefing with eye-tracking. Thirteen anesthesiology faculty tested interrater and test-retest reliability, while five experts conducted content validation of the scale. Both components were implemented in a simulation-based training program for preliminary system evaluation. Results: The behavioral marker scale demonstrated moderate content validity and high reliability. Internal consistency was strong (McDonald’s Ω = 0.928), test-retest reliability high (Spearman’s ρ = 0.952), and interrater agreement moderate (Kendall’s W = 0.412). Faculty reported the scale to be clear, comprehensive, and easy to use. Pilot implementation showed significant improvements across domains (Wilcoxon signed-rank test), indicating the system’s potential to provide targeted feedback and guide educational interventions. Conclusions: Grounded in HFE principles, the SAAAR provides a structured approach to assessing SA in anesthesia residents and demonstrates preliminary potential to inform educational strategies. Further research is required to determine its impact on clinical performance. Application: The SAAAR offers residency programs and human factors experts a practical tool for assessing SA and designing targeted training. Its adaptable framework suggests potential applicability in other high-pressure medical contexts, pending further evaluation.
KW - anesthesia and perioperative care
KW - medical simulation/training and assessment
KW - patient safety
KW - situation awareness
KW - training evaluation
UR - https://www.scopus.com/pages/publications/105026411382
UR - https://www.mendeley.com/catalogue/2627b049-5078-3cf8-a05f-6ff7104267c8/
U2 - 10.1177/00187208251413484
DO - 10.1177/00187208251413484
M3 - Article
AN - SCOPUS:105026411382
SN - 0018-7208
VL - 68
SP - 673
EP - 687
JO - Human Factors
JF - Human Factors
IS - 5
ER -