Colin F. Mackenzie, MD

Yan Xiao, PhD

Peter F.M. Hu, MS, CNE

The LOTAS Group*

Judith Orasanu, PhD

*LOTAS Group: Colin Mackenzie (Chair) W. Bernhard, C. Boehm, J. Blenko, H. Cline, D. Donnelly, R. Dutton, P. Hu, M. Jaberi, M. Kalish, K. Mitchell and Y. Xiao

Prepared for

Ames Research Center

GRANT NCC2-921

("Remote Decision-Making for Trauma Patient Resuscitation", PI: Colin F. Mackenzie)

February, 1999

We would like to thank numerous people who have contributed to the project. Many of the clinicians in R Adams Cowley Shock Trauma Center at the University of Maryland worked as subject matter experts and experiment participants. Among them were the LOTAS (Level One Trauma Anesthesia Simulation) Group members (excluding authors of this report): W.N. Bernhard, J. Blenko, C. Boehm, H. Cline, D. Donnelly, R. Dutton, M. Jaberi, and F. Panico. Other anesthesiologists and nurse anesthetists participated in the projects: Drs. Fouche and Freeland and Mr. Baker. Among the Trauma Resuscitation Unit nurses who participated in the project, we would like to thank Paula Tasia, Harold Hardinger, and Victor Gustina. Four attending trauma surgeons participated as subjects in experiments and they were Drs. Tom Scalea, Philip Militello, and David Gens and Sharon Henry. Dr. Richard Horst and David Mahaffey, both of Man-Made Systems, participated in an early draft of the protocol.

We also like to thank several people who provided critiques and suggestions over the course of the project. They are Tom Nygren, Elliot Entin and David Serfaty..

Several research assistants provided invaluable contributions. They were Ben Harper, Denise Ovelgone, Sam Chang, Rick Spencer, Amaly Rahman, Fahd Habeeb, and Jacob Seagull. We would also like to thank Renee Kahn for her secretarial support and Paul Delaney for his able assistance in data analysis. In addition, Thelma McClellan typed this Final Report. Our thanks to all those who assisted on this project.

Distributed decision making is a research paradigm for understanding organizational, group, and team decision making when members are distributed in several senses: physical locations, access to information, authority, expertise and access to resources. Space missions are one of the prime examples of distributed decision making as collaborative members are distributed in all of these five senses. Another example, which is cognitively similar in that expert decision-makers are distributed in a team, is telementoring of trauma patient resuscitation from a remote location. We used previously acquired audio/videotapes of trauma patient resuscitation as experiment stimulus materials to examine how remote experts extract information from multi-media sources. We characterized the information used by remote expert decision-makers to make diagnostic judgments and interventions in emergencies occurring during trauma patient resuscitation.

Experiment 1 investigated what cues were detected and what was missed by remotely situated expert decision-makers.

The results showed that missed cues occurred for several reasons, including degradation of verbalizations and verbal communications because of background noise interference; viewing range for the remotely situated subject was restricted with a fixed camera location; visual access from this fixed location was not secure because care providers moved in and out of the camera line of sight and sometimes obstructed the view at critical moments when the cue was presented; typical video imagery used, showed the activities of 3-5 crew members of the trauma team working on patient resuscitation and such multiple actions, appeared to overwhelm the remotely situated expert decision-makers causing visual information overload.

Some clues were not picked up by all remote decision makers. Difficulties in recognizing these cues included: lack of an adequate dynamic mental model of patient status because they had not participated in patient care and were therefore cognitively "out-of-the loop" in regard to their information seeking; there was lack of context information in comparison to the on-site providers. It was not as obvious to the remotely situated person what other team members were doing or how to extrapolate their intentions; because not all concurrent activities could be simultaneously followed.

Nurses, surgeons and anesthesiologists understanding of the identical audio/video material was compared by responses to questions about the current patient status, team activities, future patient status and team objectives. The analysis of nurses, surgeons and anesthesiologists responses compared performance of correct answers against an ideal understanding of the cases and content of the answers categorized into airway, breathing circulation, patient status and injuries, team activities and other.

The performance analysis showed that anesthesiologists performed better than the other two groups. Performance scoring items that presented difficulties included detection of conflicting plans, and anticipation of nursing plans. Surgeons and nurses did poorly in determining task status of placement of a breathing tube, traditionally the responsibility of the anesthesiologists.

Content analysis showed that the distribution of answers across the six categories (above) among all three groups of subjects was similar. The surgeons provided more general comments, the anesthesiologist subjects used a higher proportion of phrases describing airway related issues whereas the nurses were consistently more focused on team work.

All subjects experienced, at one time or another, similar difficulties to those in experiment 1. An explanation for why anesthesiology subjects out performed the nurses and surgeons is that the videotapes selected for this experiment all contained the activities of airway management, a role performed by anesthesiologists on the trauma team. The traditional divisions of labor within the trauma team may have constrained the nurses and surgeons and prevented them from detecting critical cues. The nurses performed better than surgeons, this may have been motivational. The surgeons may have used different types of descriptions than nurses, but in general, surgeons provided fewer written and verbal responses than nurses or anesthesiologists.

These results suggest that experts with different experience backgrounds may appreciate different aspects of events and activities presented in audio/video sources.

Information extracted from video was identified by use of the eye tracker. Nurses, surgeons, anesthesiologists and medically naïve undergraduates (control group) participated. They provided verbal comments and answered questions about the current patient status and team activities and future patient status and team objectives, as in Experiment 2, while wearing the eye tracking device.

All subjects spent the majority of the time looking at the head and faces of the care providers on the video. The eye movements of the control group were rapid over large areas of the video in comparison to the expert subjects. Subjects with different experience backgrounds among nurses, surgeons and anesthesiologists had different visual scanning patterns. The distribution of total time spent on the area surrounding the patients’ head was just over 40% and on the care providers’ under 40% of the total viewing time. The nurse subjects scanned more around the patient and the anesthesiologists looked most at the airway manager. Viewing of the vital signs data occupied about 10% of the total viewing time.

The eye tracking data corroborated the hypothesis of the effect of experience background on information extraction during remote diagnosis.

As a result of understanding team coordination, a remote decision-maker might be able to identify decision points, high workload periods and problems in team coordination.

Videotapes were reviewed and several non-communication task coordination activities were noted including: following the protocols, following the leader, anticipation of future events, activity monitoring the task status of team members. Explicit verbal communications regarding situational assessment and future plans were relatively rare in comparison to non-verbal communication. When team members voluntarily provided their views, it occurred when the team was clearly at a decision point. There was considerable variation among team leaders in plan verbalization with some leaders providing clear intentions; while others appeared to let the events drive the team actions and the goals were inferred by these actions.

Coordination breakdowns occurred in a number of crisis situations including when: extreme difficulties or unexpected patient responses were encountered which prevented the implementation of routine procedures; the team was under pressure to seek alternative solutions; there were unexpected attempts to adopt novel solutions to acute emergency situations. These breakdown situations compromised the abilities of the supporting team members to provide assistance because of their lack of anticipation of the need. Coordination breakdown occurred when the patient was so unstable that the treatment plan had to be abandoned, such changes in plan occurred during crisis and under great time pressure, and required the team to change their process from diagnostic activities (hypothesis seeking) to action activities (hypothesis testing) rapidly.

Verbal communication was viewed as only one of many ways teams use to coordinate their activities. Other communication media include; activities, workspace, events and focus of attention of team members. In most circumstances, team coordination was achieved with a minimum of explicit verbal communication.

Forty videotaped cases were reviewed that identified a wide range of sources of uncertainty. A total of 76 uncertain items were identified by examining verbal communications and subject matter expert reviews. These uncertainties were categorized as patient related uncertainty (20%); and team/organization related uncertainty (41%). It seemed that lack of communication among team members and among personnel work in nursing, surgery, and anesthesiology contributes to many of the uncertainties identified. In addition, technological issues such as signal interference of patient vital sign monitors cause uncertainty in many crisis situations because of patient factors (low blood pressure, combativeness, etc.) cause signal detection failures. Lastly, because of overlap in task distribution among team members, uncertainties occur about who should do what and when.

The findings from these experiments have implications for the design of telecommunication systems in support of distributed decision-making.

Such objectives could be achieved by:

• Use of capture of individual care providers communications through a microphone worn during telecommunications.
• Use of directional microphones and noise cancellation technology.

The restrictions in viewing range and insecure viewing access could be overcome by:

• Use of multiple cameras including a pan/tilt/zoom camera providing a close-up detail and multiple fields of view.
• A fixed camera providing an overview of the entire area and team activities.

• Providing non-video supporting information to help remote decision-makers comprehend video images and to compensate for the lack of complete data about remote events and activities.

• Having a team of remote experts with different experience backgrounds to address concurrent tasks when remote activities are multi-disciplinary.

• Training of the remote decision-maker to appreciate tasks outside of their traditional domain specialty and division of labor in clinical practice.

• When situations evolve rapidly and multiple care providers are carrying out tasks, all of what is presented on video will not be perceived by the remote decision-maker.

• The cues missed by the remote decision-makers will be biased by their experience background.

• Training or cueing tele-consultants to systematically scan video sources may overcome such difficulties.

• The shared event space is an important media for non-verbal communication and the design of the workplace should not interfere with this.

• Improvement in uncertainty may be achieved by better intra-team communication (e.g. communication protocols at key points, checklists, etc.).
• Verbalization of explicit intentions and plans to all team members.

• Improvement in patient vital sign monitor technology to minimize signal detection interference.

• Detailed understanding of the cognitive activities involved in the medical domain is necessary for the best use of telecommunications technology in healthcare.

Distributed decision making is a research paradigm for understanding organizational, group, and team decision making when members are distributed in several senses including: physical location, access to information, authority, expertise and access to resources. Space missions are one of the prime examples of distributed decision making as collaborative members are distributed in all of these five senses. Telecommunication advances in recent decades have overcome many of the technical barriers to communication over distance and time. Increasingly telecommunication systems have become an integral part of many professions, enabling remotely located individuals to collaborate on problem-solving with expertise unavailable locally. Yet our understanding of how people work together when using communication technologies has been lacking (National Research Council, 1990; Rasmussen et al., 1991; U.S. Congress, Office of Technology Assessment, 1995). Interesting and challenging research issues arise in the use of telecommunication systems for decision making and problem solving.

Telecommunication advances enable remotely situated individuals to collaborate on problem solving with expertise that is not available locally. This project explores the cognitive demands and information use of decision-makers in medical diagnosis and treatment. Existing videotapes of real trauma patient resuscitation and management at the Shock Trauma Center were used as the stimulus material. There are clearly differences between the Space environment and a trauma center. In Space, there is protracted isolation and confinement. There can be physical deterioration of the astronaut due to weightlessness and illnesses. In the spacecraft, unlike the trauma center, there are finite supplies and limited and fixed resources. As a result, there are limited options available to deal with unanticipated events. However, using audio/video records of human processes in real life trauma patient resuscitation as surrogate material allows testing the understanding of decision making by remote experts, and examining how these experts view the multidisciplinary teams’ function in dynamic and stressful situations. In addition, such an approach can be used to identify the information that remote decision-makers can extract from audio/video records.

Like Space flight, there are many uncertainties confronting the trauma team decision-makers. There are unknowns about the patient (site and extent of injury, past medical history), and because emergencies are unpredictable, the incoming patient workload is unpredictable. In an analysis of the impact of uncertainty on trauma team performance (Xiao & Mackenzie, 1998), we found 76 uncertain items in 40 patient resuscitations. These were categorized as patient-related uncertainties (26%), (including reports provided by distributed pre-hospital team members, effect of treatment interventions and mechanism of patient injury). If, by analogy, the spacecraft were the patient, many of these factors would also probably be causes of uncertainty for the astronaut. The other major cause of uncertainty experienced by the trauma team was team/organization related uncertainties (41%) (including task distribution among team members, interaction of other team members, status of team members task accomplishment, and resource availability and schedules). In acute events, such as emergencies aboard the spacecraft, similar uncertainties will probably exist for the astronaut.

The domain of trauma resuscitation is used as a "laboratory" to develop and test general characteristics of how remotely situated decision-makers understand events in dynamic domains. Although the domain of trauma resuscitation is a highly specialized medical domain, it shares many similarities with Space missions as described above. In order to understand how real crew’s function in real, stressful situations, the "laboratory" we studied can be a valuable surrogate to provide insight into the Space environment. We paid special attention to the underlying theoretical concepts, such as task urgency, and uncertainty so that our results can be generalizable to other domains.

In addition to providing data that may yield insights into the cognitive processes involved in skilled performance and decision-making, these studies of distributed decision-making in trauma resuscitation may have medical applications of interest to NASA and the medical community (e.g. diagnosis and treatment of medical problems by telecommunication links).

While we do not yet have a thorough understanding of the knowledge structures, mental capabilities, allocation of attention, decision-making strategies, nor team interaction dynamics that predict successful performance in the resuscitation and treatment of trauma patients, our previous findings (see Mackenzie, et. al., 1993, 1994) suggest the possibility that the involvement of a remote expert, depending upon the information available to him/her, may assist the on-site team in avoiding certain pitfalls. For example, some errors in this task environment have been attributed to the team fixating inappropriately on suspected instrumentation problems, at the expense of continued observation and physical examination of the patient. A remotely located expert might be less prone to being caught up in such inappropriate allocations of collective attention. Trauma teams functioning in high stress, emergency cases have been shown to take procedural short cuts, which can be counterproductive, e.g., failing to make use of available instrumentation. Remote experts, to the extent that they retain a "big picture" perspective of the case, might more readily detect procedural oversights or other errors that are due to the stress of the moment rather than to lack of knowledge. Likewise, it may be easier for the more detached, remotely located expert to focus on trends in patient vital signs, and to formulate diagnostic conclusions therefrom, while the on-site decision-maker may be burdened with concurrent tasks to the extent that he/she only has the working memory capacity to monitor moment to moment.

These studies all demonstrate that in highly trained teams with experienced members, communication patterns varied and there are ways for leaders to exert influence without explicit communication. In contrast to many previous studies on leadership, verbal activities are usually the only ways in which leaders function. Such difference would have direct bearing on the potential impact of new communication technologies on leadership.

Verbal communications have often been studied as the major form of coordination process (Kanki, Folk, & Irwin, 1991). The concept of "implicit coordination" was introduced when teams were found to be able to coordinate with reduced communications (Serfaty, Entin, & Volpe, 1993), especially under high workload situations. To investigate factors promoting implicit coordination, it has been hypothesized that "shared mental models", or shared understanding of goals and tasks, is a key, since division of labor in most work settings may have prevented team members from understanding other people’s tasks. Volpe et al (1996) tested this hypothesis and found that cross training, in which team members were trained in other people’s tasks, improved team performance by prompting implicit coordination. The concepts of shared mental models and implicit coordination and related empirical data highlight the issue of communication cost. When workload and time pressure is high, reducing the cost or workload related to communication has obvious advantages. If it is important for team members to share an understanding of each other’s tasks and goals, which are relatively stable, it is equally important for team members to be aware of task situations and each other’s activities, plans and work focus, all of which are changing in dynamic work settings.

The advances in computation and communication technologies have introduced new ways to communicate, especially for those whose work is in office settings with documents as the primary work objects. The great potential in using new communication media has given rise to a research field "Computer Supported Cooperative Work" (CSCW) and a new line of products have emerged for facilitating work by multiple people: groupware. To guide the exploration of various technologies, Ellis, et al (1991) proposed a taxonomy to describe communication media: (1) same place or distributed across locations, and (2) synchronous or asynchronous. Electronic mail, for example, is a medium that allows exchange of information asynchronously among people at different places. In comparison, information exchange in face-to-face meetings occurs synchronously and at the same place.

What seems remarkable is that the "group task circumplex", proposed by McGrath in 1984 (described in McGrath & Hollingshead, 1994) was not specifically for addressing issues in group supporting technology, but provides explanation for much of the experimental results on the use of technology in groups (McLeod, 1992). McGrath (1990, 1991) and his colleagues (McGrath, et al, 1993; Straus and McGrath, 1994) considered the role of communication medium in the context of workgroups working together over time and carrying out tasks requiring different levels of coordination (e.g. idea generation, judgment, and multiple choice). Two major findings from their studies are (1) that when groups gain experience in working together, the need for communication is reduced and the importance of high-media communications (such as in face-to-face meetings and in video-conferencing) decreases and (2) that face-to-face communications benefit tasks requiring high levels of coordination.

Another major contribution of CSCW is a better understanding of the face-to-face medium. Through experiments on use of technology, a number of previously often overlooked aspects about face-to-face communications were emphasized. For example, face-to-face communications carry many informal and redundant cues through auditory and visual channels, and video conferencing systems often are unable to provide these cues (e.g., Krauss, et al, 1977). Even with the rather elaborate system described by Abel (1990), users still did not communicate in the same way as in the way in which they did at the same site (e.g. conversations were never as intimate as those carried out face-to-face). Using a survey methodology, Michailidis and Rada (1997) compared electronic mail, face-to-face, fax, post, and telephone in terms of coordination (commitment management, decision-making, awareness, communication, and transparency, perceptions). Face-to-face was the most effective mode of communication.

Trauma patient resuscitation is a specialized domain in which critically ill or injured patients are resuscitated and treated in a dedicated facility. (At the end of this report there is a glossary explaining the medical terms used in this report.) Prototypical activities of a resuscitation session start with a radio dispatch from the field. Initial examination and resuscitation is done in a patient admitting area, 10 of which make up the trauma resuscitation unit. If needed, the patient is then transported to an operating room for further procedures. The first goal of the resuscitation team is to maintain the life processes in the patient. A work domain analysis (Rasmussen et al, 1994, Ch.2) of the initial stages of trauma patient resuscitation produced a hierarchical representation of means-end relationship, which is shown as an abstraction hierarchy in Figure 1. The figure also describes the distribution of the responsibility among the team members.

The personnel in a typical team can be divided into three crews: surgical, anesthesia, and nursing, each with its own crew leader. The surgical crew has one or two surgeons, one of which is the assigned team leader, and an emergency medicine fellow. The anesthesia crew has a nurse anesthetist and one or two anesthesiologists. The nursing crew has one or two nurses and a trauma technician. In comparison to other work domains, such as command and control centers, the teams in the domain of trauma patient resuscitation has the following characteristics:

1. Team members are closely located and share the same physical workspace.
2. Team members are trained very differently (e.g. surgical versus anesthesia).
3. Teams may contain personnel under training and their supervisors, even in regular operations.
4. The composition of each team changes from case to case, and work schedules of each of the three crews may rotate differently.
5. Resuscitation teams have not gone through extensive training in teamwork, and the roles and job specifications for each team member are largely the result of tradition.

During the analysis of team activities, we found that it was important to define who the team leader is in the domain of patient resuscitation. Although every resuscitation team has an assigned team leader (the surgeon), he or she may not lead all resuscitation efforts. In particular, the anesthesia crew takes the full responsibility of airway management. During different stages of resuscitation different crews’ activities may dominate the team and the leader in the dominant crew leads the team’s activities. Thus the designation of team leader should be viewed as dynamic and task specific. In this report, all the references to team leader are made in this sense. Resuscitation teams were situated in the patient admitting areas (see Figure 2 for an overview of the areas) and were surrounded by resuscitation equipment (e.g. intravenous fluids and airway management and monitoring equipment).

Uncertainty and time-pressure are two of the major characteristics of the domain of trauma patient resuscitation. Patients presenting for emergency care often have unknown medical histories that may include pre-existing diseases and allergies. The extent, site, and mechanism of injuries may not be clear. While undergoing resuscitation, the patient’s condition can change rapidly and dramatically, the pace of resuscitation activities is dictated by patient responses. There is a very brief window of opportunity during which a great deal can be done to save the patient’s life. (For example, damage to the cerebral cortex can occur within a few minutes if the oxygen supply is inadequate.) Previous statistics have shown that about 80% of trauma deaths occur in the first four hours after injury (Brown, 1987). The time-pressure factor also compounds with the uncertainty, since the resuscitation team may not have the luxury of waiting for extensive patient monitoring information but have to act with what is available.

Our investigation included detailed examination of an important part of trauma patient resuscitation: the management of the patient’s airway (for breathing). Airway obstruction is a most rapid cause of death and protection of airway is the first priority in resuscitation of trauma patients. Adequate ventilation of the clear airway is the second priority and perfusion of the vital organs the third priority. Emergency airway management (known as intubation-see below) is challenging in the trauma patient due to time pressure and uncertainty about the patient’s injuries and because of labile vital signs. It poses an interesting context in which to study human performance. Intubation needs to be carried out expeditiously, because during attempts at the procedure, the patient receives no oxygen and is not ventilated. (Mackenzie, et al, 1996). The trauma anesthesiologist is the trauma resuscitation team member who is responsible for management of the airway, providing ventilation and maintenance of physiological values of heart rate, blood pressures and oxygenation. In addition, the anesthesiologist may induce anesthesia often before the patient is stabilized or during the resuscitative effort, to allow surgical correction of trauma induced injuries. The acute management of major trauma, therefore, provides a rare opportunity to study both individual and team performance under stress in a setting that shares some of the same cognitive milieu as military decision making.

The importance of studying performance of airway management is emphasized by the American Society of Anesthesiologists’ Closed Claims Project (Caplan, Posner, Ward and Cheyney, 1990; Cheyney Posner and Caplan 1991) which found that inadequate ventilation, esophageal intubation and difficult tracheal intubation were frequent causes of critical events during airway management. Those events are the most common mechanisms of respiratory related adverse outcomes. Among 2,046 cases with adverse outcome from anesthesia that were examined, 762 (37%), were associated with respiratory events; of these, 678 (89%), were problems associated with airway management. Many of these events involved human error, in that health care workers failed to recognize respiratory problems. In 300 cases with airway trauma, obstruction, aspiration, bronchospasm or pneumothorax, (all events seen with increasing frequency in trauma patients) the incidence of severe injury (brain damage or death) was 47% (141/300 patients).

Tracheal intubation is a sequence of steps to pass a tube (endotracheal tube or ET tube) through the patient’s mouth and vocal cords and into the trachea. Figure 3 is a photographic illustration of tracheal intubation. This procedure was chosen as the task focus of our efforts for several reasons. First, the activities involved in intubation are reasonably well-defined, and intubation has a clear start and finishing point. Second, comparisons could be made across different types of cases as it is used in circumstances with variable time pressure. Third, it occurs frequently enough to allow a reasonable number of the procedures to be recorded. Fourth, the process of intubation is generally considered stressful by clinicians because of the frequency and severity of adverse outcomes and it is high in workload.

Intubation is used when there is a need to protect the patient’s airway against aspiration of vomit and prevent obstruction. For many trauma patients, spontaneous breathing and self-protection of the airway are compromised by injury (for example, if the patient is in a coma or breathing is obstructed by blood). The first priority for emergency medical personnel is to ensure that the patient’s airway is unobstructed and the patient is ventilated adequately. In patients with a compromised airway or obstructed breathing, assurance of oxygenation and ventilation is achieved by intubation. The urgency for tracheal intubation in emergency care varies primarily due to the level of the patient’s spontaneous breathing efforts and the nature of the injury suffered by the patient. If the patient is breathing adequately and there are no indications to protect the airway, intubation may be delayed until the patient requires surgery.

The personnel involved in emergency care usually includes anesthesia care providers, surgeons, emergency medicine physicians, nurses, trauma technicians, and other specialists. It is the responsibility of the anesthesia care providers to perform intubation. While intubation is in progress, other caregivers may perform other tasks, such as the placement of intravenous lines. In this report the term "team" is used to denote the emergency care personnel directly involved in the care of the patient and present in the vicinity of the patient, although considerable independence exists for the sub-groups of surgical, anesthesia, and nursing crews who compose the trauma team.

Six steps are involved in intubation. The first step (pre-oxygenation) is to build up oxygen reserve in the lungs by providing the patient with 100% oxygen. The second step (induction) is to induce anesthesia and muscle paralysis by intravenous injection of drugs to suppress the patient’s gag reflexes to facilitate insertion of a tube into the trachea through the vocal cords. The third step (laryngoscopy) is visualization of the vocal cords through a special instrument (laryngoscope) inserted into the mouth so that the fourth step (tube insertion) of passing a tube through the vocal cords into the trachea can readily occur. Direct visualization may be difficult due to blood and secretion in the mouth and the throat, which may require suctioning. In case of inadequate direct visualization, there will be initial uncertainty as to where the tube is placed. The fifth step (verification}) is to verify the position of the tube in the trachea by listening to the patient’s chest (with a stethoscope) and by looking at patient monitors. It is essential to have the tube in the trachea as opposed to the esophagus or the bronchus. If the tube is mistakenly placed in the esophagus, which is next to the trachea, oxygen will be delivered to the stomach instead of the lungs and no ventilation and oxygenation is possible. If the tracheal tube is inserted too deep, it will pass into the bronchus and only one of the two lungs will be ventilated. The sixth step (connection) is to connect the tracheal tube to a mechanical ventilator (an automatic device that delivers oxygen and other gases, and removes exhaled carbon dioxide) and confirm that the ventilator is functioning by repeated examination of the chest or analysis of external gas for CO2. Using a mechanical ventilator frees the clinicians’ attention and hands so they can perform tasks other than patient ventilation.

We embarked on this project to address two research questions, one general to distributed decision making and the other specific to medical applications of distributed decision making.

As a first general question, the effort was directed at answering the question, what information is used by a remote decision-maker? For a decision-maker to effectively participate in a decision making process, a prerequisite is to be able to assess the situation and problems at hand. In a distributed decision making context, this requirement means that the decision maker has to rely on telecommunication links (e.g. computer, telephone, and video networks) to achieve situation assessment and to understand problems to be tackled. This requirement may be fulfilled relatively easily when events evolve slowly, but it can be difficult to satisfy when situations change rapidly (a similar argument is put forward by Allely, 1995). Little empirical data have been reported on how people can assess dynamically changing situations and problems through telecommunication links. Therefore, there is little empirical basis existing to guide the design of telecommunication systems in support of distributed decision-making.

A second question, specific to medical applications of distributed decision-making was also addressed, how should we make use of remote expertise? This question is related to a broadly defined field of telemedicine: the use of telecommunication technologies in the practice of medicine and healthcare. With the development of technologies, many of the long-time desires of medical practice seem to come true: the physicians can see and talk to the patient over long distances, physicians themselves can use video teleconferences to save travel costs. Much of the efforts on telemedicine have been driven by technology and have been based on untested assumptions about the impact of technology. As evidenced in the research on the impact of technology and on the use of video teleconferencing systems in organizations (e.g. Woods, et al, 1995; Finn, et al, 1997), each use of technology is an experimentation with unexpected outcome and creates a new work environment with new tasks and requirements. New modes of errors and new patterns of workload will result when technology is deployed, sometimes seemingly innocently replacing or automating a component in the work environment.

It is unclear what information a remote medical decision-maker requires to supervise management of medical emergencies and how effective remote management is at producing appropriate and timely diagnosis and management of humans with medical problems. It is also not known how different types of medical subject matter experts (surgeons, anesthesiologists, nurses) function as independent remote decision-makers and thirdly, how the response of the on-site trauma patient managers affects the remote decision-maker is also uncertain.

As a first step to address these research questions, our project examined the ability of trauma experts to remotely manage trauma patients through telecommunication links, and identify how telecommunication systems should be designed to facilitate such tasks. Important features of the domain of trauma patient resuscitation are that the patient’s condition changes rapidly and is often uncertain, and that the resuscitation effort is carried out by a multi-disciplinary team. Apart from being used as a research "laboratory", trauma patient resuscitation could benefit from telecommunication because in many situations injured patients are spatially remote from expert care providers

Using our existing videotapes and database (see below-under video library) including transcriptions of reviews of the management by participant and non-participant subject matter experts (SME's) and summaries of diagnostic and surgical findings and laboratory and radiological data, we examined the following specific aims:

1. Characterize the importance of various information providing factors in remote decision-making for the emergency management of the trauma patient. These results would address questions about the relative importance of patient vital signs (heart rate, blood pressure, oxygen saturation, etc.), and physical examination in determining appropriate emergency medical management of the trauma patient.
2. Assess the effects of different types of subject matter experts (surgeons, anesthesiologist, trauma nurse) functioning independently as the remote decision-makers. This specific aim would examine how strategies of information-gathering data interpretation and integration differ among medical subject matter experts working independently.
3. Determine how team coordination and breakdowns in coordination might impact on the decision-making of a remote expert and to identify what remote experts were uncertain about when viewing events through multimedia telecommunication links.

The studies were carried out in the R Adams Cowley Shock Trauma Center of the University of Maryland. This facility is a Level One trauma center that is regarded as one of the pre-eminent facilities of its kind in the world. As such, it serves as a training ground for trauma anesthesiology and surgery residents and faculty from all over the world. By drawing upon staff members involved in trauma anesthesiology surgery and nursing over the course of an approximately three-year period, we were able to gather data from individuals spanning a wide range of age, trauma treatment experience, and training backgrounds.

We used a comprehensive audio-video-data acquisition system (VASNET) to collect audio/videotapes in this real environment. Synchronization of various data sources to within 100th of a sec (e.g. audio, patient vital signs, and multiple images) occurred by means of the machine readable time code.

A unique feature of the video recordings was that the video images contained overlaid patient vital signs (Figure 4). The video images are overlaid with patient vital signs obtained from a serial interface on the patient’s monitors (Mackenzie, Hu, Horst, et al, 1995). These vital signs are essential to understanding of the decision-making process of the Resuscitation Team. They include heart rate, oxygen levels in the patients blood (SpO₂), measures of ventilation (end-tidal CO₂) and blood pressure, temperature and filling pressures of the heart. Such a recording method makes video analysis efficient as trauma resuscitation activities are initially guided by the goals of diagnosis of the causes of abnormality in the vital signs and normalization of vital signs.

Using the VASNET system, we established a video library of team performance during trauma patient resuscitation. The video library contains over 120 cases of real trauma patient resuscitation. These existing videotapes and other materials were used as stimulus material in this study of distributed decision-making. Aside from video and audio recordings, medical records (e.g. patient admission records, anesthetic records, discharge summary, vital signs, and blood chemistry) were also collected. After patient identifiers were removed, these were copied and became part of the database. A majority of these cases were reviewed by subject matter experts, both neutral (i.e. not in the recorded cases) and participant (i.e. in the recorded cases). A task analysis was carried out for the segment of airway management to generate a list of task landmarks and task performance measures (subjective ratings as well as task completion times) (Figure 5).

It is worth noting that the video images in this library were not recorded specifically for evaluation of remote diagnosis. To prepare stimulus materials, these case segments were extensively analyzed based not only on the video-audio recordings, but also on patient admission records, discharge summaries, and the transcripts from the interviews with the case participants while they reviewed the videotaped cases. The case analysis yielded the causes and rationale for patient status changes and resuscitation efforts. The stimulus materials used in the experiment contained:

1. audio-video recordings captured in real life (see Figure 4 for a screen dump from the video recordings),
2. continuous measurement of patient vital signs during the course of resuscitation, and
3. description of the patient history upon admission to the Shock Trauma Center. Patient history was given to the subjects at the beginning of each experimental session.

Stop points were chosen in each case segments based on the stages in the resuscitation effort. For each stop point, 1-3 items of descriptions were generated based on the analysis results to represent the ideal understanding of the status of the patient and of the resuscitation activities, and these items were used to score the questionnaires filled by the subjects. Thus even though questionnaires were generic, the scoring items were dependent on the specific stop point (see Table 2).

Four case segments (5-8 minutes each) were used in the experiment, with 3-4 stop points in each case segment. Table 2 describes the scoring items for all the stop points. These case segments were selected to represent a wide range of trauma patient resuscitation scenarios, and they were relatively complex.

The rationale for case selection was to find cases that allow testing of the abilities of the subjects as remote decision-makers to (1) anticipate out-of-viewing range events (2) identify team coordination problems (3) anticipate potential risky plans (4) track patient status dynamically (5) detect pressure for aggressive approach (6) identify failed task status (7) recognize positive and negative cues for task status (8) anticipate team's decisions (9) identify patient problems from trends in the patient' vital signs.

Experimental subjects were recruited from anesthesiology, surgery and nursing clinicians who were well experienced in resuscitating trauma patients. While answering questionnaires, the subjects were encouraged to think aloud and audio-recordings were made for later interpretation of experimental results. The subjects were given one practice run to familiarize themselves with viewing videotaped resuscitation and with answering questionnaires at stop points. One case segment was shown in each experimental session. At the end of the session, the subject was debriefed on the case and on the final outcome of the patient. The presenting order of case segments was randomized across subjects. The subjects volunteered their time for the experiments without direct compensation of any kinds.

Answers to questionnaires were used to measure the subjects’ ability to extract information from audio-video sources. See Table 2 for a list of questions in the questionnaire. For the eye-tracking experiment, no performance measures were made.

This experiment was designed to address the issue of what cues were detected and missed by experienced subjects. Questions to be answered by this experiment were: At what level can a remotely located expert function as a consultant? What are their abilities and limitations in such a role?

Segments from four cases were selected for this experiment. Overview of these cases are shown in Table 3. Three subjects went through a total of 12 experiment sessions (4 case segments each subject). Two subjects had one year and one subject 10 years of experience in the same Shock Trauma Center where the stimulus materials were videotaped. All were anesthesiologists. The subjects spent between 10 to 20 minutes at each stop point to complete questionnaires. The questionnaires were then scored against the left four columns in Table 2 (repeated in Table 4) and the results are in the right three columns of Table 4.

• Case 1, Stop Point One (SP1) and SP2. The "MAST-off" (MAST: military anti-shock trousers) event represents a major disturbance to the patient physiology and the care providers have to coordinate the timing of the event to prepare and to compensate for the disturbance. In this case, the event occurred outside the field of view of the video camera, but there were cues in the team activities indicating that the event was impending and occurred. None of the subjects anticipated the event and none of the subjects detected the event, either.

• Case 1, SP3. Establishing intravenous (IV) access was by tradition the responsibility of the surgeons. The slow progress by the surgeons was identified by one of the subjects, and none of the subjects detected that the anesthesia care providers (ACP) expanded their roles to compensate for the lack of efficient progress by the surgeons.

• Case 2, SP2 and SP3. Intubation is a process to protect the patient's airway from obstruction and prevent regurgitation of stomach contents into the lungs. In this case segment, the patient needed to be paralyzed for intubation, and the obstacle was that the team had not established IV access at the moment and therefore could not delivery paralyzing drugs via an IV port. All subjects at SP1 detected the obstacle to intubation. However, none of them detected the two parallel, circumventing plans being carried out (i.e. nasal intubation and intramuscular drug injection into the tongue), and none of them identified the danger of intubating an unparalyzed patient (e.g. the attempts at passage of the tracheal tube may cause reflex vomiting and aspiration into the lungs).

• Case 2, SP4. A patient is usually paralyzed within 60-90 seconds after the injection of paralyzing drugs. At SP4, 90 seconds after the injection of paralyzing drugs, patient was still exhibiting movement that identified that the patient was not paralyzed. None of the subjects detected such a delay nor did the subjects propose an explanation for the delay in onset of the paralyzing drug (which occurred because of shock and slow circulation) although the team members in real life detected the delay and commented on it.

• Case 4, SP3. Two of the subjects detected the very low and still decreasing blood pressure (BP) reading, but they chose not to intervene. In comparison, the anesthesiologist in the actual case intervened immediately after the stop point.

Among the items that one or more subjects scored, we will describe those in Case 3, partly because the case segment contained an error that could lead to fatal outcome. (This case was reported in Xiao, Mackenzie, & LOTAS, 1995 as an illustrative case to describe the findings on fixation errors.) All the subjects identified cues indicating that the endo-tracheal tube (ETT) was misplaced (at SP1). The actual case progressed from this point without correcting this error until after the last stop point (SP3). One of the subjects (S1) also identified cues indicating that the ETT may be placed correctly. At SP3, this subject, unlike the other two, did not propose correction of the error and removal of the ETT. It appears that this subject who noticed false confirmatory cues made a similar type of error in judgment as occurred in the real case (Xiao et al., 1995).

The think-aloud data revealed that the subjects used correlating information to compensate for lack of complete patient data. For example, when tachycardia (high heart rhythm) was observed (from the heart rate data, one of the most readily available patient monitoring data), the subjects inferred that the patient must be hypovolemic (low blood volume) given the type of injury. The subjects were also found to utilize secondary cues reflected in facial expressions of the resuscitation team. In Case 3, (in which the tube was passed into the esophagus) at SP1, one of the cues that the subjects mentioned was the hesitating and not-so-confident look in the face of the person who performed the laryngoscopy and intubation.

In debriefing, one of the subjects commented that the reason she missed several obvious cues was that she was concentrating on one line of activities on the video screen and did not notice other concurrent activities.

The results reported here were from a small sample. Nonetheless, these results lead to several interesting observations that can be used as hypotheses in guiding future efforts.

Among the possible reasons why the subjects did not score all the items, was that many important cues seemed to be missed by the subjects. Possible explanations for this are proposed here. Firstly, the verbalization and verbal communications in the stimulus materials were degraded by the background noise of a typical patient resuscitation setting. Secondly, the viewing range for the subject was restricted. Thirdly, the visual access was not secure because the people in the video often moved into the line of sight and sometimes obstructed the crucial viewing angle. Fourthly, typical video displays used in the experiment contained the activities of 3-5 people working on different aspects of resuscitation (e.g. cannulation, ventilating the patient, preparing syringes, etc.). The multiple action threads in the video screen may have overwhelmed the subjects and caused visual information overload.

It should be emphasized that all these factors are likely to be present in the circumstances under which distributed decision-makers will have to work and they are not simply a matter of implementation technology.

Another reason why the subjects did not score some of the items may be because of the out-of-control-loop phenomenon (e.g. Endsley & Kiris, 1995). In Case 2 at SP4, none of the subjects detected the delay in achieving muscle relaxation, whereas in the actual case, immediately after SP4, the team discussed the possible reasons why the patient was not paralyzed. It seems that the subjects did not have the anticipation of patient status changes due to an action (i.e. injection of paralyzing drugs) that they did not perform, and consequently did not detect the delay. In generalized terms, this hypothesis can be reworded as that the subjects did not have an adequate dynamic mental model of the patient status to guide their information searching process.

A third reason could be that the subjects did not have complete context information as the on-site care givers. In Case 4, SP3, the cue of the very low, decreasing BP did not trigger the intervention as it did in the real case. One subject offered the explanation that she did not intervene because she would like to know what other team members were doing (e.g. whether the surgeons were ready to start, which would stimulate the patient and could cause a rise in BP). This piece of context information was not readily available in the stimulus materials. The lack of complete context information may also explain the inability of the subject in anticipating potential significant events in Case 1, SP1-2 and Case 2, SP2-3.

Several limitations in the methods of all the current experiment should be pointed out here. First and foremost, the subjects did not have the opportunity to interact with the remote team and there were no two-way communications between the subjects and the remote team. In contrast with real remote diagnosis tasks, the subjects could not intervene in the resuscitation activities, and the remote team could not volunteer information. Second, the remote subjects had worked in the work environment where the stimulus materials were recorded and thus they knew the work environment very well. In real remote diagnosis tasks the ability to understand remote events and activities is likely to be less than what was observed here. Third, the stop points used in the experiment introduced pauses in the subjects' mental efforts. The questionnaire took 10-20 minutes to fill. The effect of such long delays was difficult to measure and assess.

In summary, the results from this preliminary experiment put forward the following possible reasons to explain the difficulties for a decision-maker to assess the situation and tasks at remote sites:

• Background noise, viewing range restriction, and insecure access to remote sites

• Visual information overload due to the multiple action threads at remote sites

• Lack of adequate dynamic mental models of remote events and activities

• Lack of context information about remote sites

The findings from this experiment have implications for the design of telecommunication systems in support of distributed decision making and for future empirical studies on remote diagnosis in dynamic task environments. Although the information carried in video is rich, our findings indicate that there is still a requirement to provide non-video supporting information to help users in comprehending video images and in compensating for the lack of complete data about remote events and activities. Video information alone will place serious limitations on an expert's ability to make situation assessment. In addition, when multiple, concurrent threads of activities are involved, a remote expert could be overloaded and miss important cues. A team of remote experts may be needed in such circumstances.

This experiment was a follow-up of Experiment 1 designed to address a hypothesized major contributing factor in influencing what cues were missed by subjects. In Experiment 1, it seemed that subjects with different experience background would dwell on certain region or certain lines of activities. Consequently, cues may have been missed, resulting in incomplete understanding of remote activities. An important characteristic of trauma patient resuscitation is that patient conditions change rapidly and many tasks have to be carried out simultaneously in a short period. During the initial phase of resuscitation, rapid situational assessment is needed, even though the patient’s condition is often uncertain. The resuscitation activities are multi-disciplinary efforts. Nurses, surgeons and anesthesiologists all actively participate in the diagnosis and treatment of the patient.

One of the questions regarding remote diagnosis with team activities is: what is the effect of the remote decision maker's experience background on his or her capabilities in extracting information from audio-video sources? For example, will a surgeon use the same strategies, as does a nurse (both are experts in their respective fields of expertise) in making remote diagnosis? Such questions have direct impact on the design of teleconsultation systems for situations where remote activities are multidisciplinary. We report here an experiment which was designed to simulate remote diagnosis tasks in which experts provide consultation through audio-video linkage.

Four trauma nurses, four trauma anesthesiologists, and four trauma surgeons participated in the experiments. All anesthesiologists and surgeons were at the level of attending faculty and all nurses were registered nurses with a minimum of two years of experience in trauma patient resuscitation. The four same case segments (5-8 minutes each), as those used in Experiment 1, were used.

The subjects were first briefed on the general goals of the project and signed a consent form approved by the Institutional Review Board. They were told that they were to function as a remote decision-maker through an audio-video-data linkage. One trial run was given to familiarize the subjects with the data collection procedures. Patient history was given to the subjects at the beginning of the presentation of each case segment. The presentation of the video recordings was paused at each stop point, at which the subjects filled in a questionnaire that contained four statements:

• I would use the following phrases to describe the current patient status
• I would use the following phrases to describe the current team activities:
• I would use the following phrases to describe the anticipated patient status:
• I would use the following phrases to describe the anticipated team objectives:

The questionnaire was a simplified version of that used in Experiment 1. The subjects were told that they could write down their answers, verbalize answers to an audio recorder, or do a combination of both. At the end of each case segment, the subject was debriefed on the case and on the final outcome of the patient. The order of case segment presentation was randomized across subjects.

The written and verbal answers from the subjects were keyed into a database. Two types of analysis were carried out: performance judged against an ideal understanding of the cases, and content of answers. In the performance analysis, the answers were scored according to a system which was developed to measure the subject's ability to assess the status of the patient and the progress of the resuscitation effort. In this scoring system, for each stop point, 1-3 items of descriptions were generated based on the analysis of the case segments to represent an `ideal understanding' of the status of the patient and of the resuscitation activities (Table 2). This ideal understanding was the result of extensive analysis of the selected cases by subject matter experts, all of whom were experienced trauma care providers. To ensure reliability, two analysts jointly scored the answers.

In the content analysis, the answers were categorized to compare the responses from the three groups of subjects (i.e. nurses, anesthesiologists, and surgeons). All written and verbal responses were included in the content analysis. Phrases in the answers were divided into the following categories:

• airway
• breathing
• circulation
• patient status and injuries
• team activities
• other

The airway category was further divided into general comments, progress of intubation (placement of the breathing tube, which is a central task in airway management), technique of intubation, specific steps of intubation, and verification (Figure 5). The categorization was based on the medical context (e.g. the steps involved in patient resuscitation) and on a review of all answers.For each subject and each case, the proportion of phrases falling into each of six categories was calculated. The mean proportion for each category was determined for each group.