Colin F. Mackenzie, MD

Yan Xiao, PhD

Peter F.M. Hu, MS, CNE

The LOTAS Group*

Judith Orasanu, PhD

*LOTAS Group: Colin Mackenzie (Chair) W. Bernhard, C. Boehm, J. Blenko, H. Cline, D. Donnelly, R. Dutton, P. Hu, M. Jaberi, M. Kalish, K. Mitchell and Y. Xiao

Prepared for

Ames Research Center

GRANT NCC2-921

("Remote Decision-Making for Trauma Patient Resuscitation", PI: Colin F. Mackenzie)

February, 1999

We would like to thank numerous people who have contributed to the project. Many of the clinicians in R Adams Cowley Shock Trauma Center at the University of Maryland worked as subject matter experts and experiment participants. Among them were the LOTAS (Level One Trauma Anesthesia Simulation) Group members (excluding authors of this report): W.N. Bernhard, J. Blenko, C. Boehm, H. Cline, D. Donnelly, R. Dutton, M. Jaberi, and F. Panico. Other anesthesiologists and nurse anesthetists participated in the projects: Drs. Fouche and Freeland and Mr. Baker. Among the Trauma Resuscitation Unit nurses who participated in the project, we would like to thank Paula Tasia, Harold Hardinger, and Victor Gustina. Four attending trauma surgeons participated as subjects in experiments and they were Drs. Tom Scalea, Philip Militello, and David Gens and Sharon Henry. Dr. Richard Horst and David Mahaffey, both of Man-Made Systems, participated in an early draft of the protocol.

We also like to thank several people who provided critiques and suggestions over the course of the project. They are Tom Nygren, Elliot Entin and David Serfaty..

Several research assistants provided invaluable contributions. They were Ben Harper, Denise Ovelgone, Sam Chang, Rick Spencer, Amaly Rahman, Fahd Habeeb, and Jacob Seagull. We would also like to thank Renee Kahn for her secretarial support and Paul Delaney for his able assistance in data analysis. In addition, Thelma McClellan typed this Final Report. Our thanks to all those who assisted on this project.

Distributed decision making is a research paradigm for understanding organizational, group, and team decision making when members are distributed in several senses: physical locations, access to information, authority, expertise and access to resources. Space missions are one of the prime examples of distributed decision making as collaborative members are distributed in all of these five senses. Another example, which is cognitively similar in that expert decision-makers are distributed in a team, is telementoring of trauma patient resuscitation from a remote location. We used previously acquired audio/videotapes of trauma patient resuscitation as experiment stimulus materials to examine how remote experts extract information from multi-media sources. We characterized the information used by remote expert decision-makers to make diagnostic judgments and interventions in emergencies occurring during trauma patient resuscitation.

Experiment 1 investigated what cues were detected and what was missed by remotely situated expert decision-makers.

The results showed that missed cues occurred for several reasons, including degradation of verbalizations and verbal communications because of background noise interference; viewing range for the remotely situated subject was restricted with a fixed camera location; visual access from this fixed location was not secure because care providers moved in and out of the camera line of sight and sometimes obstructed the view at critical moments when the cue was presented; typical video imagery used, showed the activities of 3-5 crew members of the trauma team working on patient resuscitation and such multiple actions, appeared to overwhelm the remotely situated expert decision-makers causing visual information overload.

Some clues were not picked up by all remote decision makers. Difficulties in recognizing these cues included: lack of an adequate dynamic mental model of patient status because they had not participated in patient care and were therefore cognitively "out-of-the loop" in regard to their information seeking; there was lack of context information in comparison to the on-site providers. It was not as obvious to the remotely situated person what other team members were doing or how to extrapolate their intentions; because not all concurrent activities could be simultaneously followed.

Nurses, surgeons and anesthesiologists understanding of the identical audio/video material was compared by responses to questions about the current patient status, team activities, future patient status and team objectives. The analysis of nurses, surgeons and anesthesiologists responses compared performance of correct answers against an ideal understanding of the cases and content of the answers categorized into airway, breathing circulation, patient status and injuries, team activities and other.

The performance analysis showed that anesthesiologists performed better than the other two groups. Performance scoring items that presented difficulties included detection of conflicting plans, and anticipation of nursing plans. Surgeons and nurses did poorly in determining task status of placement of a breathing tube, traditionally the responsibility of the anesthesiologists.

Content analysis showed that the distribution of answers across the six categories (above) among all three groups of subjects was similar. The surgeons provided more general comments, the anesthesiologist subjects used a higher proportion of phrases describing airway related issues whereas the nurses were consistently more focused on team work.

All subjects experienced, at one time or another, similar difficulties to those in experiment 1. An explanation for why anesthesiology subjects out performed the nurses and surgeons is that the videotapes selected for this experiment all contained the activities of airway management, a role performed by anesthesiologists on the trauma team. The traditional divisions of labor within the trauma team may have constrained the nurses and surgeons and prevented them from detecting critical cues. The nurses performed better than surgeons, this may have been motivational. The surgeons may have used different types of descriptions than nurses, but in general, surgeons provided fewer written and verbal responses than nurses or anesthesiologists.

These results suggest that experts with different experience backgrounds may appreciate different aspects of events and activities presented in audio/video sources.

Information extracted from video was identified by use of the eye tracker. Nurses, surgeons, anesthesiologists and medically naïve undergraduates (control group) participated. They provided verbal comments and answered questions about the current patient status and team activities and future patient status and team objectives, as in Experiment 2, while wearing the eye tracking device.

All subjects spent the majority of the time looking at the head and faces of the care providers on the video. The eye movements of the control group were rapid over large areas of the video in comparison to the expert subjects. Subjects with different experience backgrounds among nurses, surgeons and anesthesiologists had different visual scanning patterns. The distribution of total time spent on the area surrounding the patients’ head was just over 40% and on the care providers’ under 40% of the total viewing time. The nurse subjects scanned more around the patient and the anesthesiologists looked most at the airway manager. Viewing of the vital signs data occupied about 10% of the total viewing time.

The eye tracking data corroborated the hypothesis of the effect of experience background on information extraction during remote diagnosis.

As a result of understanding team coordination, a remote decision-maker might be able to identify decision points, high workload periods and problems in team coordination.

Videotapes were reviewed and several non-communication task coordination activities were noted including: following the protocols, following the leader, anticipation of future events, activity monitoring the task status of team members. Explicit verbal communications regarding situational assessment and future plans were relatively rare in comparison to non-verbal communication. When team members voluntarily provided their views, it occurred when the team was clearly at a decision point. There was considerable variation among team leaders in plan verbalization with some leaders providing clear intentions; while others appeared to let the events drive the team actions and the goals were inferred by these actions.

Coordination breakdowns occurred in a number of crisis situations including when: extreme difficulties or unexpected patient responses were encountered which prevented the implementation of routine procedures; the team was under pressure to seek alternative solutions; there were unexpected attempts to adopt novel solutions to acute emergency situations. These breakdown situations compromised the abilities of the supporting team members to provide assistance because of their lack of anticipation of the need. Coordination breakdown occurred when the patient was so unstable that the treatment plan had to be abandoned, such changes in plan occurred during crisis and under great time pressure, and required the team to change their process from diagnostic activities (hypothesis seeking) to action activities (hypothesis testing) rapidly.

Verbal communication was viewed as only one of many ways teams use to coordinate their activities. Other communication media include; activities, workspace, events and focus of attention of team members. In most circumstances, team coordination was achieved with a minimum of explicit verbal communication.

Forty videotaped cases were reviewed that identified a wide range of sources of uncertainty. A total of 76 uncertain items were identified by examining verbal communications and subject matter expert reviews. These uncertainties were categorized as patient related uncertainty (20%); and team/organization related uncertainty (41%). It seemed that lack of communication among team members and among personnel work in nursing, surgery, and anesthesiology contributes to many of the uncertainties identified. In addition, technological issues such as signal interference of patient vital sign monitors cause uncertainty in many crisis situations because of patient factors (low blood pressure, combativeness, etc.) cause signal detection failures. Lastly, because of overlap in task distribution among team members, uncertainties occur about who should do what and when.

The findings from these experiments have implications for the design of telecommunication systems in support of distributed decision-making.

Such objectives could be achieved by:

• Use of capture of individual care providers communications through a microphone worn during telecommunications.
• Use of directional microphones and noise cancellation technology.

The restrictions in viewing range and insecure viewing access could be overcome by:

• Use of multiple cameras including a pan/tilt/zoom camera providing a close-up detail and multiple fields of view.
• A fixed camera providing an overview of the entire area and team activities.

• Providing non-video supporting information to help remote decision-makers comprehend video images and to compensate for the lack of complete data about remote events and activities.

• Having a team of remote experts with different experience backgrounds to address concurrent tasks when remote activities are multi-disciplinary.

• Training of the remote decision-maker to appreciate tasks outside of their traditional domain specialty and division of labor in clinical practice.

• When situations evolve rapidly and multiple care providers are carrying out tasks, all of what is presented on video will not be perceived by the remote decision-maker.

• The cues missed by the remote decision-makers will be biased by their experience background.

• Training or cueing tele-consultants to systematically scan video sources may overcome such difficulties.

• The shared event space is an important media for non-verbal communication and the design of the workplace should not interfere with this.

• Improvement in uncertainty may be achieved by better intra-team communication (e.g. communication protocols at key points, checklists, etc.).
• Verbalization of explicit intentions and plans to all team members.

• Improvement in patient vital sign monitor technology to minimize signal detection interference.

• Detailed understanding of the cognitive activities involved in the medical domain is necessary for the best use of telecommunications technology in healthcare.

Distributed decision making is a research paradigm for understanding organizational, group, and team decision making when members are distributed in several senses including: physical location, access to information, authority, expertise and access to resources. Space missions are one of the prime examples of distributed decision making as collaborative members are distributed in all of these five senses. Telecommunication advances in recent decades have overcome many of the technical barriers to communication over distance and time. Increasingly telecommunication systems have become an integral part of many professions, enabling remotely located individuals to collaborate on problem-solving with expertise unavailable locally. Yet our understanding of how people work together when using communication technologies has been lacking (National Research Council, 1990; Rasmussen et al., 1991; U.S. Congress, Office of Technology Assessment, 1995). Interesting and challenging research issues arise in the use of telecommunication systems for decision making and problem solving.

Telecommunication advances enable remotely situated individuals to collaborate on problem solving with expertise that is not available locally. This project explores the cognitive demands and information use of decision-makers in medical diagnosis and treatment. Existing videotapes of real trauma patient resuscitation and management at the Shock Trauma Center were used as the stimulus material. There are clearly differences between the Space environment and a trauma center. In Space, there is protracted isolation and confinement. There can be physical deterioration of the astronaut due to weightlessness and illnesses. In the spacecraft, unlike the trauma center, there are finite supplies and limited and fixed resources. As a result, there are limited options available to deal with unanticipated events. However, using audio/video records of human processes in real life trauma patient resuscitation as surrogate material allows testing the understanding of decision making by remote experts, and examining how these experts view the multidisciplinary teams’ function in dynamic and stressful situations. In addition, such an approach can be used to identify the information that remote decision-makers can extract from audio/video records.

Like Space flight, there are many uncertainties confronting the trauma team decision-makers. There are unknowns about the patient (site and extent of injury, past medical history), and because emergencies are unpredictable, the incoming patient workload is unpredictable. In an analysis of the impact of uncertainty on trauma team performance (Xiao & Mackenzie, 1998), we found 76 uncertain items in 40 patient resuscitations. These were categorized as patient-related uncertainties (26%), (including reports provided by distributed pre-hospital team members, effect of treatment interventions and mechanism of patient injury). If, by analogy, the spacecraft were the patient, many of these factors would also probably be causes of uncertainty for the astronaut. The other major cause of uncertainty experienced by the trauma team was team/organization related uncertainties (41%) (including task distribution among team members, interaction of other team members, status of team members task accomplishment, and resource availability and schedules). In acute events, such as emergencies aboard the spacecraft, similar uncertainties will probably exist for the astronaut.

The domain of trauma resuscitation is used as a "laboratory" to develop and test general characteristics of how remotely situated decision-makers understand events in dynamic domains. Although the domain of trauma resuscitation is a highly specialized medical domain, it shares many similarities with Space missions as described above. In order to understand how real crew’s function in real, stressful situations, the "laboratory" we studied can be a valuable surrogate to provide insight into the Space environment. We paid special attention to the underlying theoretical concepts, such as task urgency, and uncertainty so that our results can be generalizable to other domains.

In addition to providing data that may yield insights into the cognitive processes involved in skilled performance and decision-making, these studies of distributed decision-making in trauma resuscitation may have medical applications of interest to NASA and the medical community (e.g. diagnosis and treatment of medical problems by telecommunication links).

While we do not yet have a thorough understanding of the knowledge structures, mental capabilities, allocation of attention, decision-making strategies, nor team interaction dynamics that predict successful performance in the resuscitation and treatment of trauma patients, our previous findings (see Mackenzie, et. al., 1993, 1994) suggest the possibility that the involvement of a remote expert, depending upon the information available to him/her, may assist the on-site team in avoiding certain pitfalls. For example, some errors in this task environment have been attributed to the team fixating inappropriately on suspected instrumentation problems, at the expense of continued observation and physical examination of the patient. A remotely located expert might be less prone to being caught up in such inappropriate allocations of collective attention. Trauma teams functioning in high stress, emergency cases have been shown to take procedural short cuts, which can be counterproductive, e.g., failing to make use of available instrumentation. Remote experts, to the extent that they retain a "big picture" perspective of the case, might more readily detect procedural oversights or other errors that are due to the stress of the moment rather than to lack of knowledge. Likewise, it may be easier for the more detached, remotely located expert to focus on trends in patient vital signs, and to formulate diagnostic conclusions therefrom, while the on-site decision-maker may be burdened with concurrent tasks to the extent that he/she only has the working memory capacity to monitor moment to moment.

These studies all demonstrate that in highly trained teams with experienced members, communication patterns varied and there are ways for leaders to exert influence without explicit communication. In contrast to many previous studies on leadership, verbal activities are usually the only ways in which leaders function. Such difference would have direct bearing on the potential impact of new communication technologies on leadership.

Verbal communications have often been studied as the major form of coordination process (Kanki, Folk, & Irwin, 1991). The concept of "implicit coordination" was introduced when teams were found to be able to coordinate with reduced communications (Serfaty, Entin, & Volpe, 1993), especially under high workload situations. To investigate factors promoting implicit coordination, it has been hypothesized that "shared mental models", or shared understanding of goals and tasks, is a key, since division of labor in most work settings may have prevented team members from understanding other people’s tasks. Volpe et al (1996) tested this hypothesis and found that cross training, in which team members were trained in other people’s tasks, improved team performance by prompting implicit coordination. The concepts of shared mental models and implicit coordination and related empirical data highlight the issue of communication cost. When workload and time pressure is high, reducing the cost or workload related to communication has obvious advantages. If it is important for team members to share an understanding of each other’s tasks and goals, which are relatively stable, it is equally important for team members to be aware of task situations and each other’s activities, plans and work focus, all of which are changing in dynamic work settings.

The advances in computation and communication technologies have introduced new ways to communicate, especially for those whose work is in office settings with documents as the primary work objects. The great potential in using new communication media has given rise to a research field "Computer Supported Cooperative Work" (CSCW) and a new line of products have emerged for facilitating work by multiple people: groupware. To guide the exploration of various technologies, Ellis, et al (1991) proposed a taxonomy to describe communication media: (1) same place or distributed across locations, and (2) synchronous or asynchronous. Electronic mail, for example, is a medium that allows exchange of information asynchronously among people at different places. In comparison, information exchange in face-to-face meetings occurs synchronously and at the same place.

What seems remarkable is that the "group task circumplex", proposed by McGrath in 1984 (described in McGrath & Hollingshead, 1994) was not specifically for addressing issues in group supporting technology, but provides explanation for much of the experimental results on the use of technology in groups (McLeod, 1992). McGrath (1990, 1991) and his colleagues (McGrath, et al, 1993; Straus and McGrath, 1994) considered the role of communication medium in the context of workgroups working together over time and carrying out tasks requiring different levels of coordination (e.g. idea generation, judgment, and multiple choice). Two major findings from their studies are (1) that when groups gain experience in working together, the need for communication is reduced and the importance of high-media communications (such as in face-to-face meetings and in video-conferencing) decreases and (2) that face-to-face communications benefit tasks requiring high levels of coordination.

Another major contribution of CSCW is a better understanding of the face-to-face medium. Through experiments on use of technology, a number of previously often overlooked aspects about face-to-face communications were emphasized. For example, face-to-face communications carry many informal and redundant cues through auditory and visual channels, and video conferencing systems often are unable to provide these cues (e.g., Krauss, et al, 1977). Even with the rather elaborate system described by Abel (1990), users still did not communicate in the same way as in the way in which they did at the same site (e.g. conversations were never as intimate as those carried out face-to-face). Using a survey methodology, Michailidis and Rada (1997) compared electronic mail, face-to-face, fax, post, and telephone in terms of coordination (commitment management, decision-making, awareness, communication, and transparency, perceptions). Face-to-face was the most effective mode of communication.

Trauma patient resuscitation is a specialized domain in which critically ill or injured patients are resuscitated and treated in a dedicated facility. (At the end of this report there is a glossary explaining the medical terms used in this report.) Prototypical activities of a resuscitation session start with a radio dispatch from the field. Initial examination and resuscitation is done in a patient admitting area, 10 of which make up the trauma resuscitation unit. If needed, the patient is then transported to an operating room for further procedures. The first goal of the resuscitation team is to maintain the life processes in the patient. A work domain analysis (Rasmussen et al, 1994, Ch.2) of the initial stages of trauma patient resuscitation produced a hierarchical representation of means-end relationship, which is shown as an abstraction hierarchy in Figure 1. The figure also describes the distribution of the responsibility among the team members.

The personnel in a typical team can be divided into three crews: surgical, anesthesia, and nursing, each with its own crew leader. The surgical crew has one or two surgeons, one of which is the assigned team leader, and an emergency medicine fellow. The anesthesia crew has a nurse anesthetist and one or two anesthesiologists. The nursing crew has one or two nurses and a trauma technician. In comparison to other work domains, such as command and control centers, the teams in the domain of trauma patient resuscitation has the following characteristics:

1. Team members are closely located and share the same physical workspace.
2. Team members are trained very differently (e.g. surgical versus anesthesia).
3. Teams may contain personnel under training and their supervisors, even in regular operations.
4. The composition of each team changes from case to case, and work schedules of each of the three crews may rotate differently.
5. Resuscitation teams have not gone through extensive training in teamwork, and the roles and job specifications for each team member are largely the result of tradition.

During the analysis of team activities, we found that it was important to define who the team leader is in the domain of patient resuscitation. Although every resuscitation team has an assigned team leader (the surgeon), he or she may not lead all resuscitation efforts. In particular, the anesthesia crew takes the full responsibility of airway management. During different stages of resuscitation different crews’ activities may dominate the team and the leader in the dominant crew leads the team’s activities. Thus the designation of team leader should be viewed as dynamic and task specific. In this report, all the references to team leader are made in this sense. Resuscitation teams were situated in the patient admitting areas (see Figure 2 for an overview of the areas) and were surrounded by resuscitation equipment (e.g. intravenous fluids and airway management and monitoring equipment).

Uncertainty and time-pressure are two of the major characteristics of the domain of trauma patient resuscitation. Patients presenting for emergency care often have unknown medical histories that may include pre-existing diseases and allergies. The extent, site, and mechanism of injuries may not be clear. While undergoing resuscitation, the patient’s condition can change rapidly and dramatically, the pace of resuscitation activities is dictated by patient responses. There is a very brief window of opportunity during which a great deal can be done to save the patient’s life. (For example, damage to the cerebral cortex can occur within a few minutes if the oxygen supply is inadequate.) Previous statistics have shown that about 80% of trauma deaths occur in the first four hours after injury (Brown, 1987). The time-pressure factor also compounds with the uncertainty, since the resuscitation team may not have the luxury of waiting for extensive patient monitoring information but have to act with what is available.

Our investigation included detailed examination of an important part of trauma patient resuscitation: the management of the patient’s airway (for breathing). Airway obstruction is a most rapid cause of death and protection of airway is the first priority in resuscitation of trauma patients. Adequate ventilation of the clear airway is the second priority and perfusion of the vital organs the third priority. Emergency airway management (known as intubation-see below) is challenging in the trauma patient due to time pressure and uncertainty about the patient’s injuries and because of labile vital signs. It poses an interesting context in which to study human performance. Intubation needs to be carried out expeditiously, because during attempts at the procedure, the patient receives no oxygen and is not ventilated. (Mackenzie, et al, 1996). The trauma anesthesiologist is the trauma resuscitation team member who is responsible for management of the airway, providing ventilation and maintenance of physiological values of heart rate, blood pressures and oxygenation. In addition, the anesthesiologist may induce anesthesia often before the patient is stabilized or during the resuscitative effort, to allow surgical correction of trauma induced injuries. The acute management of major trauma, therefore, provides a rare opportunity to study both individual and team performance under stress in a setting that shares some of the same cognitive milieu as military decision making.

The importance of studying performance of airway management is emphasized by the American Society of Anesthesiologists’ Closed Claims Project (Caplan, Posner, Ward and Cheyney, 1990; Cheyney Posner and Caplan 1991) which found that inadequate ventilation, esophageal intubation and difficult tracheal intubation were frequent causes of critical events during airway management. Those events are the most common mechanisms of respiratory related adverse outcomes. Among 2,046 cases with adverse outcome from anesthesia that were examined, 762 (37%), were associated with respiratory events; of these, 678 (89%), were problems associated with airway management. Many of these events involved human error, in that health care workers failed to recognize respiratory problems. In 300 cases with airway trauma, obstruction, aspiration, bronchospasm or pneumothorax, (all events seen with increasing frequency in trauma patients) the incidence of severe injury (brain damage or death) was 47% (141/300 patients).

Tracheal intubation is a sequence of steps to pass a tube (endotracheal tube or ET tube) through the patient’s mouth and vocal cords and into the trachea. Figure 3 is a photographic illustration of tracheal intubation. This procedure was chosen as the task focus of our efforts for several reasons. First, the activities involved in intubation are reasonably well-defined, and intubation has a clear start and finishing point. Second, comparisons could be made across different types of cases as it is used in circumstances with variable time pressure. Third, it occurs frequently enough to allow a reasonable number of the procedures to be recorded. Fourth, the process of intubation is generally considered stressful by clinicians because of the frequency and severity of adverse outcomes and it is high in workload.

Intubation is used when there is a need to protect the patient’s airway against aspiration of vomit and prevent obstruction. For many trauma patients, spontaneous breathing and self-protection of the airway are compromised by injury (for example, if the patient is in a coma or breathing is obstructed by blood). The first priority for emergency medical personnel is to ensure that the patient’s airway is unobstructed and the patient is ventilated adequately. In patients with a compromised airway or obstructed breathing, assurance of oxygenation and ventilation is achieved by intubation. The urgency for tracheal intubation in emergency care varies primarily due to the level of the patient’s spontaneous breathing efforts and the nature of the injury suffered by the patient. If the patient is breathing adequately and there are no indications to protect the airway, intubation may be delayed until the patient requires surgery.

The personnel involved in emergency care usually includes anesthesia care providers, surgeons, emergency medicine physicians, nurses, trauma technicians, and other specialists. It is the responsibility of the anesthesia care providers to perform intubation. While intubation is in progress, other caregivers may perform other tasks, such as the placement of intravenous lines. In this report the term "team" is used to denote the emergency care personnel directly involved in the care of the patient and present in the vicinity of the patient, although considerable independence exists for the sub-groups of surgical, anesthesia, and nursing crews who compose the trauma team.

Six steps are involved in intubation. The first step (pre-oxygenation) is to build up oxygen reserve in the lungs by providing the patient with 100% oxygen. The second step (induction) is to induce anesthesia and muscle paralysis by intravenous injection of drugs to suppress the patient’s gag reflexes to facilitate insertion of a tube into the trachea through the vocal cords. The third step (laryngoscopy) is visualization of the vocal cords through a special instrument (laryngoscope) inserted into the mouth so that the fourth step (tube insertion) of passing a tube through the vocal cords into the trachea can readily occur. Direct visualization may be difficult due to blood and secretion in the mouth and the throat, which may require suctioning. In case of inadequate direct visualization, there will be initial uncertainty as to where the tube is placed. The fifth step (verification}) is to verify the position of the tube in the trachea by listening to the patient’s chest (with a stethoscope) and by looking at patient monitors. It is essential to have the tube in the trachea as opposed to the esophagus or the bronchus. If the tube is mistakenly placed in the esophagus, which is next to the trachea, oxygen will be delivered to the stomach instead of the lungs and no ventilation and oxygenation is possible. If the tracheal tube is inserted too deep, it will pass into the bronchus and only one of the two lungs will be ventilated. The sixth step (connection) is to connect the tracheal tube to a mechanical ventilator (an automatic device that delivers oxygen and other gases, and removes exhaled carbon dioxide) and confirm that the ventilator is functioning by repeated examination of the chest or analysis of external gas for CO2. Using a mechanical ventilator frees the clinicians’ attention and hands so they can perform tasks other than patient ventilation.

We embarked on this project to address two research questions, one general to distributed decision making and the other specific to medical applications of distributed decision making.

As a first general question, the effort was directed at answering the question, what information is used by a remote decision-maker? For a decision-maker to effectively participate in a decision making process, a prerequisite is to be able to assess the situation and problems at hand. In a distributed decision making context, this requirement means that the decision maker has to rely on telecommunication links (e.g. computer, telephone, and video networks) to achieve situation assessment and to understand problems to be tackled. This requirement may be fulfilled relatively easily when events evolve slowly, but it can be difficult to satisfy when situations change rapidly (a similar argument is put forward by Allely, 1995). Little empirical data have been reported on how people can assess dynamically changing situations and problems through telecommunication links. Therefore, there is little empirical basis existing to guide the design of telecommunication systems in support of distributed decision-making.

A second question, specific to medical applications of distributed decision-making was also addressed, how should we make use of remote expertise? This question is related to a broadly defined field of telemedicine: the use of telecommunication technologies in the practice of medicine and healthcare. With the development of technologies, many of the long-time desires of medical practice seem to come true: the physicians can see and talk to the patient over long distances, physicians themselves can use video teleconferences to save travel costs. Much of the efforts on telemedicine have been driven by technology and have been based on untested assumptions about the impact of technology. As evidenced in the research on the impact of technology and on the use of video teleconferencing systems in organizations (e.g. Woods, et al, 1995; Finn, et al, 1997), each use of technology is an experimentation with unexpected outcome and creates a new work environment with new tasks and requirements. New modes of errors and new patterns of workload will result when technology is deployed, sometimes seemingly innocently replacing or automating a component in the work environment.

It is unclear what information a remote medical decision-maker requires to supervise management of medical emergencies and how effective remote management is at producing appropriate and timely diagnosis and management of humans with medical problems. It is also not known how different types of medical subject matter experts (surgeons, anesthesiologists, nurses) function as independent remote decision-makers and thirdly, how the response of the on-site trauma patient managers affects the remote decision-maker is also uncertain.

As a first step to address these research questions, our project examined the ability of trauma experts to remotely manage trauma patients through telecommunication links, and identify how telecommunication systems should be designed to facilitate such tasks. Important features of the domain of trauma patient resuscitation are that the patient’s condition changes rapidly and is often uncertain, and that the resuscitation effort is carried out by a multi-disciplinary team. Apart from being used as a research "laboratory", trauma patient resuscitation could benefit from telecommunication because in many situations injured patients are spatially remote from expert care providers

Using our existing videotapes and database (see below-under video library) including transcriptions of reviews of the management by participant and non-participant subject matter experts (SME's) and summaries of diagnostic and surgical findings and laboratory and radiological data, we examined the following specific aims:

1. Characterize the importance of various information providing factors in remote decision-making for the emergency management of the trauma patient. These results would address questions about the relative importance of patient vital signs (heart rate, blood pressure, oxygen saturation, etc.), and physical examination in determining appropriate emergency medical management of the trauma patient.
2. Assess the effects of different types of subject matter experts (surgeons, anesthesiologist, trauma nurse) functioning independently as the remote decision-makers. This specific aim would examine how strategies of information-gathering data interpretation and integration differ among medical subject matter experts working independently.
3. Determine how team coordination and breakdowns in coordination might impact on the decision-making of a remote expert and to identify what remote experts were uncertain about when viewing events through multimedia telecommunication links.

The studies were carried out in the R Adams Cowley Shock Trauma Center of the University of Maryland. This facility is a Level One trauma center that is regarded as one of the pre-eminent facilities of its kind in the world. As such, it serves as a training ground for trauma anesthesiology and surgery residents and faculty from all over the world. By drawing upon staff members involved in trauma anesthesiology surgery and nursing over the course of an approximately three-year period, we were able to gather data from individuals spanning a wide range of age, trauma treatment experience, and training backgrounds.

We used a comprehensive audio-video-data acquisition system (VASNET) to collect audio/videotapes in this real environment. Synchronization of various data sources to within 100th of a sec (e.g. audio, patient vital signs, and multiple images) occurred by means of the machine readable time code.

A unique feature of the video recordings was that the video images contained overlaid patient vital signs (Figure 4). The video images are overlaid with patient vital signs obtained from a serial interface on the patient’s monitors (Mackenzie, Hu, Horst, et al, 1995). These vital signs are essential to understanding of the decision-making process of the Resuscitation Team. They include heart rate, oxygen levels in the patients blood (SpO₂), measures of ventilation (end-tidal CO₂) and blood pressure, temperature and filling pressures of the heart. Such a recording method makes video analysis efficient as trauma resuscitation activities are initially guided by the goals of diagnosis of the causes of abnormality in the vital signs and normalization of vital signs.

Using the VASNET system, we established a video library of team performance during trauma patient resuscitation. The video library contains over 120 cases of real trauma patient resuscitation. These existing videotapes and other materials were used as stimulus material in this study of distributed decision-making. Aside from video and audio recordings, medical records (e.g. patient admission records, anesthetic records, discharge summary, vital signs, and blood chemistry) were also collected. After patient identifiers were removed, these were copied and became part of the database. A majority of these cases were reviewed by subject matter experts, both neutral (i.e. not in the recorded cases) and participant (i.e. in the recorded cases). A task analysis was carried out for the segment of airway management to generate a list of task landmarks and task performance measures (subjective ratings as well as task completion times) (Figure 5).

It is worth noting that the video images in this library were not recorded specifically for evaluation of remote diagnosis. To prepare stimulus materials, these case segments were extensively analyzed based not only on the video-audio recordings, but also on patient admission records, discharge summaries, and the transcripts from the interviews with the case participants while they reviewed the videotaped cases. The case analysis yielded the causes and rationale for patient status changes and resuscitation efforts. The stimulus materials used in the experiment contained:

1. audio-video recordings captured in real life (see Figure 4 for a screen dump from the video recordings),
2. continuous measurement of patient vital signs during the course of resuscitation, and
3. description of the patient history upon admission to the Shock Trauma Center. Patient history was given to the subjects at the beginning of each experimental session.

Stop points were chosen in each case segments based on the stages in the resuscitation effort. For each stop point, 1-3 items of descriptions were generated based on the analysis results to represent the ideal understanding of the status of the patient and of the resuscitation activities, and these items were used to score the questionnaires filled by the subjects. Thus even though questionnaires were generic, the scoring items were dependent on the specific stop point (see Table 2).

Four case segments (5-8 minutes each) were used in the experiment, with 3-4 stop points in each case segment. Table 2 describes the scoring items for all the stop points. These case segments were selected to represent a wide range of trauma patient resuscitation scenarios, and they were relatively complex.

The rationale for case selection was to find cases that allow testing of the abilities of the subjects as remote decision-makers to (1) anticipate out-of-viewing range events (2) identify team coordination problems (3) anticipate potential risky plans (4) track patient status dynamically (5) detect pressure for aggressive approach (6) identify failed task status (7) recognize positive and negative cues for task status (8) anticipate team's decisions (9) identify patient problems from trends in the patient' vital signs.

Experimental subjects were recruited from anesthesiology, surgery and nursing clinicians who were well experienced in resuscitating trauma patients. While answering questionnaires, the subjects were encouraged to think aloud and audio-recordings were made for later interpretation of experimental results. The subjects were given one practice run to familiarize themselves with viewing videotaped resuscitation and with answering questionnaires at stop points. One case segment was shown in each experimental session. At the end of the session, the subject was debriefed on the case and on the final outcome of the patient. The presenting order of case segments was randomized across subjects. The subjects volunteered their time for the experiments without direct compensation of any kinds.

Answers to questionnaires were used to measure the subjects’ ability to extract information from audio-video sources. See Table 2 for a list of questions in the questionnaire. For the eye-tracking experiment, no performance measures were made.

This experiment was designed to address the issue of what cues were detected and missed by experienced subjects. Questions to be answered by this experiment were: At what level can a remotely located expert function as a consultant? What are their abilities and limitations in such a role?

Segments from four cases were selected for this experiment. Overview of these cases are shown in Table 3. Three subjects went through a total of 12 experiment sessions (4 case segments each subject). Two subjects had one year and one subject 10 years of experience in the same Shock Trauma Center where the stimulus materials were videotaped. All were anesthesiologists. The subjects spent between 10 to 20 minutes at each stop point to complete questionnaires. The questionnaires were then scored against the left four columns in Table 2 (repeated in Table 4) and the results are in the right three columns of Table 4.

• Case 1, Stop Point One (SP1) and SP2. The "MAST-off" (MAST: military anti-shock trousers) event represents a major disturbance to the patient physiology and the care providers have to coordinate the timing of the event to prepare and to compensate for the disturbance. In this case, the event occurred outside the field of view of the video camera, but there were cues in the team activities indicating that the event was impending and occurred. None of the subjects anticipated the event and none of the subjects detected the event, either.

• Case 1, SP3. Establishing intravenous (IV) access was by tradition the responsibility of the surgeons. The slow progress by the surgeons was identified by one of the subjects, and none of the subjects detected that the anesthesia care providers (ACP) expanded their roles to compensate for the lack of efficient progress by the surgeons.

• Case 2, SP2 and SP3. Intubation is a process to protect the patient's airway from obstruction and prevent regurgitation of stomach contents into the lungs. In this case segment, the patient needed to be paralyzed for intubation, and the obstacle was that the team had not established IV access at the moment and therefore could not delivery paralyzing drugs via an IV port. All subjects at SP1 detected the obstacle to intubation. However, none of them detected the two parallel, circumventing plans being carried out (i.e. nasal intubation and intramuscular drug injection into the tongue), and none of them identified the danger of intubating an unparalyzed patient (e.g. the attempts at passage of the tracheal tube may cause reflex vomiting and aspiration into the lungs).

• Case 2, SP4. A patient is usually paralyzed within 60-90 seconds after the injection of paralyzing drugs. At SP4, 90 seconds after the injection of paralyzing drugs, patient was still exhibiting movement that identified that the patient was not paralyzed. None of the subjects detected such a delay nor did the subjects propose an explanation for the delay in onset of the paralyzing drug (which occurred because of shock and slow circulation) although the team members in real life detected the delay and commented on it.

• Case 4, SP3. Two of the subjects detected the very low and still decreasing blood pressure (BP) reading, but they chose not to intervene. In comparison, the anesthesiologist in the actual case intervened immediately after the stop point.

Among the items that one or more subjects scored, we will describe those in Case 3, partly because the case segment contained an error that could lead to fatal outcome. (This case was reported in Xiao, Mackenzie, & LOTAS, 1995 as an illustrative case to describe the findings on fixation errors.) All the subjects identified cues indicating that the endo-tracheal tube (ETT) was misplaced (at SP1). The actual case progressed from this point without correcting this error until after the last stop point (SP3). One of the subjects (S1) also identified cues indicating that the ETT may be placed correctly. At SP3, this subject, unlike the other two, did not propose correction of the error and removal of the ETT. It appears that this subject who noticed false confirmatory cues made a similar type of error in judgment as occurred in the real case (Xiao et al., 1995).

The think-aloud data revealed that the subjects used correlating information to compensate for lack of complete patient data. For example, when tachycardia (high heart rhythm) was observed (from the heart rate data, one of the most readily available patient monitoring data), the subjects inferred that the patient must be hypovolemic (low blood volume) given the type of injury. The subjects were also found to utilize secondary cues reflected in facial expressions of the resuscitation team. In Case 3, (in which the tube was passed into the esophagus) at SP1, one of the cues that the subjects mentioned was the hesitating and not-so-confident look in the face of the person who performed the laryngoscopy and intubation.

In debriefing, one of the subjects commented that the reason she missed several obvious cues was that she was concentrating on one line of activities on the video screen and did not notice other concurrent activities.

The results reported here were from a small sample. Nonetheless, these results lead to several interesting observations that can be used as hypotheses in guiding future efforts.

Among the possible reasons why the subjects did not score all the items, was that many important cues seemed to be missed by the subjects. Possible explanations for this are proposed here. Firstly, the verbalization and verbal communications in the stimulus materials were degraded by the background noise of a typical patient resuscitation setting. Secondly, the viewing range for the subject was restricted. Thirdly, the visual access was not secure because the people in the video often moved into the line of sight and sometimes obstructed the crucial viewing angle. Fourthly, typical video displays used in the experiment contained the activities of 3-5 people working on different aspects of resuscitation (e.g. cannulation, ventilating the patient, preparing syringes, etc.). The multiple action threads in the video screen may have overwhelmed the subjects and caused visual information overload.

It should be emphasized that all these factors are likely to be present in the circumstances under which distributed decision-makers will have to work and they are not simply a matter of implementation technology.

Another reason why the subjects did not score some of the items may be because of the out-of-control-loop phenomenon (e.g. Endsley & Kiris, 1995). In Case 2 at SP4, none of the subjects detected the delay in achieving muscle relaxation, whereas in the actual case, immediately after SP4, the team discussed the possible reasons why the patient was not paralyzed. It seems that the subjects did not have the anticipation of patient status changes due to an action (i.e. injection of paralyzing drugs) that they did not perform, and consequently did not detect the delay. In generalized terms, this hypothesis can be reworded as that the subjects did not have an adequate dynamic mental model of the patient status to guide their information searching process.

A third reason could be that the subjects did not have complete context information as the on-site care givers. In Case 4, SP3, the cue of the very low, decreasing BP did not trigger the intervention as it did in the real case. One subject offered the explanation that she did not intervene because she would like to know what other team members were doing (e.g. whether the surgeons were ready to start, which would stimulate the patient and could cause a rise in BP). This piece of context information was not readily available in the stimulus materials. The lack of complete context information may also explain the inability of the subject in anticipating potential significant events in Case 1, SP1-2 and Case 2, SP2-3.

Several limitations in the methods of all the current experiment should be pointed out here. First and foremost, the subjects did not have the opportunity to interact with the remote team and there were no two-way communications between the subjects and the remote team. In contrast with real remote diagnosis tasks, the subjects could not intervene in the resuscitation activities, and the remote team could not volunteer information. Second, the remote subjects had worked in the work environment where the stimulus materials were recorded and thus they knew the work environment very well. In real remote diagnosis tasks the ability to understand remote events and activities is likely to be less than what was observed here. Third, the stop points used in the experiment introduced pauses in the subjects' mental efforts. The questionnaire took 10-20 minutes to fill. The effect of such long delays was difficult to measure and assess.

In summary, the results from this preliminary experiment put forward the following possible reasons to explain the difficulties for a decision-maker to assess the situation and tasks at remote sites:

• Background noise, viewing range restriction, and insecure access to remote sites

• Visual information overload due to the multiple action threads at remote sites

• Lack of adequate dynamic mental models of remote events and activities

• Lack of context information about remote sites

The findings from this experiment have implications for the design of telecommunication systems in support of distributed decision making and for future empirical studies on remote diagnosis in dynamic task environments. Although the information carried in video is rich, our findings indicate that there is still a requirement to provide non-video supporting information to help users in comprehending video images and in compensating for the lack of complete data about remote events and activities. Video information alone will place serious limitations on an expert's ability to make situation assessment. In addition, when multiple, concurrent threads of activities are involved, a remote expert could be overloaded and miss important cues. A team of remote experts may be needed in such circumstances.

This experiment was a follow-up of Experiment 1 designed to address a hypothesized major contributing factor in influencing what cues were missed by subjects. In Experiment 1, it seemed that subjects with different experience background would dwell on certain region or certain lines of activities. Consequently, cues may have been missed, resulting in incomplete understanding of remote activities. An important characteristic of trauma patient resuscitation is that patient conditions change rapidly and many tasks have to be carried out simultaneously in a short period. During the initial phase of resuscitation, rapid situational assessment is needed, even though the patient’s condition is often uncertain. The resuscitation activities are multi-disciplinary efforts. Nurses, surgeons and anesthesiologists all actively participate in the diagnosis and treatment of the patient.

One of the questions regarding remote diagnosis with team activities is: what is the effect of the remote decision maker's experience background on his or her capabilities in extracting information from audio-video sources? For example, will a surgeon use the same strategies, as does a nurse (both are experts in their respective fields of expertise) in making remote diagnosis? Such questions have direct impact on the design of teleconsultation systems for situations where remote activities are multidisciplinary. We report here an experiment which was designed to simulate remote diagnosis tasks in which experts provide consultation through audio-video linkage.

Four trauma nurses, four trauma anesthesiologists, and four trauma surgeons participated in the experiments. All anesthesiologists and surgeons were at the level of attending faculty and all nurses were registered nurses with a minimum of two years of experience in trauma patient resuscitation. The four same case segments (5-8 minutes each), as those used in Experiment 1, were used.

The subjects were first briefed on the general goals of the project and signed a consent form approved by the Institutional Review Board. They were told that they were to function as a remote decision-maker through an audio-video-data linkage. One trial run was given to familiarize the subjects with the data collection procedures. Patient history was given to the subjects at the beginning of the presentation of each case segment. The presentation of the video recordings was paused at each stop point, at which the subjects filled in a questionnaire that contained four statements:

• I would use the following phrases to describe the current patient status
• I would use the following phrases to describe the current team activities:
• I would use the following phrases to describe the anticipated patient status:
• I would use the following phrases to describe the anticipated team objectives:

The questionnaire was a simplified version of that used in Experiment 1. The subjects were told that they could write down their answers, verbalize answers to an audio recorder, or do a combination of both. At the end of each case segment, the subject was debriefed on the case and on the final outcome of the patient. The order of case segment presentation was randomized across subjects.

The written and verbal answers from the subjects were keyed into a database. Two types of analysis were carried out: performance judged against an ideal understanding of the cases, and content of answers. In the performance analysis, the answers were scored according to a system which was developed to measure the subject's ability to assess the status of the patient and the progress of the resuscitation effort. In this scoring system, for each stop point, 1-3 items of descriptions were generated based on the analysis of the case segments to represent an `ideal understanding' of the status of the patient and of the resuscitation activities (Table 2). This ideal understanding was the result of extensive analysis of the selected cases by subject matter experts, all of whom were experienced trauma care providers. To ensure reliability, two analysts jointly scored the answers.

In the content analysis, the answers were categorized to compare the responses from the three groups of subjects (i.e. nurses, anesthesiologists, and surgeons). All written and verbal responses were included in the content analysis. Phrases in the answers were divided into the following categories:

• airway
• breathing
• circulation
• patient status and injuries
• team activities
• other

The airway category was further divided into general comments, progress of intubation (placement of the breathing tube, which is a central task in airway management), technique of intubation, specific steps of intubation, and verification (Figure 5). The categorization was based on the medical context (e.g. the steps involved in patient resuscitation) and on a review of all answers.For each subject and each case, the proportion of phrases falling into each of six categories was calculated. The mean proportion for each category was determined for each group.

The content analysis showed that the distribution of answers across the six categories from all three groups followed a similar pattern (Figure 6). In cases 2 and 3, anesthesiologist subjects used a higher proportion of phrases to describe airway related issues, whereas in case 1, surgeon subjects provided a higher proportion of phrases related to airway. However, when the subcategories of airway were examined (Figure 7), many of the phrases provided by the surgeon subjects were general comments (e.g. "the patient needs an airway"), rather than specifics involved in airway management. In contrast, anesthesiologist subjects used phrases more specific in relation to steps (e.g. "why is cricoid pressure not continuous") and techniques (e.g. "change to oral intubation") than the other groups. Nurse subjects were consistently more focused on teamwork than the other subject groups, while anesthesiologist subjects were consistently less focused on teamwork.

Overall, all subjects experienced the difficulties as found in Experiment 1. When the performance of the three subject groups are compared, surgeon subjects and nurse subjects seem to have more difficulties in detecting critical cues from the audio-video sources. One explanation is offered here. The cases selected for this experiment all contained the activities of airway management, a task primarily assigned to anesthesiologists. The traditional division of labor may have constrained the attention of the surgeon and nurse subjects. For example, the surgeon subjects used a smaller proportion of phrases to describe tasks and patient status related to specific aspects of airway management. The anesthesiologist subjects, by the nature of their experience background, appreciated the tasks involved in airway management more than did the other two groups and picked up more critical cues as a result.

A surprising result was that the nurse subjects seem to perform better than did the surgeon subjects. The level of motivation was not controlled in the current experiment and may partly explain such findings. Surgeon subjects tended to provide fewer written and verbal responses to the questionnaires than did nurse subjects. Another explanation would be that the surgeon subjects used different types of description than those used by the nurse subjects and anesthesiologist subjects, but such differences were not detected by the analysis methods used here.

It is interesting to note that there were relatively fewer phrases related to teamwork used by the anesthesiologist subjects. Although no direct measurement of eye gaze was made and no objective data were available in this experiment, anesthesiologist subjects seemed to focus on airway management and thus paid less attention to other activities. This may explain their apparent infrequent reference to teamwork. Their focus during the experiment may have been restricted by their specialized contribution to trauma patient resuscitation of airway management.

The results suggest that experts with different experience backgrounds may appreciate different aspects of the events and activities presented in audio-video sources, although future experiments are needed to expand the selection of case segments to include situations where the surgical experts are expected to out-perform the anesthesiology experts. If such conclusion can be supported as suggested by the present experiments, two design guidelines can be recommended: 1) a team of experts with varying experience background is needed for addressing various aspects of the tasks when the remote events and activities are multidisciplinary; and 2) special training is needed for the remote decision maker to appreciate tasks outside his or her specialty and the boundaries of traditional divisions of labor.

This experiment was designed to evaluate video as a medium to convey information to a remote decision-maker. As a versatile and powerful communication medium, video has often been considered as a key to the ultimate objective of "being there" or "tele-presence" (Ellis et al., 1991). However, what information people actually extract from video is not well understood. What are remote experts' abilities and limitations when provided with video access? Can a remote expert effectively participate in a joint decision making task when situations change rapidly? Does a remote expert possess the same ability to anticipate needed interventions as that of an on-site expert?

Answers to these questions have direct impact on the design of collaborative systems (e.g. telemedicine systems) and on the specific training requirements for experts to function as tele-consultants. In contrast to increasingly wide usage of video in distributed decision-making, there are few published reports on the role of video in assessing remote situations (remote diagnosis).

In Experiments 1 and 2, we found that (1) numerous critical visual cues were missed by remote experts and (2) experts with different training and experience background would focus on different aspects of a remote, multidisciplinary task. These findings were based on the analysis of verbal reports from subjects performing remote diagnosis tasks. It was difficult to determine what information was actually picked up from video sources by the subjects. For example, it is possible that a domain specific expert only looks at activities and events on the video which correspond to his or her own expertise domains, with detrimental effect on monitoring of overall activities and events. Consequently, a remotely located expert, even provided with an overview of the remote activities, may only still focus on a portion of the overall activities and might not be able to provide decision support for the whole on-site team.

In Experiment 3, an eye-tracking device was used to determine visual scanning patterns of domain expert observers. Although eye-tracking devices have been used in human performance research (e.g. Harris & Christhilf, 1980; Sirevaag et al., 1993), studies of visual scanning patterns during video viewing have rarely been reported, with the exceptions of studies in sports by Ripoll and colleagues (Ripoll, 1988, on volleyball players and Ripoll et al., 1995, on boxers).

Subjects were presented with stimulus materials of video recordings of real trauma patient resuscitation and were requested to answer two questions: What was the patient's status and the team's activities? What would the patient's status and team's objectives be in the immediate future?

Apparatus. The eye-tracking device used was from ASL (model 4000U, Boston, MA). The stationary scene camera option was used, with a head-tracking device so that the subjects could move their heads relatively freely. Due to practical circumstances, the experiments were conducted at two locations. With the exception of the presenting video monitor, the equipment setup was identical.

Video recordings of stimulus materials were displayed on a 20 inch video monitor (viewing area: 40x30 cm) at location A and a 29 inch video monitor (viewing area: 56x42 cm) at location B. The distances between subjects' eyes and video monitors were around 60 cm (location A) and 120 cm (location B). The horizontal viewing angles were estimated between 25 to 40 degrees. A computer program was used to control the timing of video presentation, so the start and end times of presentation were the same for all subjects.

Both analog and digital outputs from the eye-tracking device were recorded. Analog outputs were in the format of VHS video, which contained the original video of patient resuscitation overlaid with eye gaze positions (represented as a white cross hair or a white cursor). The digital outputs included time, fixation coordinates, and fixation duration.

Subjects. In this experiment, medically naive undergraduate students were recruited as a control group, in addition to experienced clinicians. All subjects participated as volunteers with no monetary reward.

Procedure. The stimulus materials presented to the subjects were two real trauma patient resuscitation cases videotaped in the admitting areas of the Shock Trauma Center. The video recordings were obtained from a fixed camera angle with mono-audio. Domain expert subjects were familiar to the settings where the videotapes were acquired. They were, however, ignorant of the cases to be presented since (1) the subjects were not the care providers in the selected cases, (2) the cases occurred four years before these experiments, and (3) the subjects had never viewed the video recordings before.

The patient in Case I was found unconscious in a burning building, with second and third degree burns on the buttocks and the back. The patient in Case II had gunshot wounds to the head and the neck that required cardio-pulmonary resuscitation in the field. The two cases were analyzed in terms of significant clinical events and a short video segment was selected from each case. Both selected segments showed the initial patient resuscitation immediately after the patient was admitted to the trauma center. The video segment for Case I lasted 5'7", and Case II 2'46". The video presentation was paused briefly three and two times (same points for all subjects) for Case I and Case II, respectively, to collect subjects' comments and verbal response to questions. As a result, there were four and three data collection periods for Case I and Case II, respectively.

Before an experiment started, written materials were given to the subject to describe the purpose of the experiment. The layout of the screen was explained to each subject with sample video scene screens. The eye-tracking device was calibrated for each subject.

Although the subjects were requested to provide verbal comments and answer questions on the events and activities in the video they were viewing, the verbal responses were not used in the analysis of eye gaze data and are not described here.

The analog (video images) and digital (eye-gaze coordinates) data from the eye-tracking device were analyzed separately. For analog data, the eye-gaze positions were manually coded, whereas digital data contained fixation timings and coordinates and were ready for processing.

Analog data and manual coding were necessary because dynamic images were displayed to the subjects. It was not possible, for example, to determine from the digital outputs of the eye-tracking device what exactly the subject was looking at. Digital Data: Two measures of visual scanning patterns were chosen for the analysis of the digital data: fixation and dwell. A fixation is a cessation of apparent eye movement; a dwell is a consecutive sequence of fixations within a given area of interest (AOI). The subjects may have numerous fixations within a dwell. Therefore the dwell time (i.e. the time duration between the start of the first fixation of the dwell and the end of the last fixation) is an indication of how focused the subjects were on a particular AOI. Longer dwell time at an AOI would be interpreted as more focused on that AOI.

Five rectangles of the video images, each representing discrete AOIs, were drawn for the purpose of deriving duration of dwells (Figure 8). These areas of interest were chosen to correspond as closely as possible with both a priori identified areas of interest, as well as clusters of fixation points revealed by a review of all of the participants' scanning patterns on analog outputs.

Analog data. The overlaid eye-gaze positions were coded by reviewing the analog output (video tapes) using a touch coder which obtained time information directly from the videotape playing equipment. Seven codes were assigned to each eye fixation: (1) patient head and the adjacent area where airway management activities occurred (e.g. face-mask and ventilation bag), (2) airway manager (either hand, head, or face), an anesthesia care provider who managed the patient's airway at the patient's head, (3) other care providers (either hand, head, or face), (4) vital sign strip, which was overlaid on top of the video images, (5) anesthesia equipment, which included the ventilator, anesthesia machine and work bench, (6) IV equipment, which included intravenous fluid bags, stands, and warmer, and (7) miscellaneous: the rest of the areas. These seven codes are illustrated in Figure 9.

Apart from the total eye-gaze time spent on each of the seven objects, an eye-fixation sequence measure was developed: excursion length, which was defined as the number of fixated objects intervening between fixations on a given object. For example, suppose a subject looks at A, B, C, B, and A. The excursion length from object A would be 3 (i.e. three intervening fixated objects: B, C and B).

A total of 12 subjects (four attending anesthesiologists, three attending trauma surgeons, three trauma nurses, and two medically naive students as control) participated in the experiment. The digital data of the first data collection period (30-sec) for Case I were corrupted for two subjects and were not included for analysis.

Analog Data. The analog data (in the form of overlaid video) were reviewed to assess qualitatively the visual scanning patterns. Two observations were made: (1) subjects seemed to spend the majority of their fixations looking at the head and face of the care providers on the video, regardless of which group the subjects were from; (2) the fixation of the novice subjects seemed to move rapidly over large areas of the video in comparison to expert subjects. Figure 10 showed visual scanning paths for one segment. It seemed apparent that subjects with different experience background had different visual scanning patterns.

The analog outputs from seven subjects (three anesthesiologists, two nurses, and two surgeons) for the first segment of Case I were coded according to the coding definition in Figure 9. Figure 11 shows the distribution of total time spent on each of the seven objects by anesthesiologist subjects, nurse subjects and surgeon subjects. No statistical tests were applied due to the limited size of data. Over 40% of the time was spent on the area surrounding the patient's head, and just under 40% of the time on care providers (the airway manager plus other care providers). Compared to other two groups of subjects, the surgeon subjects seemed to look less at the airway manager and more at the patient's head and other care providers.

Since all subjects spent about 40% of the time on the area surrounding the patient's head, this area was chosen as the base to calculate the excursion length of eye fixation sequences, as defined above. The results are in Figure 12. The averaged excursion length for all seven subjects was 2.07 (1.2). Nurse subjects seemed have the longest excursion length among the three groups, suggesting that they scanned more around the patient than the surgeons and the anesthesiologists.

Digital Data. Fixation: Anesthesiologists (median: 0.28 s) had the longest fixations and control subjects (median: 0.2 s) the shortest. Kruskal-Wallis one way ANOVA on ranks indicated significant differences (p < 0.05) among the four groups of subjects. Pairwise comparison (the Dunn procedure) showed that except surgeons (median: 0.25 s) versus nurses (median: 0.25 s), all pairwise comparisons were significant ( p < 0.05). Dwell: The three expert groups followed similar patterns of distribution of time spent on the six AOIs across the two cases (Figure 13). All groups spent more time at AOI anesthesia work areas (labeled as B), where the faces and heads of care providers would appear during the majority of time, and airway (labeled as E), where airway management activities and the patient's head usually would be.

Patient vital signs, displayed in AOI A, attracted about 10% of the subjects eye-gaze time. Interestingly, anesthesiologists looked at the vital signs for a smaller proportion of time than the other two expert groups did. Two-way ANOVA on subject groups AOIs indicated significant interaction (p < 0:001) for both cases. Within AOI pairwise comparisons were made to assess the differences in three AOIs: vital signs (A), anesthesia work area (B), and airway (E).

When only the second period (instead of the entire case) was selected for similar analysis (Figure 14), the anesthesiologist subjects' focus on the airway AOI was very prominent. They spent over 50% of the time during this period looking at airway management (in the region labeled E).

In Figure 15 are the results of median dwell time for each of the three expert groups. The results from the two cases show similar patterns. The surgeons focused more (p < 0:05 for both cases) at the AOI airway during the first period and the anesthesiologists focused more (p < 0:05) for Case II) at the AOI airway during the second period.

The traditional roles for the three expert groups are that surgeons are responsible for overall activities; anesthesiologists for airway management; and nurses for supporting roles. Consequently, it was expected that

• the surgeon subjects would spend time evenly over various activities including patient vital sign observation, and make many transitions among the occurring activities
• the anesthesiologist subjects would spend time primarily in and focused on airway management
• the nurse subjects would spend time over assisting activities occurring in areas such as intravenous bags, patient torso, and would spend time evenly over various activities

The distribution of total dwell time over AOIs and the focusedness measure (i.e. the median dwell time) based on the eye-gaze data had face validity corresponding to the above expectations. For example, the surgeon subjects first surveyed the patient airway status and then focused less on the airway, as confirmed in Figure 15. In comparison, the anesthesiologists first looked at overall activities and then focused on the airway. The excursion by length, a measure developed here, also indicated that the nurse subjects looked around more frequently possibly to obtain information related to their usual coordinating and supporting roles.

The visual scanning patterns reported here corroborated several findings from Experiments 1 and 2. First, subjects in previous experiments indicated that they used the facial expression of the care providers on the video as secondary information to obtain such information as the confidence level of the care providers in their assessment of the patient. Visual scanning patterns indeed indicated that the subjects spent a considerable amount of time looking at the faces and heads of the care-providers in the video.

Second, although the subjects in the three expert groups spent about the same amount of time at each of the areas of interest and the objects defined, they exhibited a different focus in terms of how long the subjects looked at one AOI or object before moving to others. The anesthesiologists, for example, focused on airway management (their primary role in a trauma team) more at the second period (Figure 15) during which airway management was carried out, at the expense of looking less at the patient's vital signs (Figures 13 and 14). These findings support strongly the effect of background experience found in Experiment 2.

Differences in fixation duration between the novice control subjects and expert subjects showed the impact of expertise in general on visual scanning patterns. Without being able to appreciate the events and activities, fixation of the novice subjects wandered in quick successions over the entire area of the video monitor.

The current study has implications to applications where decision-makers provide consultations over audio-video linkages. When situations evolve rapidly and tasks are carried out by multiple people, one cannot assume that what is presented on video will be perceived by the remote decision-makers. Furthermore, the visual cues missed by the remote decision-makers will likely be biased by their experience background. Strategies countering such tendencies are therefore indicated. In addition to those suggested previously as a result of Experiment 2, interventions such as training or cueing tele-consultants to systematically scan the video sources. Cueing could be deployed to enhance the effectiveness of distributed decision-making.

The current study has several methodological limitations, which need to be pointed out here. Although real-life video recordings and domain experts were used in the study, the task of remote diagnosis was simulated in the sense that no two-way interaction took place between on-site care providers and remote decision-makers. The camera view angle was relatively limited. (Indeed multiple camera views are possible in remote diagnosis. However, that will only exacerbate the workload involved). The tools for analyzing eye-gaze data were still primitive, considering that video scenes change over time.

In summary of Experiment 3, eye-tracking devices were found to be an effective tool for studies of remote diagnosis task. In our experiment, the eye-tracking device provided quantitative data on the information extraction processes, which to study previously, one had relied on verbal reports. The visual scanning patterns corroborated the hypothesis of the effect of experience background during remote diagnosis. Although eye gaze data can provide a wealth of information about the underlying cognitive process in many tasks (c.f. Senders, 1980), further development of data analysis methods for eye gaze data are sorely needed for effective exploitation of these techniques.

To examine how a remote decision-maker could collaborate as an important member of the distributed trauma resuscitation team, we conducted two experiments to identify collaboration in such a complex medical system. We believe that to understand how a remotely situated team member would collaborate through multi-media telecommunication links requires some understanding of how teams collaborate in this domain. The first study examined how the trauma team coordinated and when coordination breakdown occurred and the second study examined uncertainty in resuscitation and team communication. We used the existing video library and previous video analysis that included:

1. audio commentary by subject matter experts
2. coding of verbal communications
3. event flow analysis of selected cases (see Figure 16 for an example)
4. performance evaluation using a normative task model.

Reviews by subject matter experts provided insight into the potential cognitive processes involved in making diagnosis, plans, and decisions, and have become an important data source themselves.

Of special interest to the topic of collaboration is event flow analysis, which is a process of constructing detailed event flows with hypotheses or theories of what might contribute to the underlying cognitive activities. To facilitate video analysis, a system was developed, VINA – (Figure 17), which linked video with transcriptions of verbal communications, codings of events, and patient vital signs.

This study was driven by the fundamental question of how it was possible for team members to function so smoothly most of the time with little apparent effort spent on coordination. The analysis taken to address this question was qualitative, aimed to:

1. categorize ways in which team coordination was achieved, and
2. hypothesize the nature of breakdowns in team coordination.

Three types of critical incidents were analyzed: decision points, high workload periods, and apparent problems in team coordination.

The findings related to coordination strategies that were used by resuscitation teams are reported in two separate areas: task coordination, or the distribution and delegation of tasks, and information flow, or the passage of information regarding patient status and contingency plans.

During the course of resuscitating a trauma patient, many physical tasks were performed. Some of them had to be coordinated among team members within a crew or across crews. This was so either because the tasks needed synchronous effort from multiple people (e.g., lifting the patient), or because the tasks relied on preconditions (e.g., suctioning equipment must be ready before usage), or because multiple tasks need to be accomplished within a short period of time (e.g., establish the airway and restore blood circulation).

Several forms of non-communication task coordination activities were noted in video analysis. Four of them are listed below.

These strategies of task coordination, without the use of explicit, verbal or gestural, communications enabled the resuscitation teams to perform smoothly in most situations.

One of the most interesting aspects of team coordination is the explicit, verbal communications regarding situational assessment and future plans, even though, such communications were relatively rare (Figure 18). In the situations where such information flow was detected, we found most of them had clear indications that the team was at a decision point. The team members voluntarily provided their views of the situation based on the decisions that the teams were facing at the time. For example in Figure 18, in one case when the patient was still not paralyzed 90 seconds (the usual duration) after the injection of drugs, several team members, without request, provided their assessment of the patient condition and of the reasons why the patient had not been paralyzed. In another case, while an anesthesia care provider was determining whether the patient was receiving oxygen, the surgeon provided his assessment of the situation unsolicited by saying "the patient was stable".

The amount of verbal communications varied greatly among different teams. Some team leaders verbalized their plans clearly while other team leaders let the team members to infer their goals and intentions through actions.

Figure 20. Coordination breakdowns when a sudden change of action occurred. N, S, and ACP represent three lines of activities of the nursing, surgical, and anesthesia crews, respectively.

To summarize the strategies of team coordination, verbal communications can be viewed as one of many media that the team used to communicate. These other types of communications media include, in addition to utterance and explicit gestures, (1) activities, (2) workspace, (3) events, (4) foci of attention. These media were possible because team members worked in closed physical workspaces. Although not sufficient in all occasions, they provide an efficient means for the team to coordinate. A distributed Team member should be trained to appreciate use of these non-verbal means of communication as secondary aids in remote decision-making.

The coordination break downs that our video analysis identified can be described in the following four forms: (1) conflicting plans, (2) inadequate support in crisis situations, (3) inadequate verbalization of problems, and (4) lack of task delegation. Their occurrence indicates gaps between what was needed and what the team had done in terms of team coordination.

The video recordings in our study show that team coordination was achieved in most situations with minimum explicit, verbal communication. When team coordination broke down, it often occurred in situations where there was a lack of explicit communication. In the following, we evaluate these findings against three previous studies done by Serfaty et al (1993), Orasanu (1990), and Segal (1994).

In studies of team coordination patterns under stressful and unstressful situations, Serfaty et al (1993) found that high performance teams were able to adapt their coordination strategies in stressful situations to reduce the cost of explicit communications. It appears that the teams in our study had adapted to the implicit coordination due to the high workload in many situations. Although no quantitative comparison was made between high stress and low stress situations, our observations show that in non-stressful situations, verbal communications contained considerable amount of non-essential information, some of which did not relate directly to the case involved. Such an adaptation could probably be better explained by the adaptation of workload management, as described by Sperandio (1971) in his analysis of communications between air traffic controllers and pilots.

Orasanu (1990) also contrasts team activities between high and low performance teams. Her major finding was that the content of communications was different between high and low performance teams. High performance teams communicated explicitly about problems and plans. However, the small amount of verbal communications in the cases that we recorded did not allow us to compare across different scenarios.

Segal’s study (1994) of non-verbal communications had similar findings to ours. He found that visual monitoring of team members’ activities was an important part of team coordination. Through the analysis of visual checking patterns, Segal provided quantitative data to support the notion that visible activity is an essential part of teamwork.

There are several implications of our findings for workplace design. Similar to what Segal (1994) argues, one has to beware of implicit communication channels, as they had important roles in team coordination in our studies. Practitioners utilized various non-verbal media for coordination: through activities monitoring and through shared event space. These media have important functional roles, including allowing team members to compensate for team mates and to schedule their own activities. The ability to monitoring on-going activities and events also enable the team to have a coherent shared mental models (Cannon-Bowers et al, 1991; Orasanu, 1990), thus team members could provide needed information and support without an explicit request.

Our findings also provide guidance to studies of team activities in simulated environments. On the one hand, the current study highlights the importance of non-verbal communications and various types of medium used in communication. Stripping these methods of communication away in a laboratory study, for example, could dramatically change how a team coordinates and imposes extra workload on the team. Consequently, the problems in coordination observed in such a simulated setting may have a very limited validity in settings like emergency rooms. On the other hand, the three types of scenarios where coordination breakdowns were observed could lead investigators to focus on these scenarios and understand more about coordination breakdowns.

A key characteristic of trauma patient resuscitation is the uncertainty involved: many unknowns about the patient and the incoming workload are unpredictable. An analysis was conducted to determine the impact of uncertainty on team performance.

Because of the availability of video recordings, it was possible to view these case segments repeatedly and to compare with the review comments provided retrospectively by the case participants. The analysis was carried out in order to answer the question of "What is uncertain to the team specifically related to this case segment?" A list of items uncertain to the team was generated for each case segment and the items from all cases were sorted.

A wide range of sources of uncertainty was identified. To illustrate how various types of uncertainty may arise during resuscitation, one case segment examined is described here:

Based on the analysis of all case segments, a list of ways in which uncertainties occurred during resuscitation, either mentioned directly by subject matter expert reviewers, or identified by analysts, was summarized here:

• The mechanism and the extent of injury.
• The patient’s prior medical history.
• Working status of patient monitors. In comparison to many industrial settings, the monitors used during resuscitation frequently produce artifactual readings for various reasons. For example, several case segments involved displaced patient monitor probes.
• The effect of treatment. In the case segment described above, the patient was in shock with reduced blood circulation and was hemorrhaging. It was difficult to predict drug effects and identify the delays in onset under such circumstances.
• The availability of team members. There were several case segments in which the team was waiting for a member, uncertain when he would arrive. Possibilities for such uncertainty include that the members may have been with other patients. Concurrent patient admissions also made it difficult in several cases to predict the availability of team members, and consequently to decide whether to take measures to ameliorate potential adverse effects of reduced personnel resources.
• Task distribution among team members. Although each team member in the studied center has a nominally defined role, many case segments involved "negotiation" of who should do what. The frequent changes in team composition, the presence of personnel in training, and occasionally over-staffed resuscitation teams were three of the possible reasons.
• The intention of other team members.
• The resources and schedules of other parts of the institution (e.g. the availability of operating rooms and diagnostic devices)
• What has occurred during the time from injury to arrival in the trauma center and the status of the patient during field management and transport

A total of 76 uncertain items were identified in 40 cases by examining verbal communications and subject matter expert reviews. They were categorized as:

• Patient Related Uncertainty (26%): Mechanism and extent of injury (8/76), reports by pre-hospital care providers (1/76), patient’s prior medical history (5/76), effect of treatment in traumatized patient (6/76)
• Team/Organization Related Uncertainty (41%): resources and schedules (6/76), status/availability of team members (8/76), task distribution among team members (10/76), intention of other team member(7/76)

Interestingly, most of the uncertain items were related to team activities and the organization as whole as opposed to those related to patient conditions. This observation led to the possibility of improving performance by better intra-team communication (e.g. standardization, checklist) and by using data/video transmission of patient data.

Trauma patients, by definition, have traumatic changes in physiology and anatomy. These changes always cause uncertainty about the site and extent of patient injury. Part of the expertise of the people working in the domain is to deal with such uncertainty. However, when the activities during resuscitation were examined beyond the strict context of decision making, this study found that there were a number of sources of uncertainty that the clinicians had to deal with. In particular, uncertainties related to team activities and the organization as whole stood out.

In comparison with some of industrial settings where job specification and work process are well spelled out and team structure is clear to every team member, medical practices lack formal work structures. The interactions among the individuals caring for a patient are mostly informal in nature and are usually not codified. Within subgroups of professionals (such as surgical care providers or anesthesia care providers) there exist hierarchical structures, but the care providers of a patient as a whole are not subject to formal rules that govern the exchange of information and materials. The lack of extensive standard operating procedures only compounds the informal characteristics of medical practices. Observed procedures changed from one leading surgeon to the next. Duplicate execution of tasks, critical tasks unattended, lack of communication of key information, conflicting plans, frustration of having to second-guess others’ intention, etc, have been reported (e.g. Mackenzie et al, 1994; Donchin, et al, 1995, Xiao et al, 1995, Mackenzie, 1996).

It seems that lack of communication among the team members and among the personnel working in different departments contributes to much of the uncertainties identified. Such uncertainties could be reduced by, for example, explicitly informing other team members one’s intention and plans. Improved communications on schedules and availability of resources (e.g. CT scanners) should also reduce uncertainties. Another source of uncertainties identified here was from patient monitors. Patient monitors are subject to many interfering factors, which often make monitors render faulty readings.

Little past literature examines uncertainty with the consideration of team and organization factors. Hutchins (1995) made a noteworthy recent attempt along this direction. He postulated that, in order for a team to deal with varying task workload, there must be overlap among the team members’ capabilities. As a consequence, there are remaining degrees of freedom in terms of task distribution among team members, which means uncertainties about who should do what and when.

The issue of temporal scale Through our analysis of videotaped performance and our observation of collaboration in trauma patient care, it appears to be useful to view collaboration in three somewhat overlapping categories of effort in terms of temporal scope:

• seconds-minutes
• hours-shift, and
• day-weeks

At different temporal scopes, different issues are at hand. For example, activity sequencing may be an important factor for collaboration over the span of seconds-minutes whereas development of team performance norms would be a concern when the temporal scope is day-weeks. In the case of a trauma patient resuscitation team, members may be concerned with physical access to the patient and activities coordination over the span of seconds-minutes, but they may be concerned with assignment of roles and workload negotiation over the shift (hours-shift). Any concerns of procedures and staffing levels will be considered over the spans of day-weeks.

Although they are highly related, research emphasis about collaborative efforts over these three different temporal scopes can and should be different, as the fundamental issues of collaboration at these three temporal scopes are different in terms of training, design of information technology solutions, and research methodology. The studies reported above are mostly on the efforts in the temporal scope of seconds-minutes. Few would doubt that much can be learned by expanding the temporal scope to hours-shifts. Recent advances in information technology, for example, could be used to improve collaboration over longer time spans.

Human performance in regular context is almost always collaborative: individual performance is usually undefined when actual performance in any work settings is analyzed. In fact, performance should always be viewed as a result of collaboration among people in the context of tool using. In contrast, studies of human performance have not provided research methodologies, framework, and descriptive languages for performance observed in actual work settings. Such contrast can be seen readily in our studies. Systematic and methodical analysis of video-based performance data are needed. Data synthesizing and measuring instruments need to be established and tested.

The research community in the areas of human factors and cognitive engineering has started to face the challenge of studying performance in the actual work environment. In medical domains, the increasing use of telecommunication and computation tools brought the issue of collaboration to the forefront. Previously isolated components in medical systems are more and more interconnected. Video-based studies on performance in the real environment, as demonstrated above, can contribute much to the understanding of collaborative work.

The experiments reported here are examples of using cognitive engineering methodologies in the field of telemedicine. The research questions and methodologies are in contrast to those typically seen in telemedicine research where new technology and novel use of technology are frequently the focus.

Detailed understanding of the cognitive activities involved in the information process in medical domains is important to the use of telecommunication technology in healthcare. Particularly, from the point of view of patient safety, we need the ability to predict new modes of errors, new patterns of workloads, new working relationships among the care providers, and new skill requirements. Investing in cognitive engineering research in telemedicine will be fundamental to the success of telemedicine and to ensuring patient safety. With the increasing role of technology in healthcare, long-term investment in cognitive engineering research similar to that in the safety improvement in other industries (e.g. aviation and nuclear power generation) should be made.

When multiple, concurrent threads of activities are involved, a remote expert could be overloaded and miss important cues. When situations evolve rapidly and tasks are carried out by multiple people, one cannot assume that all of what is presented on video will be perceived by the remote decision-makers. Furthermore, the visual cues missed by the remote decision-makers will likely be biased by their experience background. Strategies countering such tendencies are therefore indicated including training of the remote decision-maker to appreciate tasks outside their specialty and the boundaries of traditional divisions of labor, cueing teleconsultants to systematically scan the video source at regular intervals.

Abel, MJ, (1990). Experiences in an exploratory distributed organization. In Galegher, J, Kraut, R E and Edigo, C (Eds.): Intellectual teamwork: Social and technological foundations of cooperative work. Hillsdale, NJ: LEA. 486-510

Allely, EB (1995). Synchronous and asynchronous telemedicine. Journal of Medical Systems, 19:207-212

Brown DL (1987) Trauma Management:The Anesthesiologist, Role International Anesthesiology Clinics 25:1-18.

Cannon-Bowers, JA, Tannenbaum, SI, Salas, E, and Converse, SA (1991). Toward an Integration of training theory and technique. Human Factors, 33:281-292.

Caplan, RA, Posner, KL, Ward, RJ, and Cheyney, FW (1990) Adverse Respiratory Events in Anesthesia. A Closed Claims Analyses Anesthesiology 72:828-833.

Cheyney, FW, Posner, KL, and Caplan, RA (1991) Adverse Respiratory Events Infrequently Leading to Malpractice Suits. Anesthesiology 75:932, 939.

Donchin, Y, Gopher, D, Olin, M, Badihi, Y, Biesky, M, Sprung, CL, Pizov, R, and Cotev, S. (1995). A look into the nature and causes of human errors in the intensive care unit. Critical Care Medicine, 23, 294-300

Ellis, CA, Gibbs, S and Rein, G (1991). Groupware: The research and development issues. Communications of the ACM. 34(1):38-58

Endsley, MR, and Kiris, EO (1995). The out-of-the-loop performance problem and level of control. Human Factors, 37:381-394.

Finholt, T, Sproull, L and Kiesler, S (1990). Communication and performance in ad hoc task groups. In Galegher, J., Kraut, RE and Edigo, C (Eds.): Intellectual teamwork: Social and technological foundations of cooperative work. Hillsdale, NJ: LEA. 291-325

Harris, RL, and Christhilf, D M (1980). What do pilots see in displays? In: Proceedings of Human Factors Society 24th Annual Meeting.

Hutchins, E (1989). The technology of team navigation. In: Galeglier, J, Kraut, RE, and Egido, (Eds.), Intellectual Teamwork: Social and Technical Bases of Cooperative Work. Hillsdale, NJ: Lawrence Erlbaum Associates.

Hutchins, E (1995). Cognition in the wild. Cambridge, MA: MIT Press.

Kanki BG, Folk VG and Irwin CM (1991) Communication Variations and Aircrew Performance International Journal of Anation Psychology 1:149-162

Kanki, BG, Lozito, S, and Foushee, HC (1989). Communication indices of crew coordination. Aviation, Space, and Environmental Medicine, 60:56-60. Mackenzie, CF, Jefferies, MJ, Hunter, WA, Bernhard, WN, Xiao Y and The LOTAS Group Comparing of Self-Reporting of Deficiencies in Airway Management with Video Analysis of Actual Performance. Human Factors 38:623-635.(1996)

Kiesler, S, Siegel, J, and McGuire, T (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39:1123-1134.

Krauss, RM, Garlock, CM, Bricker, PD and McMahon, LE (1977). The role of audible and visible back-channel responses in interpersonal communication. Journal of Personality and Social Psychology. 35:523-529

Mackenzie, CF, Craig, GR, Parr, MJ, Horst, R, and the LOTAS Group. (1994). Video analysis of two emergency tracheal intubations identifies flawed decision-making. Anesthesiology, 81:911-919.

Mackenzie, CF, Hu, PF, Horst, RL, and the LOTAS Group. (1995). An audio-video acquisition system for automated remote monitoring in the clinical environment. Journal of Clinical Monitoring, 11:335-341.

Mackenzie, CF, Martin, Xiao, Y, and the LOTAS Group. (1996). Video analysis of prolonged uncorrected esophageal intubation. Anesthesiology, 84:1394-1503.

McGrath, JE.(1990). Time matters in groups. In Galegher, J, Kraut, RE and Edigo, C.(Eds.): Intellectual teamwork: Social and technological foundations of cooperative work. Hillsdale, NJ: LEA . 23-61

McGrath, JE.(1991). Time, interaction, and performance (TIP): A theory of groups. Small Group Research. 22(2):147-174

McGrath, JE.and Hollingshead, AB (1994). Groups Interacting with Technology: Ideas, Evidence, Issues and an Agenda. Sage: Thousand Oaks, CA

McGrath, JE, Arrow, H, Gruenfeld, DH, Hollingshead, AB and O'Connor, KM (1993). Groups, tasks, and technology: The effects of experience and change. Small Group Research. 24(3):406-420

McLeod, PL (1992). An assessment of the experimental literature on electronic support of group work: Results of a meta-analysis. Human Computer Interaction. 7(3):257-280

Michailidis, A and Rada, R (1997). Activities and communication modes. International Journal of Human-Computer Studies. 46(4):469-483

Morrissette, JO, Hornseth, J P, and Shellar, K (1975). Team organization and monitoring performance. Human Factors, 17:296-300.

National Research Council. (1990). Distributed Decision-Making. Washington, DC: National Academy Press.

Orasanu, J (1990). Shared mental models and crew decision making. Tech. Rept. 46. Cognitive Science Laboratory, Princeton University, Princeton, NJ.

Orasanu, J, and Salas, E (1993). Team decision making in complex environments. In: Klein, G.A, Orasanu, J, Calderwood, R, and Zsambok, CE (Eds.), Decision-Making in Action: Models and Methods. Norwood, NJ: Ablex.

Rasmussen, J, Brehmer, B, and Leplat, J (Eds.). (1991). Distributed Decision Making: Cognitive Models for Cooperative Work. Chichester: Wiley.

Ripoll, H (1988). Analysis of visual scanning patterns of volley ball players in a problem-solving task. International Journal of Sport Psychology, 19:9-25.

Ripoll, H, Kerlirzin, Y, Stein, JF and Reine, B (1995). Analysis of information processing, decision making, and visual strategies in complex problem solving sport situations. Human Movement Science, 14:325-349.Rogalski, J., and Samurcay, R (1993). Analyzing communication in complex distributed decision-making. Ergonomics, 36:1329-1343.

Segal, LD (1994). Actions speak louder than words: How pilots use nonverbal information for crew communications. In: Proceedings of the Human Factors and Ergonomics Society 38^th Annual Meeting, 21-25.

Senders, JW (1980). Visual Scanning Processes. University of Tilburg, the Netherlands: Published PhD Thesis.

Serfaty, D, P7. 1989, Entin, E. E., and Volpe, C (1993). Adaptation to stress in team decision-making and coordination In: Proceedings of the Human Factors and Ergnomics Society 37^th Annual meeting, 1228-1232.

Serfaty, D, Entin, EE, and Volpe, C (1993). Adaptation to stress in team decision making and coordination. In: Proc Human Factors and Ergnomics Society 37^th Annual meeting. 1228-232

Sirevaag, E, Kramer, A, Wickens, C, Reisweber, M, Strayger, D, and Grenell, J. (1993). Assessment of pilot performance and workload in rotary win helicopters. Ergonomics, 9:1121-1140.

Sperandio, J C.(1971). Variation of operator’s strategies and regulating effects on workload. Ergonomics, 14:571-577.

Straus, SG and McGrath, J.E. (1994). Does the medium matter? The interaction of task type and technology on group performance and member reactions. Journalof Applied Psychology. 79(1):87-97

U.S. Congress, Office of Technology Assessment. (1995). Bringing Health Care Online: The Role of Information Technologies (OTA-ITC-624). Washington, DC: U.S. Government Printing Office.

Valacich, JS. George, JE, Nunamaker, JF, Jr., and Douglas, VR (1994). Physical proximity effects on computer-mediated group idea generation. Small Group Research. 25(1):83-104

Volpe, CE, Cannon-Bowers, JA, Salas, E and Spector, PE (1996). The impact of cross-training on team functioning: an empricial investigation. Human Factors. 38:87-100.

Xiao, Y, and Mackenzie, C F (1996). Remote diagnosis in dynamic task environments. In: Proceedings of Human Factors and Ergonomics Society 40th Annual Meeting, 218-222. Santa Monica, CA: Human Factors and Ergonomics Society.

Xiao, Y, Hunter, ., Mackenzie, C, Jefferies, N, Horst R, and the LOTAS Group. (1996). Task complexity in emergency medical care and its implications or team coordination. Human Factors, 38:636-645.

Xiao, Y, Mackenzie, CF, and Spencer, R (1997). The effect of experience background on remote diagnosis. In: Proceedings of Human Factors and Ergonomics Society 41st Annual Meeting, 254-258. Santa Monica, CA: Human Factors and Ergonomics Society.

Xiao, Y, Mackenzie, CF, and the LOTAS Group. (1995). Decision making in dynamic environments: Fixation errors and their causes. In: Proceedings of Human Factors and Ergonomics Society 39^th Annual Meeting, 469-473.

Xiao, Y, Mackenzie, C F, NATO

Zsambok, CE, and Klein, GA, (Eds.). (1997). Naturalistic Decision-Making. Hillsdale, NJ: Lawrence Erlbaum Associates.

Airway Management -- includes maintaining a patent airway in conscious and unconscious patients by means of head tilt, jaw thrust, and tracheal intubation. In unconscious patients, the airway is protected against aspiration of regurgitated stomach contents (see below) by use of an inflatable cuff on the tracheal tube.

Anesthesia Bag -- A 3L-rubber bag included in the patient ventilating circuit that can be manually compressed to provide patient ventilation.

Anesthesia Record -- A written document that records patient vital signs, every 5 min, and documents the timing of anesthetic and surgical interventions and any drugs and fluids given. It describes any unusual patient responses or events.

Anesthesia Care Providers -- physicians (Attending or faculty, Fellow or Resident-in-Training) Or nurses (Certified Registered Nurse Anesthetists).

Anesthesia Quality Assurance (AQA) -- A process of peer review designed to capture all occurrences, some of which result in harm or a poor outcome.

Aspiration -- Passage of regurgitated stomach contents into the trachea because laryngeal reflexes are obtunded due to anesthesia, muscle paralysis or unconsciousness.

Auscultate -- To listen to, using a stethoscope applied over the heart, lungs or abdomen.

Cricoid Pressure -- Digital pressure applied with thumb and index finger over the cricoid cartilage situated just below the larynx in the neck. This pressure prevents regurgitation of stomach contents when anesthesia is induced in emergency circumstances in patients with full stomachs.

Disconnect Alarm -- An auditory alarm that sounds when the ventilating circuit between the anesthesia machine or mechanical ventilator becomes disconnected from the patient's airway.

Elective Intubation -- The airway is electively intubated, usually with the patient adequately investigated and comprehensively monitored.

Emergency intubation -- In these studies, defined as an intubation required within 10 min of the patient's arrival into the trauma center. Emergency intubations are considered more risky than elective intubation because less is known about the patient's status.

End-Tidal CO₂ (ETCO₂) -- The value of CO₂ concentration at the end of exhalation. ETCO₂ approximates the arterial CO₂. A value of ETCO₂ greater than 30 mmHg for 5 consecutive breaths confirms that the tracheal tube is in the trachea not the esophagus.

Esophageal Intubation -- The accidental passage of a tracheal tube into the esophagus. If undetected this will cause lack of oxygen in the circulation. Detection of esophageal intubation is made by auscultation of the chest and abdomen and a failure to detect ETCO₂ for 5 breaths.Esophagus – The passage connecting the mouth to the stomach. The esophagus leaves the mouth just below the entrance to the windpipe (larynx).

Extubation -- The removal of a tube from the trachea. It can be intentional, as at the end of anesthesia, or accidental in a semi-conscious patient.

In-line Neck Stabilization -- In trauma patients with suspected neck injuries, the neck is stabilized by an assistant during intubation to minimize potentiation of a neck injury.

Intravenous (IV) Access -- Cannulae placed into the veins that allows administration of fluids and blood.

Intubation -- Passage of a tracheal tube through the upper airway (nose or mouth) and larynx into the trachea.

Level One Trauma Anesthesia Simulation (LOTAS) Group -- A group of anesthesia care providers who have met regularly for eight years to answer research questions from videotaped trauma anesthesia care

Manual Ventilation -- Ventilation achieved by compression of an anesthesia bag or self-inflating resuscitator bag

Mechanical Ventilator -- A device that is interfaced to the patient by means of a cuffed tracheal tube and allows positive pressure mechanical ventilation.

Muscle Paralysis -- Paralysis of neurotransmission by pharmacological use of drugs such as curare.

Oxygen Saturation (SpO₂) -- The percent O₂ saturation of hemoglobin in peripheral arterial blood. This value is monitored on a beat-by-beat basis by use of a pulse oximeter (see below).

Post-Trauma Questionnaire (PTQ) -- A research questionnaire completed by the anesthesia care provider(s) after each videotaped case

Pre-Oxygenation -- a technique to increase the O₂ reserves of a patient before induction of anesthesia. Breathing 100% O₂ through a close fitting facemask provides a reservoir of O₂ in the lungs to minimize the likelihood of the patient developing hypoxemia.

Pulse Oximeter -- Estimates the beat-to-beat O₂ saturation of peripheral arterial blood by a probe attached to the patient.

Resuscitator Bag -- A self-inflating bag that is attached to a face mask or tracheal tube. Compression of the bag is used to provide oxygenation and ventilation.

Tracheal Intubation -- Passage of a cuffed tracheal tube through the upper airway (nose or mouth) between the vocal cords and into the trachea.

Tracheal Tube Cuff -- An inflatable cuff that seals the airway to prevent aspiration and allow positive-pressure mechanical ventilation.

Ventilator Tubing -- Is the long flexible corrugated tubing used to make the connection between a patient's airway and a mechanical ventilator or anesthesia machine.

The following publications and presentations were generated by the activities either funded solely by or in part by this contract and the support from the contract was acknowledged.

1. Y Xiao, CF Mackenzie, J Orasanu, R Spencer, A Rahman, V Gunawardane and the LOTAS Group (1999). Information acquisition from audio-video-data sources: An experimental study on remote diagnosis. Telemedicine Journal, 5(2) (in press)

Full conference papers

2. Y. Xiao and C.F. Mackenzie (1998). Collaboration in complex medical systems. In "Collaborative crew performance in complex operational systems - NATO HFM Symposium", pp. 4-1--4-10, 20-22 April, 1998, Edinburgh, UK.
3. Y Xiao and CF Mackenzie (1998). Micro-theory methodology in critical incident analysis.  1998 Proc. IEEE Int. Conf. on Systems, Man, and Cybernetics, 2545-2550
4. Y Xiao, CF Mackenzie, R Patey, and LOTAS Group (1998). Team coordination and breakdowns in a real-life stressful environment. Proc. Human Factors and Ergonomics 42nd Annual Meeting, 186-190 
5. Y Xiao, CF Mackenzie,  J Orasanu, R Spencer, V Gunawardane,  and LOTAS Group (1998). Visual scanning patterns during remote diagnosis. Proc. Human Factors and Ergonomics 42nd Annual Meeting, 272-276
6. Y Xiao, CF Mackenzie, and R Spencer (1997). Nurses, surgeons, and anesthesiologists as teleconsultants: The effect of experience background on remote diagnosis. Proc. Human Factors and Ergonomics 41th Annual Meeting, 254-258 
7. Y Xiao (1997). Uncertainty in trauma patient resuscitation. Proc. Human Factors and Ergonomics 41th Annual Meeting, 168-171 
8. Y Xiao and CF Mackenzie (1996). Remote Diagnosis in Dynamic Task Environments. Proc. Human Factors and Ergonomics 40th Annual Meeting, 218-222
9. CF Mackenzie, B Harper, and Y Xiao (1996). Simulator Limitations and Their Effects on Decision Making. Proc. Human Factors and Ergonomics 40th Annual Meeting, 747-751 

1. Y Xiao, CF Mackenzie, R Spencer, M Jaberi, M Koch and The LOTAS Group (1998). Intrusiveness of auditory alarms during airway management. Anesthesiology, 89(3A):A1224.
2. Y Xiao, CF Mackenzie, D Ovelgone and the LOTAS Group (1997). Simulator realism impacts diagnostic abilities on auscultation. Anesthesiology, 87(3A):A939
3. CF Mackenzie, Y Xiao, M Jaberi, R Spencer, D Ovelgone, H Cline, J Blenko, P Delaney and the LOTAS Group (1997). Can we quantitate expertise in tracheal intubation? Anesthesiology, 87(3A):A974
4. RP Dutton, Y Xiao, W Bernhard, C Boehm, D Gabbot, A Hunter, M Jaberi, J Wesolowski, CF Mackenzie and the LOTAS Group (1997). Measured versus predicted stress during elective and emergency airway management. Anesthesiology, 87(3A):A444
5. Y Xiao, CF Mackenzie, and the LOTAS Group (1997). Do surgeons, nurses and anesthesiologists perceive the same medical care differently? Journal of Clinical Monitoring.
6. Y Xiao, CF Mackenzie, D Ovelgone, and the LOTAS Group (1997). Simulator realism impacts diagnostic abilities on auscultation. Journal of Clinical Monitoring.
7. CF Mackenzie, R Dutton, M Jaberi, B Harper, Y Xiao, and the LOTAS Group (1996). Simulator limitations affect evaluation of performance. Anesthesiology, 85(3A):A268
8. M Jaberi, Y Xiao, CF Mackenzie, and the LOTAS Group (1996). Incident monitoring by videotaping of acute trauma patient management. Anesthesiology, 85(3A):A1036
9. Y Xiao, CF Mackenzie, M Jaberi, B Harper, and the LOTAS Group (1996). Alarms: Silenced, ignored, missed. Anesthesiology, 85(3A):A422 
10. Y Xiao, CF Mackenzie, W Bernhard, JR Hebel, and the LOTAS Group (1996). Dynamics of stress during elective and emergency airway management. Anesthesiology, 85(3A):A957
11. Y Xiao, CF Mackenzie, W Bernhard, and the LOTAS Group (1995). Identifying stressors during airway management through regressional analysis. Anesthesiology, 83(3A):A1118
12. Y Xiao, CF Mackenzie, and the LOTAS Group (1995). Does stress induce more errors in airway management? Anesthesiology, 83(3A):A1119
13. M Jaberi, W Bernhard, Y Xiao, CF Mackenzie, and the LOTAS Group (1995). Can we use videotaping to analyze expertise in tracheal intubation? Anesthesiology, 83(3A):A1013
14. CF Mackenzie, A Hunter, Y Xiao, W Bernhard, and the LOTAS Group (1995). Can we use videotaping to analyze expertise in tracheal intubation? Anesthesiology, 83(3A):A216
15. CF Mackenzie, A Hunter, Y Xiao, W Bernhard and the LOTAS Group (1995). Task Priorities and Their Omission before Emergency and Elective Intubation. Anesthesiology 83:A216
16. Y Xiao, CF Mackenzie, W Bernhard and the LOTAS Group (1995). Identifying stressors during airway management through regression analysis. Anesthesiology 83:A1119

1. Coordination in Complex Medical Systems. NATO HFM Symposium on "Collaborative Crew Performance in Complex Operational Systems", 20-22 April, 1998, Edinburgh, UK
2. Defining Cognition in Working Settings, Krasnow Institute, George Mason University, April 14, 1997
3. Medical Teams. Third Annual Sumposium on Human Interaction with Complex Systems, Sponsored by IEEE Computer Society, August 25-28, 1996, Dayton, OH 
4. Remote Diagnosis. 40th Annual Meeting of Human Factors and Ergonomics Society, September 2-6, 1996, Philadelphia, PA 
5. Analyzing Video Data: Challenges and Solutions, Second Symposium on Simulators in Anesthesiology Education, Rochester, New York, June 2, 1996 
6. Observe with Tools, 11th World Congress of Anaesthesiologists, Sydney, Australia, April 13-18, 1996 
7. Remote Diagnosis for Trauma Patient Resuscitation, NASA Ames Research Center, Moffett Field, CA, January 22, 1996 
8. Human Factors Research in Emergency Medicine, Department of Psychology, George Mason University, Fairfax, VA., September 4, 1995 
9. Decision Making Models, 8th Annual Trauma Anesthesia and Critical Care Symposium, Balimore, Maryland, May 12, 1995