The Cockpit Task Management System
|
The Cockpit Task Management System |
|
Synopsis: This page describes the Cockpit Task Management System (CTMS), an aid developed to facilitate Cockpit Task Management, a precursor to Agenda Management.
Keywords: Cockpit Task Management, distributed artificial intelligence.
Last update: 24 Jun 97
Based on our findings from studies of Cockpit Task Management (CTM), we developed an aid to facilitate CTM called the Cockpit Task Management System (CTMS). This page is based on Funk and Kim (1995).
The objectives of this part of our study were to determine the feasibility of CTMS implementation through the development of a prototype CTMS and to evaluate CTMS effectiveness in the improvement of CTM performance.
The flight simulator used for this research was a simple, part-task simulator for a single pilot. We developed it by modifying the existing flight simulator used for a previous CTM study (Chou, 1991). The simulator consisted of three personal computers, each with its own monitor, a computer keyboard, two trackballs, and a sidestick controller. All of the simulator computers were linked via Ethernet, using the TCP/IP communication protocol.
The top monitor was a simulated head up display (HUD) showing aircraft heading, airspeed, and altitude; a pitch ladder; aircraft horizontal location; and autopilot status (i.e. engaged or disengaged). The bottom left monitor, called the navigation display (ND), showed aircraft horizontal position on a moving map display and provided a simple datalink system for simulated air traffic control (ATC) communication. The subsystem display (SD) on the right monitor showed synoptic displays of several simulated aircraft subsystems such as engines, a hydraulic system, and an electrical system. It also provided an interface for a simple flight path management system and warning and alerting displays.
Our goals for the CTMS were that it should help the pilot initiate, monitor, prioritize, and terminate tasks. To achieve these goals, we determined that the CTMS should provide information about task state (upcoming, active, terminated), status (satisfactory or unsatisfactory performance), and priority.
We implemented the CTMS using Smalltalk, an object-oriented computer programming language. As for TSS development, we used concepts of object-oriented design and distributed artificial intelligence in the CTMS implementation, where aircraft subsystems and flight tasks were represented by conceptual software units referred to as agents. In the CTMS, aircraft subsystems and pilot tasks were represented by system agents (SAs) and task agents (TAs), respectively. The CTMS was an object-oriented, agent-based system in which problem solving knowledge was distributed among SAs and TAs.
An SA was a representative of an aircraft subsystem. A subsystem SA received state information about its corresponding aircraft subsystem from the flight simulator, releasing this information when requested. For the CTMS, an SA was implemented as an instance of a class, and the specific behaviors or knowledge of the SA were implemented in the methods of the class.
Task agents were responsible for helping the pilot perform corresponding flight tasks. Like SAs, TAs were implemented as instances of Smalltalk classes, and the specific behavior or knowledge necessary for each TA was implemented in the methods of the class.
Following is the CTMS class hierarchy, listing all classes implemented in the system and showing class-subclass relationships.
Each TA used information from SAs and its own procedural knowledge to determine the state of its task: latent (not imminent), upcoming (imminent), in progress, suggested (requiring immediate attention), or finished. Task status (satisfactory or unsatisfactory) was determined in a similar way.
The CTMS display provided information about all tasks with respect to the following four characteristics:
The display consisted of three sections:
Task information was provided on the corresponding display sections. That is, task state information was presented using location coding.
Task status was either satisfactory or unsatisfactory and indicated by the use of color coding. That is, if a task was being performed satisfactorily, a green color was used; if its performance was unsatisfactory, a yellow or red color was used, depending upon severity.
In addition to task state and status, task priority information was presented on the STD. That is, names of the suggested tasks were displayed according to the priority of the tasks, with higher priority tasks being placed higher in the list. Using the UTD or ITD, the upcoming and in progress tasks, respectively, were displayed in hierarchical structure.
After the CTMS was implemented and interfaced to the flight simulator, an experiment was performed to evaluate its effectiveness in improving CTM performance. Twelve volunteer subjects were used for the experiment. The first four subjects were used for a pilot study to check the readiness of the experiment, and the remaining eight subjects were used for the experimental data collection test runs.
We developed a balanced experimental design for the data collection flights. To compare subject performances between flying with and without the CTMS, each subject flew two data collection scenarios - one with the CTMS and the other without it. We developed two different flight scenarios, A and B, to avoid the learning effect that would have resulted from the use of an identical scenario in the two data collection flights. We designed each to present the same complexity to minimize the effect by the differences in scenario complexity, which could have biased the results of the experiment.
We administered the experimental procedure in two lengthy sessions: a training session and a data collection session. After a four-hour first day training session followed by a two-hour second day training session, each subject flew two 50 minute data collection flight scenarios with a 5 to 10 minute break between flights.
We used four measurements for the evaluation of subject performance in the flight simulator:
Three of the four measurements, task prioritization, pilot response time, and task completion, reflected the three elements in our CTM error taxonomy: task prioritization, task initiation, and task termination, respectively. Subject performance in aircraft control (heading, altitude, and airspeed) was assessed as a comprehensive measurement of overall pilot performance. Pilot performance data from the 16 scenario flights flown by eight subjects were collected for these four performance measures. Simulator log files (containing recorded pilot actions and performance) and videotapes were used to collect performance data.
These files and tapes allowed us to compute four performance measures:
When subjects flew with the assistance of the CTMS, the mean task misprioritization rate was reduced by 41 per cent, the mean subject response time was reduced by 18 per cent, the exercise of mean unsatisfactory aircraft controls was reduced by 24 per cent, and the average number of incomplete tasks during simulator flights was reduced by 82 per cent (see figure).
In addition to comparing the subject performance averages, we performed a statistical analysis of the collected data using an analysis of variance (ANOVA). Since the hypothesis test using the ANOVA was based upon the expectation that performance with the CTMS would be better than performance without the CTMS, we employed a one-tailed test. We considered the probability of a type I error, denoted by alpha, of both 0.1 and 0.05, insofar as this form has gained acceptance for use in typical statistical analyses. In such analyses, the results of a hypothesis test are reported as a number called the p-value - a measurement of the credibility of the hypothesis test. A type I error probability, alpha, and a p-value are used to determine whether the null hypothesis, denoted by Ho, can be rejected. Since the principal concern of this experiment was CTMS effectiveness, as indicated by the p-values for the treatment effect, we present only these values in the ANOVA table.
From the results of the hypothesis test, the p-value for incomplete tasks indicated that there was significant improvement for task completion performance when subjects flew with the assistance of the CTMS, whereas the p-values for the remaining three measurements for task prioritization, task initiation, and aircraft controls were suggestive with respect to evidence of performance improvement.
These results indicate that the CTMS was effective in improving CTM performance under the experimental conditions. In other words, they show that if an aid can accurately determine what tasks the pilot is attempting to complete and how well the tasks are being performed, CTM performance can be facilitated by displaying relevant task management information, in particular, calling the pilot's attention to tasks which are not being performed in a satisfactory or timely manner. That the CTMS was successful in this is due in large part to the simplicity of the simulated aircraft, environment, and tasks.
Development of the Cockpit Task Management System was sponsored in part by the Oregon State University Research Council and the OSU Department of Industrial and Manufacturing Engineering.
Chou, C.D., Cockpit Task Management Errors: A Design Issue for Intelligent Pilot-Vehicle Interfaces. Unpublished doctoral dissertation, Oregon State University, 1991.
Chou, C.D., Madhavan, D. and Funk, K., "Studies of Cockpit Task Management Errors," International Journal of Aviation Psychology, in press.
Funk, K., "Cockpit Task Management: Preliminary Definitions, Normative Theory, Error Taxonomy, and Design Recommendations, The International Journal of Aviation Psychology, Vol. 1, No. 4, 1991, pp. 271-285.
Funk, K.H. and Lind, J.H., "Agent-Based Pilot-Vehicle Interfaces," IEEE Transactions. on Systems, Man, and Cybernetics, Vol. 22, No. 6, Nov/Dec 1992, pp. 1309-1322.
Funk, K. and Kim, J.N., "Agent-Based Aids to Facilitate Cockpit Task Management," Proceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics, Piscataway, NJ: Institute of Electrical and Electronics Engineers, 1995, pp. 1521-1526.
Kim, J.N., An Agent-Based Cockpit Task Management System: a Task-Oriented Pilot-Vehicle Interface. Unpublished doctoral dissertation, Oregon State University, 1994.