Training Management

A Preliminary Investigation of Training Order for

Introducing NextGen Tools

R. Conrad Rorie1, Ariana Kiken1, Corey Morgan1, Sabrina Billinghurst1, Gregory

Morales1, Kevin Monk1, Kim-Phuong L. Vu1, Thomas Strybel1, and Vernol Battiste2

1

California State University Long Beach

Center for Human Factors in Advanced Aeronautics Technologies

1250 N Bellflower Blvd. Long Beach, CA 90840, USA

2

San Jose State University Foundation and NASA Ames Research Center

Moffett Field, CA 94035, USA

************@*****.***, {aegkiken, coreyandrewmorgan, sabrinabillinghurst,

gregory.morales}@gmail.com, *******@*****.***,

{kvu8, tstrybel}@csulb.edu, ****.********-*@****.***

Abstract. Eleven students enrolled in a 16-week radar simulation course were

trained on current-day and NextGen tools. The order of the training was

manipulated so that half of the students received current-day training first,

followed by the training on NextGen tools, while the remaining students

received training on the NextGen tools first, followed by current-day training.

This paper reports data from the debriefing sessions following the conclusion of

the course, with the intent of determining students reaction to the training order

and their comments and suggestions for future training schedules. Results

indicated that future training should start with current-day procedures and delay

the introduction of NextGen tools until trainees have established fundamental

air traffic management skills.

Keywords: ATC, training, part-task, NextGen

1 Introduction

The Next Generation Airspace Transportation System (NextGen) will replace the

current air traffic management (ATM) system in the U.S. NextGen is being

developed to allow the national air space to handle 2-3X current-day traffic levels in

response to estimated increases in air travel in the US and world-wide by 2025 [1].

One limitation of the current ATM system is the amount of aircraft that can be safely

handled by air traffic controllers (ATC) with the existing tools. NextGen intends to

automate specific air traffic roles and responsibilities in order to reduce controller

workload and increase their capacity to handle more aircraft. Tasks that are under

consideration for automation in NextGen include conflict detection and conflict

resolution [2]. In addition, tools and technologies are being considered to provide

controllers with the capability of making route modifications that can be uplinked

directly to the flight deck [3].

The introduction of NextGen concepts of operations, tools, and technologies need

to be evaluated in terms of their effectiveness. As such, many studies and simulations

have been conducted to test potential concepts and technologies for NextGen [4, 5, 6].

One area that has not received much attention, though, is that of training NextGen

tools and procedures. Currently, ATC training is typically based on Air Traffic Basics

courses provided by Air Traffic Collegiate Training Initiative programs followed by

intensive training at the FAA Academy in Oklahoma City [7, 8]. After completing

the basic training, controllers are then sent to a location (e.g., Tower, TRACON,

Center) to receive on-site training through an apprenticeship model [7]. The

apprenticeship can last from months to years. There has been some evidence that

simulation training can reduce on the job training time [9]. In the present study, we

examine whether the order in which NextGen tools are introduced into training can

affect the learning of students who are pursuing careers in air traffic control.

Although the effectiveness of NextGen technologies needs to be tested with

experienced controllers for operational validity, students are a necessary source of

research participants because they will eventually become NextGen operators. Vu et

al. [10] showed that highly-trained students in a specific sector can show analogous

performance to retired controllers who were only given a 1-day training session to

familiarize themselves with a new sector. Moreover, students showed more

willingness to try new technologies than experienced controllers. One reason for

higher acceptance of new technologies among students is that the students have

grown up in an era where technology is dominant in everyday activities. Future ATC

training will need to incorporate the training of NextGen concepts and technologies to

fully prepare the workforce for the upcoming transformation. This study is an initial

step toward that goal by examining how the order of introduction to NextGen tools

affects student ATM learning. Results from this preliminary study can inform future

research on how to best introduce NextGen technologies into the current-day training

paradigm.

Schneider [11] described air traffic controlling as a high performance skill,

defined as a task requiring more than 100 hours of training, producing high failure

rates, and exhibiting qualitative differences between the performances of novices and

experts. As such, training programs need to be designed to be as effective as possible

in order to promote successful completion of their requirements. Sohn et al. [12]

noted that, when learning complex skills, it can be helpful to break down high-level

tasks into their component parts, otherwise known as part-task training. They

reasoned that becoming fluent on the component tasks increases the chances of

becoming fluent on the overall task. There have been many demonstrations of part-

task training compared to whole-task training in different domains [13, 14, 15].

Young et al. [16] showed that the difficulty participants experienced in training can

also influence their performance later. In their study, an unrelated secondary task was

added to increase the task s difficulty during training. Increased difficulty during

training was found to lead to better retention of skills over time. With regard to

NextGen training, these results suggest that more difficult skills should be learned

first, followed by easier skills. To test this hypothesis, we vary the order of two air

traffic management components, one involving the learning of current ATM skills

and the other involving the learning with NextGen tools that automate the task of

conflict detection and provide tools in support of conflict resolution.

Current-day air traffic management skills include detecting aircraft in conflict by

projecting aircraft trajectory and speed, using strategies for the separation of aircraft,

techniques for ensuring safe merging and spacing of aircraft, and using strategies for

overall sector management through the exclusive use of voice communications.

Introduction of NextGen technologies into ATC training will necessitate instruction

on several new, transformative tools such as Data Comm, conflict detection, and

conflict probes. Data Comm, a text-based communication system between controllers

and pilots, will significantly reduce the amount of voice communications made

between air traffic control and pilots. Although the reduction of voice communication

should reduce operator workload, the use of Data Comm will require the training of

controllers on Data Comm commands. Conflict detection tools will automatically

alert controllers of potential conflicts for pairs of aircraft that are equipped with

NextGen technologies. Enabling conflict detection will substantially reduce

controller workload since they will not need to scan for conflicts between the

equipped aircraft. Conflict probes will assist controllers in the task of conflict

resolution by providing them cues regarding whether a flight plan change is conflict-

free or not. Again, conflict probes should reduce controller workload and cognitive

demands. However, studies have shown that tools that decrease operator workload

may take operators out of the loop and reduce their situation awareness [17]. Low

levels of situation awareness can then lead to errors, especially when automation fails

and the operator must perform all tasks manually [18].

The present paper is based on a larger study examining whether the order in

which student controllers are trained with current day and NextGen procedures affects

their performance, situation awareness and workload. This paper reports data from

debriefing sessions of the larger study, with the intent of determining students

reaction to the training order, their assessment about how the training order affected

their overall learning, and their comments and suggestions for future training

schedules. A content analysis was performed on transcripts of the debriefing sessions

and written notes by an experimenter. Due to the small sample size and preliminary

nature of this training study, no formal analyses were run. Instead, we provide a

qualitative summary of the participants responses.

2 Method

2.1 Participants

Eleven students enrolled in a 16-week, radar simulation course through California

State University Long Beach s Center for Human Factors in Advanced Aeronautics

Technologies (CHAAT) participated. All students were enrolled in an aviation

program at Mount San Antonio College (Mt SAC), and are pursuing careers in air

traffic control. Students had an average of 1.5 years of study in their program and

have all completed a course in the Air Traffic Control Environment at Mt SAC, which

includes topics of aircraft characteristics, air traffic procedures, and phraseology.

2.2 Training

All students received a minimum of 6 hours per week of training on managing traffic

of enroute sectors ZID 91 and 81. Traffic in the sector consisted of arrivals and

departures to/from Louisville Standiford International Airport, as well as overflights.

Two of the 6 training hours were dedicated to general air traffic management skills

and airspace (en route) operations in a classroom setting taught by a retired, radar-

certified controller. The other 4 hours were dedicated to hands-on radar simulation

training in the CHAAT simulation lab using the MultiAircraft Control System

(MACS) software developed by the Airspace Operations Laboratory at NASA Ames

Research Center [19].

All students received instruction on the MACS software and on procedures and

strategies for managing traffic in the sector. The students were divided into two

separate training groups, differing on the timing of exposure to advanced ATC tools.

For the current-day procedures, students were taught traffic management techniques

for conflict detection and strategies for judging the threat level of potential conflicts.

In addition, they were instructed on point-to-point vectoring techniques for safety and

efficiency, and the proper phraseology for communicating with aircraft in their sector.

For the NextGen tools procedures, students were trained on how to use Data Comm

commands for issuing clearances as well as how to use the conflict alerting and

conflict probes for Data Comm-equipped aircraft to detect and resolve conflicts. All

students were also taught the procedure for taking handoffs and for handing off

aircraft using voice or Data Comm, depending on the training condition.

The Current Day-First training group received training on air traffic management

using only voice communications during Weeks 2-7 of the laboratory component of

the course and received training with NextGen tools (Data Comm and the advanced

conflict detection and resolution tools) during Weeks 9-15. Students in the NextGen-

First training group received hands-on training with Data Comm and the advanced

ATC tools during Weeks 2-7 and received current-day instruction without the

advanced tools during Weeks 9-15. Both groups received mixed training the week

before two sets of experimental trials, which occurred at the end of Weeks 8 and 16.

The experimental trials utilized a mixed equipage environment with Data Comm-

equipped and voice-only aircraft. At the end of the 16-week second test session,

participants were debriefed on the purpose of the simulation and were asked to

provide feedback regarding the training design. The conversations were recorded,

with permission of the participants, and then were subsequently transcribed for the

present analysis.

2.3 Apparatus

Participants performed their experimental trials within the same simulation

environment taught during the hands-on portion of their training. MACS is a medium

fidelity computer application simulating a radar screen of ZID Sector 91. It

accommodates both air traffic control and pilot operations. All aircraft were piloted

by pseudopilots trained specifically on MACS to provide a realistic traffic

environment for controllers. Communication between controllers and pilots was

provided by a voice server station, allowing communication via push-to-talk headsets

[20].

3 Results and Discussion

Each of the eleven subjects was asked at the end of the 16-week simulation course to

comment on the effectiveness of the training order in developing their ATM skills.

There were six subjects in the Current Day-First condition (i.e., voice first) and five

subjects in the NextGen-First condition. All six subjects in the Current Day-First

condition claimed that receiving verbal training before Data Comm training was

highly beneficial in developing their ATM skills and controller-pilot communication

skills. Perhaps more telling was that all five participants in the NextGen-First

condition reported that their order of training was ineffective, stating that it would

have been more beneficial to receive the training on manual conflict detection and

voice communication first. The particular themes that developed during the debriefing

sessions are discussed in more detail below.

3.1 Perceived Effectiveness of Training Order

All students indicated that the most important skills they needed for ATM, including

how to scan the scope for conflicts, predict flight trajectories, make speed projections,

and make effective use of strategies taught in the classroom for modifying routes,

were also the most difficult to learn. Therefore, they all reported that current day

procedures should be taught first and be given the most emphasis (about three-

quarters instead of half of the class) during training. This finding is consistent with

the difficulty of training hypothesis Young et al. [16] in that the more difficult tasks

should be trained first because it leads to better transfer of skill and retention of

learning.

Training the students with current-day procedures first allows them to acquire

fundamental ATM skills. In particular, the task of manually scanning the scope for

conflicting aircraft was the primary area that the students reported needing extended

training. Since this critical task was only practiced during the Current Day-First

condition, those in the NextGen-First condition reported feeling unprepared to

manage traffic for their first test halfway through the course. Participants also claimed

that training on the NextGen tools required less attention and therefore caused

trainees to become passive as they let the computer do the majority of the work for

them. Several students in that group reported that the passivity led them to slack off

and not critically monitor their environment. However, students in the Current Day-

First condition said they continued to monitor traffic and tried to detect conflicts

before the conflict alerting system notified them. As such, they were not as reliant on

the conflict detection technology as the NextGen-First condition. More generally, all

students felt that practice with current day tasks was the most effective at developing

a foundation for ATM skills as a whole.

Participants in the NextGen-First condition claimed they felt unprepared in the

mid-term testing for even routine tasks such as acknowledging aircraft check-ins and

giving frequency changes. The added workload needed for them to perform these

routine tasks decreased their ability to perform the more critical tasks of verbally

issuing commands to aircraft and monitoring the scope for conflicting traffic. These

students also reported that the advanced tools training emphasized the computer

system and interface rather than the actual traffic separation and monitoring

techniques. Due to this fact, when students in the NextGen-First condition made the

switch to the current-day training in the second half of the study, they felt as if they

were starting the class all over again. In other words, there was little transfer of

training. This echoes the sentiment of the Current Day-First condition, which stated

that current-day procedures provide a more supportive foundation for learning ATC

tasks. One student noted feeling held back by beginning his training with NextGen

tools. He remarked that training with NextGen tools contradicted the classroom

training because those lessons taught strategies for detecting conflicts manually;

however, during hands-on training with the advanced tools, students were told to

refrain from manual conflict detection because the computer would perform that task

for them. In this sense, the student was not able to put into practice what they learned

in the classroom component of the course.

Most students, irrespective of training condition, indicated that the advanced tools

were easy for them to learn and required little focused attention to use. It is not

surprising, then, that participants felt eight weeks of training on NextGen tools

became redundant. It is important to note, however, that the students did agree that

NextGen tools were worthy of some extended training. In their opinion, the inevitable

introduction of NextGen tools into the ATC domain necessitates sufficient levels of

training in order to bring them up to a proficient level of performance. They also

noted that NextGen tools did provide them with novel ways to separate traffic and

offload some of their workload by reducing their responsibility for predicting

potential conflicts.

3.2 Limitations and Suggestions for Future Training

This study was a preliminary investigation of how the order in which NextGen tools

are introduced affects student learning of ATM tasks. Based on feedback from

students in the course, it is recommended that current-day ATM techniques be trained

prior to the introduction of NextGen tools. However, this recommendation assumes

that NextGen air traffic will consist of both equipped and unequipped aircraft, a

feature of near term NextGen. As more aircraft become equipped with NextGen tools

there may be less need for learning current-day ATM techniques.

The results from this study were only intended to provide an initial input to

research on training of NextGen tools. In fact, we only tested the introduction of

three specific NextGen tools, so the findings may not generalize to other NextGen

technologies. Also, it should be noted that the training of current-day and NextGen

tools occurred only in an environment with all voice or all NextGen equipped aircraft,

respectively. It may be the case that introducing NextGen tools early, in a mixed

equipage environment, may increase the students ability to acquire fundamental

ATM skills while benefiting from the NextGen capabilities. In addition, other training

schedules should be examined in future investigations.

Acknowledgements. This study was supported by NASA cooperative agreement

NNX09AU66A, Group 5 University Research Center: Center for the Human factors

in Advanced Aeronautics Technologies (Brenda Collins, Technical Monitor).

References

1. Joint Planning and Development Office: Concept of Operations for the Next Generation Air

Transportation System Version 1.2, http://www.jpdo.gov/library/NextGenConOpsv12.pdf

(2007)

2. Prevot, T., Homola, J., Mercer, J.: Human-in-the-Loop Evaluation of Ground-Based

Automated Separation Assurance for NextGen. In: Congress of International Council of the

Aeronautical Sciences Anchorage, AK (2008)

3. Strybel, T.Z., Vu, K.-P. L., Bacon, L.P., Kraut, J., Battiste, V., Johnson, W.W.:

Diagnosticity of an Online Query Technique for Measuring Pilot Situation Awareness in

NextGen. In: Digital Avionics Systems Conference, pp. 4.B.1-1--4.B.1-12 (2010)

4. Ligda, S.V., Johnson, N., Lachter, J., Johnson, W.W: Human Interface and the Management

of Information: Pilot Confidence with ATC Automation Using Cockpit Situation Display

Tools in a Distributed Traffic Management Enviroment. In: Salvendy, G., Smith, M. (eds.)

LNCS, vol. 5618, pp. 816--825. Springer, Heidelberg (2009)

5. Dao, A.V., Lachter, J., Battiste, V., Brandt, S.L., Vu, K.-P. L., Strybel, T.Z., Ho, N., Martin,

P., Johnson, W.W.: Automated Spacing Support tools for Interval Management Operations

during Continuous Descent Approaches. In: Human Factors and Ergonomics Society

Annual Meeting Proceedings, pp. 21--25 (2010)

6. Dwyer, J.P., Landry, S.: Separation Assurance and Collision Avoidance Concepts for the

Next Generation Air Transportation System. In: Salvendy, G., Smith, M.J. (eds.)

Proceedings of the Symposium on Human Interface LNCS, pp. 748--757. Springer,

Heidelberg (2009)

7. Morrison, J.E., Fotohui, C.H., Broach, D.: A Formative Evaluation of the Collegiate Training

Initiative. Technical paper DOT/FAA/Am-96/6, Oklahoma City, OK: FAA Civil

Aeromedical Institute (1996)

8. Cavcar, A., Cavcar, M.: New Directions for ATC Training. Int. J. Aviat. Psychol.14, pp.

135--150 (2204)

9. Yacef, K., Alem, L.: Intelligent Tutoring Systems: Student and Expert Modelling for

Simulation-based Training. In: Frasson, C., Gauthier, G., Alan, L. (eds.) LNCS, vol. 1086,

pp. 614--622. Springer, Heidelberg (1996)

10. Vu, K.-P. L., Minakata, K., Nguyen, J., Kraut, J., Raza, H., Battiste, V., Strybel, T.: Human

Interface and the management of Information: Situation Awareness and Performance of

Student versus Experienced Air Traffic Controllers. In: Salvendy, G., Smith, M. (eds.)

LNCS, vol. 5618, pp. 865--874. Springer, Heidelberg (2009)

11. Schneider, W.: Training High-Performance Skills: Fallacies and Guidelines. Hum. Factors

27(3), pp. 285--300 (1985)

12. Sohn, M., Douglass, S.A., Chen, M., Anderson, J.R.: Characteristics of Fluent Skills in a

Complex, Dynamic Problem-Solving Task. Hum. Factors 47, pp. 742--752 (2005)

13. Carlson, R.A., Khoo, B.H., Elliot II, R.G.: Component Practice and Exposure to a Problem-

Solving Context. Hum. Factors 32(3), pp. 267--286 (1990)

14. Kurtz, S., Lee, T.D.: Part and Whole Perceptual-Motor Practice of a Polyrhythm. Neurosci.

Lett. 338, pp. 205--208 (2003)

15. Lee, F.J., Anderson, J.R.: Does Learning a Complex task have to Be Complex?: A Study in

Learning Decomposition. Cognitive Psychol. 42, pp. 267--316 (2001)

16. Young, M.D., Healy, A.F., Gonzalez, C., Dutt, V., Bourne Jr., L.E.: Effects of Training with

Added Difficulties on RADAR Detection. Appl. Cognit. Psychol. (2010)

17. Dao, A.V., Summer, B.L., Battiste, V., Vu, K.-P. L., Strybel, T., Johnson, W.W.: Human

Interface and the Management of Information: The Impact of Automation Assisted Aircraft

Separation on Situation Awareness. In: Salvendy, G., Smith, M. (eds.) LNCS, vol. 5618, pp.

73--747. Springer, Heidelberg (2009)

18. Kraut, J.M., Kiken, A., Billinghurst, S., Morgan, C.A., Strybel, T.Z., Chiappe, D., Vu, K.-P.

L.: Effects of Data Communications Failure on Air Traffic Controller Sector Management

Effectiveness, Situation Awareness, and Workload. To be presented at Human Computer

Interaction International (2010)

19. Prevot, T.: Exploring the Many Perspectives of Distributed Air Traffic management: The

Multi Aircraft Control System MACS. In: Proceedings of the HCI-Aero, pp. 149--254

(2002)

20. Canton, R., Refai, M., Johnson, W.W., Battiste, V.: Development and Integration of Human-

Centered Conflict Detection and Resolution Tools for Airborne Autonomous Operations. In:

Proceedings of the 15th International Symposium of Aviation Psychology, Oklahoma State

University, OK (2005)

Contact this candidate