We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners who may combine it with other information that you’ve provided to them or that they’ve collected from your use of their services. You consent to our cookies if you continue to use our website. Term of use & Privacy Policy

Products

FS #A02:TRACER

 

Background

 

TRACEr and Human Error Reduction in Air Traffic Management (HERA) are Human Error Identification (HEI) techniques developed specifically for use in Air Traffic Control (ATC). They provide means of classifying errors and their causes retrospectively, and to proactively predict error occurrence. TRACEr was developed in 1999 by National Air Traffic Services, UK. Similarly, HERA technique was then developed by EUROCONTROL in 2003 and used NATS as its main input. It aims to determine how and why human errors are contributing to incidents, and thus how to improve human reliability within a high-reliability system.

 

 

 

 

Key Concepts

 

The TRACEr technique is based on the human information processing paradigm which claims that the human mind is like a computer or information processor. It contains eight inter-related taxonomies that altogether describe:

 

• the context of the incident – using:
1. Task Error – describing controller errors in terms of the task that was not performed satisfactorily.
2. Information – describing the subject matter or topic of the error.
3. Performance Shaping Factors (PSF) – classifying factors that have influenced/ could influence the controller’s performance, aggravating the occurrence of errors or assisting error recovery.

• the cognitive background of the production of an error – using:
4. External Error Modes (EEMs) – classifying the external and observable manifestation of the actual or potential error.
5. Internal Error Modes (IEMs) – linked specifically to the functions of the cognitive domains and describing which of the cognitive functions failed/ could fail and in what way.
6. Psychological Error Mechanisms (PEMs) – describing the psychological nature of the IEMs within each cognitive domain.
7. Error detection – describing the error using specific keywords.

• the incident recovery – using:
8. Error correction – describing how the error was mitigated.

 

 

 

 

Benefits

 

• The TRACEr technique provides feedback on organisational performance before and after unwanted events. Its main strengths are comprehensiveness, structure, acceptability of results and usability.
• It requires moderate resources.
• It offers a possibility to derive error reduction measures.
• It helps to determine what errors could occur, their causes and their relative likelihood of recovery.
• It facilitates the identification and classification of human errors in relation to Human-Machine Interaction (HMI)

 

 

 

How It Works

 

Aviation application

 

HERA is developed based on the TRACEr frame consisting of a modular structure.
The analysis can be done in 2 ways: i) Predictive and ii) Retrospective. This document will focus on the retrospective way only, which is useful for Safety Assessment.
For ease of use, the Retrospective method can be represented as a flowchart:

 

 

To conduct retrospective TRACEr analysis, you have to follow 6 steps:

I. Analyse incident into 'error events' – classify the task steps in which an error was produced. Task Error Classification – take the first/next error from the error events list and classify it into a task error from the task error taxonomy
II. Internal Error Modes (IEMs) Classification – decide which cognitive function failed.
III. Psychological Error Mechanisms (PEMs) Classification – identify the 'psychological cause
IV. Performance Shaping Factors (PSFs) Classification – select any
V. PSFs that were evident in the production of the error under analysis.
VI. Error detection and Error correction
a. Error detection – identify errors. 4 questions are used for keywords identification:
1. How did the controller become aware of the error?
2. What was the feedback medium?
3. Did any factors, internal or external to the controller, improve or degrade the detection of the error?
4. What was the separation status at the time of error detection?
b. Error correction/ reduction – identify corrective actions. 4 questions are also used for classification:
1. What did the controller do to correct the error?
2. How did the controller correct the error?
3. Did any factors, internal or external to the controller, improve or degrade the detection of the error?
4. What was the separation status at the time of the error correction?
Once the analyst has completed step 6, the next error should be analysed. If there are no more 'error events' then the analysis is finished.
For Safety Assessment, the Predictive method can be applied in a similar manner.
Maritime Application
The TRACEr methodology was adapted to the maritime sector by the World Maritime University and named TRACEr-Mar , it consists of nine coding steps or classification schemes that can be divided into three main groups, which describe the context of the incident, the operator context (production of the error) and the recovery from the incident. Table 1 provides a brief summary of the nine TRACEr steps.
Illustrative example – ATCO-Pilot Communication
To see how the techniques can be implemented in practice we may consider the following ATCO-Pilot communication scenario which resulted in a loss of separation minima.
SCENARIO: A very busy day, young and unexperienced ATCO, tired because of lack of sleep: ATCO issuing clearance for aircraft A: “ABC123 descend FL240”; Pilot A readback: “ABC123 descend FL220”; ATCO issuing clearance for aircraft B: “XYZ789 climb FL230”; Pilot B readback: “XYZ789 climb FL230”
STCA starts (Short Term Conflict Alert – ground-based safety net intended to assist the controller in preventing collision between aircraft). ATCO: “ABC123 avoiding action turn right.”
ANALYSIS: We will conduct the analysis by following the 6 steps.

 

 

Step 1: Develop the sequence of events and identify 'error events.
To apply TRACEr, we are interested in the ATCO’s errors:
• Controller fails to notice error
• Controller is late to respond to STCA
• Controller gives indication in wrong direction

In our example we will analyse the first 'error event' only as the pattern remains the same for the next ones.
Step 2: Task Error classification: check Task Error taxonomy  Controller Pilot communication error
Step 3: IEM classification: check IEM taxonomy  Readback error
Step 4: PEM classification: check PEM taxonomy  Expectation bias
Step 5: PSF classification: check PSF classification  Traffic complexity
Step 6: Error detection and Error correction

a) Error detection
1. How did the controller become aware of the error?  Action feedback
2. What was the feedback medium?  Radar display
3. Did any factors, internal or external to the controller, improve or degrade the detection of the error?  Busy traffic, unfamiliar task, fatigue
4. What was the separation status at the time of error detection?  2000 ft
b) Error correction/ reduction
1. What did the controller do to correct the error?  Automated correction
2. How did the controller correct the error?  Climb
3. Did any factors, internal or external to the controller, improve or degrade the detection of the error?  Busy traffic, unfamiliar task, fatigue
4. What was the separation status at the time of the error correction?  2000 ft
After completing the 6 steps for the first 'error event', the same steps must be applied for the other 2 identified errors

 

 

Illustrative Example

 

SCENARIO: On December 2012, the dry cargo vessel Beaumont ran aground on Cabo Negro on the north Spanish coast while on passage from Coruna to Aviles. At the time of the grounding she was proceeding at full speed, and the officer of the watch (OOW) was asleep. An inspection of the vessel’s internal compartment quickly established that, despite being driven hard aground on a rocky ledge, there was no breach of the hull. The MAIB investigation identified that the OOW had fallen asleep soon after sending his night lookout off the bridge. Available bridge resources that could have alerted the crew and/or awoken a sleeping OOW were not used resulting in Beaumont steaming at 11.5 knots with no one in control on the bridge for over an hour. The table below presents an example of applying the TRACEr-MAR taxonomy to the task error: 1st Officer fell asleep on the bridge.

 

 

FS #A10:SAFETY CULTURE

 

Background

 

The roots of Safety Culture are spread across several industries (oil and gas, nuclear power, space, air transport, rail, medical and maritime), being first mentioned officially in a report of the Chernobyl nuclear power accident in 1986. The International Atomic Energy Agency (IAEA) used safety culture to explain the organizational conditions that led to the violations of the front-line operators that created the path to disaster. A weak safety culture is seen as a strong contributory factor in various important accidents such as the King’s Cross underground station fire (Fennell, 1998), the Herald of Free Enterprise passenger ferry sinking (Sheen, 1987), the Clapham Junction passenger train crash (Hidden, 1989), the Dryden air crash (Maurino et al. 1995), the Uberlingen mid-air collision (2002), and – more controversially – the two recent Boeing 737-Max air crashes where poor safety culture appeared to apply more to the design, development, validation and oversight phases.

Even though there is no single definition of safety culture, various authors such as the International Nuclear Safety Agency Group (INSAG-4, 1991), Cox and Cox (1991), the UK Health and Safety Commission (HSC 1993) and Guldenmund (2000) agree on what safety culture represents. In broad terms, they all endorse the idea that safety culture embodies the practices, attitudes, beliefs, norms, perceptions and/or values that employees or groups of employees in a company share in relation to managing risks and overall safety. In addition, EUROCONTROL (2008) specified that safety culture can be described as “the way safety is done around here”, suggesting that it needs a practical approach.

Traditionally, safety culture is applied to operational organizations such as air traffic organizations (most in Europe undertake periodic safety culture surveys, certain airlines, and more recently, airports. However, the approach has been applied to one research and development centre where the focus was on designing new operational concepts for air traffic management (Gordon et al, 2007). While it is difficult to directly link equipment design and safety culture, and it is perhaps impossible for a designer to predict the safety culture that may evolve in the future when their design goes into operation, there are some salient points that designers need to consider:

• Safety culture surveys sometimes do raise issues concerning design and the need for design improvement.
• Design of equipment, workplace or interfaces that are cumbersome, difficult to use or complex/confusing may lead human operators to perform ‘workarounds’ that are easier in practice, but may possibly be less safe in certain circumstances.
• Whenever design choices are being made, and there is a safety element, and especially when there is a potential safety-efficiency trade-off, the designer should favour safety. As one senior project manager put it, safety at any cost doesn’t make sense, but safety as a starting point does.
• In stressful or emergency scenarios, where the operators have to make judgement calls between safety and productivity, if indications in the interface are not clear and integrated to help rapid decision-making that errs on the side of safety and caution, then a different outcome is likely to arise. Designers must ensure that, in the heat of the moment, the required safe action will be as clear as it can be.
• Designers must be clear as to which equipment and interface elements are safety critical.Moreover, the chief designer must have a holistic (rather than piecemeal or fragmented) view of the system and the safety critical role of the human operators, as well as a clear idea of the competence of those intended operators.
• The designer cannot ‘leave safety to the safety assessor’, or assume that oversight authorities can pick things up later. This is poor design culture, and, frankly, no one understands design intent better than the designer. The safety role of the human operator must be con sidered from the outset by the designer and be ‘built-in’.
• The designer cannot assume something is fit for purpose based on a few prototyping sessions with a few operational experts. Proper validation with a range of typical end users and abnormal as well as normal scenarios needs to occur.
• Designers should not underestimate the positive impact of design on safety culture, particularly where new and better equipment – user-friendly, making use of human aptitudes – enters a workplace which has hitherto been dogged by equipment problems.
• Designers should read accident and incident reports. This is the only way to look ahead, and to see how well-intended designs can end up in accidents.
• Designers should stay close to real operations and operators as much as possible, and their design organizations should encourage and enable such contact. This is the best way for designers to see around the corner, and to understand the user and their working context and culture as it evolves.


More generally, safety culture (or lack of it) can of course be felt by those working in design organizations. If for example, as a designer or engineer you think you are being pressurized to produce or accept a poorer design because of commercial pressures, who are you going to tell, and will they take you seriously and support your safety concern, or will they bow to the commercial pressure?

 

 

 

 

Key Concepts

 

Safety culture has a number of dimensions, such as the following (Kirwan et al, 2016):

• Management commitment to safety (extent to which management prioritize safety);
• Communication (extent to which safety-related issues are communicated to staff, and people ‘speak up for safety’);
• Collaboration and involvement (group involvement and attitudes for safety management);
• Learning culture (extent to which staff are able to learn about safety-related incidents);
• Just culture and reporting (extent to which respondents feel they are treated in a just and fair manner when reporting incidents);
• Risk handling (how risk is handled in the organisation);
• Colleague commitment to safety (beliefs about the reliability of colleagues’ safety behaviour);
• Staff and equipment (extent to which the available staff and equipment are sufficient for the safe development of work);
• Procedures and training (extent to which the available procedures and training are sufficient for the safe development of work).

Safety culture is typically evaluated using an online questionnaire of validated questions that is distributed across the organisation and filled in anonymously (usually taking about 10 minutes). Based on the responses, analysts then identify the safety culture ‘hotspots’, which are then discussed in confidential workshops, leading to recommendations for the organisation to improve its safety culture. At a macro level, typically a ‘spider diagram’ is produced based on the results, highlighting which safety culture dimensions need most attention. In the example figure to the right, ‘Colleague commitment to safety’ is doing well in safety culture terms, whereas ‘Staffing and Equipment’ requires attention.

 

 

 

 

Benefits

 

Safety culture surveys, particularly when anonymous and carried out by an independent organisation, are often seen as one of the only ways to find out all your risks and what people really think about the state of safety and safety culture in an organisation. They can pinpoint both current issues that can be corrected (so-called ‘quick wins’) without too much trouble, as well as deeper problems that may be entrenched in the organisational culture. Most safety culture surveys do identify the safety culture strengths of an organisation as well as vulnerabilities, so there is always some ‘good news’ as well as areas for improvement. Because the approach uses workshops of operational personnel, usually the recommendations that arise from surveys are useful and practicable, since they come from the organisation itself. The survey process is long, however, often taking 9 months from the first decision to have a survey, to having a report with recommendations. The management need to be committed to doing the process and realise that it is not a quick fix, and there will be work to do afterwards. If management are doing the survey just to gain ‘a tick in the box’, then it may be better not to do one at all.

 

 

How It Works

 

Aviation Sector

There are two main phases used to evaluate the safety culture in a safety culture survey:

Phase 1: Questionnaire – The safety culture survey uses an electronic version of the Safety Culture Questionnaire, over a period of typically 3 weeks. The confidential data are anonymised by The London School of Economics and then independently analysed by EUROCONTROL

Phase 2: (Optional) Workshops – After an initial analysis of the questionnaire results, there is a visit to the organisation’s premises to hold a number of confidential workshops with front-line staff (without managers present) to clarify and explore further certain aspects emerging from the questionnaires. An example of what might be discussed in such a workshop is indicated by the figure below, which shows results for an organisation in the safety culture dimension ‘Staff and Equipment’. With respect to the questionnaire statement ‘We have the equipment needed to do our work safely’, whilst more than 70% were in agreement (the green bar), the rest were either unsure or in disagreement. The workshop would explore equipment issues or concerns people have, which are underlying this response.

 

 

Illustrative Example

 

Illustrative Case Study – Airbus Design

It should be noted that this was seen as an exploratory survey and were focused on a single particular segment of the organization – design [Airbus ‘EY’]– rather than being company-wide (see Kirwan et al, 2019).

For the Airbus survey, the focus was on the Design part of the organization, which involves around 2,000 of the 70,000 employees. This survey required considerable tailoring of the questionnaire, as the job of design and systems engineering is somewhat different from airline operations, even if most underlying issues, except fatigue, and dimensions remain relevant. But the tailoring worked, with the same issues raised during workshops as via the questionnaires. Participation in the workshops by designers from all three of Airbus’ main design locations (France, UK, and Germany) was intense. The general principle espoused in each workshop was that a safety defect in a design of an aircraft could simply not be allowed to happen; therefore, any safety issues had to be resolved carefully and thoroughly before continuing, even if resolution efforts delayed progress toward production. One observation from the study team was that the managers in the management workshop were connected with safety and detailed safety concerns, more so than is often the case for middle managers. This is because many of the managers had held operational positions or positions ‘close to operations,’, so that they understood the operational pressures that could affect airlines and pilots. Another had been to an accident site after an air crash, and noted that after such an experience ‘safety never leaves you.’

The workshops focused a lot on internal communications between the different design departments, as well as the resolution of safety issues that could delay production, and the importance of middle manager support during such resolution periods. A number of constructive recommendations were made to Airbus in the final survey report.

Illustrative Case Study – Safety Culture Assessment

A safety culture assessment and an implementation framework were developed by the University of Strathclyde to enhance maritime safety. The focus was on the development of a safety culture assessment tool which covers all of the safety related aspects in a shipping company and measures will be taken proactively and reactively in order to enhance safety of the shipping industry.

Questionnaires were developed with an interdisciplinary group to ensure that they capture the fight information to conduct a comprehensive analysis. Each safety factor has specific questions which try to address the employee’s attitudes and perceptions amongst crew members and shore staff. There are a total of 85 questions which are asked to the employees, under the headings of communication, employer-employee trust, feedback, involvement, problem identification, promotion of safety, responsiveness, safety awareness and training and competence. The aim of the surveys is to analyse strengths and weaknesses in the company and perform benchmarking between crew and shore staff’s attitudes towards safety. Participants of the surveys were asked questions about demography, safety factors and open-ended questions about company policies. The safety culture assessment framework requires commitment from all of the bodies in a shipping company for successful implementation. Based on the findings from the questionnaires a set of recommendations were provided to the shipping company to improve communication, employer-employee trust, feedback, involvement, mutual trust, problem identification, responsiveness, safety awareness, training and competence, and the promotion of safety.

FS #A14:EYE TRACKING

 

Background

 

Experts can tell us a great deal about the job they do, and how they do it; to a certain degree. They can talk us through situations, or past experiences and walk us through exactly what they did and how they did it. But this inevitably misses half the picture. It focusses on the procedural, rule following aspects of their task. It misses the short glances at pieces of information, it misses the bits of incomplete information that was gathered minutes, or hours before that only arranged itself into a complete picture at the precise moment the operator needed it.

When it gets to the point that someone is so skilled and practised at a task that they don’t realise they are doing a particular action or looking at something for information, it becomes impossible for them to verbalise it. But these are the important information aspects we need to capture to fully understand the nuances of that task and expert performance. These are the elements that make up that ‘expert’. This is where eye tracking comes into its own. Eye tracking can be an extremely useful tool in initial skills capture, and also to assist in measuring task performance and mental workload as it can provide objectivity. It can help us to build that complete picture and provide insights that are vital to fully understanding the complexities of tasks, actions and behaviours. For a designer, it completes the picture of how an operator is truly using an interface – what they find useful and what they do not, in a range of situations.

 

 

 

Key Concepts

 

The key concept of eye tracking is to be able to identify exactly what someone was looking at, when, and for how long. Eye tracking can be used in isolation, or in addition to other methods and techniques such as simulation or as part of a task analysis.

 

 

 

Benefits

 

• Eye tracking outputs can be particularly powerful when communicating results. This is especially true when working on multi-disciplinary projects. A diagram clearly showing how often someone looked at something or the scan path they followed in order to make a decision can paint a very clear picture to non-specialists. As already stated, eye tracking outputs can be used to give us an insight into peoples’ cognitive processes which they would otherwise be unable to communicate.

• Eye tracking outputs, specifically video outputs, can be used during interviews as an aid to the interviewee: “can you explain what you were doing then?” or “what was going through your mind during this action?” The prompt of seeing their actions again ‘through their own eyes’ can often trigger the interviewee to remember specific thoughts or feelings and allow a deeper insight into the actions at that time.

• The main downside of eye tracking is the requirement of specialist equipment, and the need acquire a specific product for analysing the data depending on the eye tracker product. This is a substantial outlay, but they are also available for hire via short-term licenses. The actual use of the eye tracking equipment also has some downsides. Depending on the product used, some of the head mounted glasses-type eye trackers can obstruct peripheral vision as the frames can be quite thick. Some newer models are frameless for this reason. Also, the equipment requires calibration before use, and can easily de-calibrate if the user touches the equipment or moves suddenly.

• Another point for consideration is the analysis of data. Eye tracking produces large quantities of data that quite often requires substantial ‘cleaning’ before they can be analysed. This takes time, but when this is done the actual analysis is fairly straightforward and most of the software packages are user friendly.

 

 

How It Works

 

Eye tracking requires specialist software to both gather and analyse data. The good news is that there are now some extremely good quality, affordable options, which has opened up this area of analysis, which in turn has led to more research in the area and an increased understanding of outputs and metrics.

Typically there are two types of eye tracking device: head-mounted, or screen / panel mounted. Screen or panel mounted devices are typically used where the operator is in a fixed position and normally looking in one direction. These devices are used a lot for User Experience (UX) and web design applications to elicit where a user is looking in order to design webpages more effectively. These devices are also sometimes used in activities such as car driving, as the user is sat in one position and typically looking in one direction. Head mounted eye trackers are worn by the operator and are a lot more versatile as the operator can move freely. These head mounted trackers can resemble a pair of glasses, and have multiple cameras tracking the operator’s pupils and eye movements, and also filming what they are looking at. Both types of device can be used in lab-based or applied settings since they are relatively non-intrusive.

Gathering data is fairly straightforward and depending on the device, can produce many different metrics. The most common eye tracking metrics are based on fixations and / or saccades. Fixations are discrete stable points of where the eye is looking for at least 300ms (and therefore visual information is processed by the brain), and saccades are defined as the eye movements between these fixations (where visual information is not processed). Typically, the first stage of analysis would be to separate the area the operator is working in or looking at into ‘areas of interest’. These then allow you to focus your analysis. Fixation metrics can include number of fixations, fixation durations, total number of fixations in one area of interest, fixation density, and repeat fixations. Fixation metrics can tell us a lot about the operator’s engagement with certain tasks or information. For example, if an operator is focussing on an area for a long time, it could indicate a higher cognitive effort. If an operator is fixating on many different points, it could indicate that the information the operator requires is scattered around, and we should aim to work out why, and if the presentation of information could be improved.

Saccade metrics can provide us with information regarding decision-making, but typically saccades and fixations are used in combination to provide scan-path data. Scan-paths can give us insight into how people go about looking for information, and allow us to analyse how information is gathered and used in certain situations. Scan-paths can be compared between operators, and between novice and expert users to understand the differences between approaches and the impacts these may have. We may be able to identify certain scan-paths that are always straight, and quick, indicating that this task is efficient. Conversely, we may be able to identify scan-paths which are erratic which may help us to understand any aspects of the task or information provision which isn’t currently working as well as it should. These metrics can provide us with some powerful insights into how work is carried out and can greatly assist with redesign of interfaces, displays, and tasks.

 

 

Illustrative Example

 

SCENARIO: As part of a larger project, we wanted to understand how pilots behaved in a complex situation containing multiple incidents. This was carried out in a full motion simulator, and a complex scenario was played out which resulted in the co-pilot flying the plane, a number of go-arounds, a runway change, and low fuel throughout the scenario. From the other data (cockpit data, and voice recordings) and through expert analysis, it was clear there were pilots who dealt with the scenario more successfully than others. By using eye tracking outputs, we were able to develop theories as to why some pilots performed better than others, and attach metrics to the data, such as dwell times on specific instruments, or scan-paths which were repeated time and time again. We were then able to triangulate this with the cockpit transcripts and understand why certain things were looked at at certain times, or indeed, allow us to explore later during interview why they weren’t. This information helped inform more advanced cockpit instrumentation designs, as well as informing emergency training requirements.

As a specific example of the insights afforded by eye movement tracking (EMT), it was possible to see how the Captain and First Officer (FO) were cross-checking each other during the escalating emergency scenario, and how they supported each other. Typically, the FO, whose main fixation was the PFD (Primary Flight Display) was able to ‘offload’ a degree of the situation awareness to the Captain so that he could focus on diagnosing the problems affecting their flight. EMT was also able to clearly pinpoint the moment when each crew realised they were low on fuel, and how they used various cockpit instruments to analyse an electrical failure. In some areas instruments such as the PFD were fully supporting the flight crew, whereas in others there was clearly room for improvement.

 

Illustrative example - Maritime domain

SCENARIO: In this study, navigator’s eye movement on the navigation bridge simulator was analysed during passage through the Istanbul Strait. A measurement device called “EMR-8” was used for eye movement tracking. In the application, three examinee groups were selected and they divided according to their onboard experience level. Each group has 4 examinees in real-time bridge simulator. Group 1 consisted of deck department students who had up to 2.5 months sea experience onboard ship as a cadet. Group 2 included junior officers who had already completed 12 months training on-board ship. Group 3 consisted of oceangoing masters who had wide experiences at sea.

The scenario about passing Istanbul Strait was given to each examinee. The visual field was divided into three parts; inside, outside and others. The inside part has three components; instruments, indicators and an engine telegraph. The outside part has three components; the sea condition, navigational equipment and target ships. The scenario was recorded by eye movement tracking. In the light of findings, Group1 (beginners), gave minimum attention to navigation euqipments, target ships, regulations and sailing routes. Group 2, the examinees had intermediate characteristics as a navigation officer between professionals and beginners. They had enough knowledge and ability to use navigational equipments, target ships, regulations and routes. They had a longer duration on the bridge compared to Group 1. On the other hand, Group 3 had high level experience and knowledge. The examinees gave utmost attention to navigational equipments, sailing routes, target ships and regulations. Group 3 could stand on the bridge for the longest time without losing concentration.

FS #B05:NEUROID

 

Background

 

NEUROID is a technique that employ signals gathered from the operator to characterize Human Factors like Mental Workload, Stress, Vigilance, and Engagement from a neurophysiological perspective. NEUROID aims at providing information linked to the insights of the operators while interacting with the surrounding environments (e.g. aircraft cockpit, ship commands) and executing operative tasks without interfering with them, and it allows to adapting the system itself depending on the operators’ psychophysical states. The NEUROID was developed by UNISAP and validated together with DBL and ENAC in several operational contexts like Automotive, Aviation, and ATM, to determine how the combination of the considered HFs can be used to define the Human Performance Envelope (HPE) of the operator, and consequently employed to monitor the operator while interacting with high-automated system and unexpected malfunctions, or under challenging and demanding situations.

 

 

image courtesy Stress project

 

 

Key Concepts

 

The NEUROID is based on data-mining and machine-learning algorithms to make the HFs neurophysiological models fitting each operator and minimize inter-user variability, therefore obtaining reliable operator’s assessment. More than 88% of all general aviation accidents are attributed to human error. Therefore, NEUROID generates an innovative and systematic approach to quantify and objectively measure HFs by taking into account, at the same time, the behaviours, emotions, and the mental reactions of the operators themselves, and integrating them with the data related to accidents and incidents investigations. The general concept at the base of NEUROID is that brain, body, and operator’s experience are reciprocally coupled, and that accurate assessment of the operator’s can only be achieved through a well-defined combination of all the available data. Each biological activity is regulated by the human Nervous System, therefore variations of such biological activities correspond to internal reactions because of modification of external (environment) and internal (mental, motivations, emotions, etc.) factors. There are numerous types of neurophysiological measures, such as Electroencephalogram (EEG, related to brain activity), Electrocardiogram (ECG, hearth activity), Electrooculogram (EOG, ocular activity), Galvanic Skin Response (GSR, skin sweating), and so on. Such neurophysiological measures can be seen as the physical interface of the NEUROID technique that will enable to gather insights about all the aspects relating to HFs of the operator like Mental Workload, Stress, Vigilance, and Engagement. The key concept of NEUROID is the Human Performance Envelope (HPE) taxonomy: instead of considering one or two single human factors, the HPE investigates a set of interdependent factors, working alone or in combination, which allows to completely characterize the operator. In other words, these concepts are proposed as performance shaping factors, which can differentially and interactively affect successful completion of a task. The HPE theory explicitly declares that boundaries exist where performance can degrade in line with the theoretical underpinnings for the considered HFs.

 

 

Benefits

 

Traditional methods to catch information about the operators’ HFs are usually based on self-reports and interviews. However, it has been widely demonstrated how such kind of measures could suffer of poor resolution due to a high intra- and inter-user variability depending on the nature of the measure itself (i.e. subjective). In addition, the main limitation in using subjective and behavioural measures alone is due to the impossibility of quantifying ‘‘unconscious’’ phenomena and feelings underlying human behaviours, and most importantly it is not possible to measure such unconscious reactions experienced by operators while performing working activities. In fact, the execution of a task is generally interrupted to collect subjective evaluations. On the contrary, neurophysiological measures exhibit an unobtrusive and implicit way to determine the operator’s affective and cognitive mental states on the basis of mind-body relations. Therefore, the benefit and advantage of the NEUROID technique is to use neurophysiological measures and machine-learning algorithms to overcome such limitations, especially to (1) objectively assess the operator’s mental states while dealing with operative tasks; (2) identify the most critical and complex conditions and correlated them with accident and incident investigations; and (3) create a closed-loop between the systems and operators (i.e. Joint Human Machine Cognitive system) to continuously and non-invasively monitor the operators themselves.

Finally, modern wearable technology can help in overcome the invasiveness (e.g. many cables, uncomfortable) of the equipment necessary to collect the neurophysiological signals from the operators.

 

 

 

How It Works

 

NEUROID consists in 3 main phases as reported in the figure. The neurophysiological signals are gather from the operators by wearable technology while dealing with working tasks. Afterward, the neurophysiological data are processed through a series mathematical steps towards the machine-learning phase, in which each HF is classified. Finally, the considered HFs are combined to define the Operator’s HPE, and integrated with the operator’s behaviour, and data related to accidents and incidents (e.g. SHIELD database and HURID framework) for a comprehensive operator’s assessment which can be employed for different applications, for example real-time mental states assessment, or triggering adaptive automations.

 

 

 

 

 

 

 

 

Illustrative Example

 

Illustrative example #1: Aviation

To see how the NEUROID technique can be implemented in practice we can consider the following scenario.

Scenario: An Air Traffic Controller (ATCO) is managing the air traffic and at the same time we are acquiring the ATCO’s EEG, ECG, and GSR neurophysiological signals. In the first phase of the scenario, the ATCO can rely on the support of high-automated systems to manage high traffic situation (High). Then, the traffic demand returns to a normal condition (Baseline), but at a certain moment the automations crash (Malfunction) and the ATCO will have to keep managing the traffic, and find out what is wrong with the automations.

In this context, we can use the NEUROID to measure how the ATCO’s HPE changes. In this regard, the neurophysiological data will be employed to estimate the ATCO’s Mental Workload, Stress, Attention, Vigilance, And Cognitive Control Behaviour based on the S-R-K model. Finally, those HFs will be combined to define the ATCO’s HPE on the three operational conditions as shown in the following figure. In particular, the values of the considered mental states have been normalized within the [0 ÷ 1] range: “0” means Low, while “1” mean High. Concerning the S-R-K aspect, “0” represent the S (Skill), “0.5” the R (Rule), and “1” the K (Knowledge) level.
 

 

 

The analysis of the HPE in the different scenario conditions exhibits interesting variations on the configuration of the mental states. In fact, the failure of the automations (red line) induced higher Vigilance, lower Attention, and shift from Skill to Rule level in the Cognitive Control Behaviour with respect to the HIGH (orange line) and BASELINE (blue line) conditions. Furthermore, considering the HPE changes over time throughout the ATM scenario allows to identify those moments, thus situations, corresponding to high (RICH) or low (POOR) performance. For example, for the considered ATCO, the HPE corresponding to RICH (green plot) and POOR (red plot) performance are reported in the following figure. Generally, in correspondence of RICH performance condition, the vigilance is higher and the stress is lower than during POOR performance. In conclusion, the NEUROID could be used, for example, to monitor the operators and warn them when they are working under low vigilance and high stress condition.

 

 

 

Illustrative example #2: Maritime

To see how the NEUROID technique can be implemented in Maritime practice we can consider the following scenario.

Scenario: The collision occurred between tanker ship and livestock ship during Dardanelle Strait passage at sea. The collision occurred during overtaking. The VTS (Vessel traffic services) was informed about the collision since it occurred during Dardanelle Strait passage. The weather was partly cloudy and sea state was calm at the time of collision. The time was early morning. The officer was keeping the navigational watch at the time of collision. It was early morning at the time of collision.

The HF analysis of the ship collision during Strait passage would highlight the following potential root causes:
• Mental Fatigue: Rather than physical fatigue, over alertness and overload (decision making, concentration, mental load). Watchkeeper Officers continuously working load can affect his/her decisions.
• Physical Fatigue: Rather than mental fatigue, physical fatigue can be seen due to intensive work load on-board ship.
• Inadequate Manning on Bridge: According to sailing zone, increase the number of watchmen on bridge may help perception of the dangers earlier.
• Inadequate Policy, Standards and Application: Increase of the inspections due to sailing rules and minimum CPA-TCPA values and activation of the existing rules may help avoiding dangerous situations and provide clearance from the plotted vessels.
• Inadequate / Lack of Communication: Necessary manoeuvre could not be applied due to inadequate communications between vessels. Lack of communications between vessels and VTS also caused vessels to sail in close range. Every communication methods on board must be managed in most efficient manner.
• Poor Risk Evaluation: Main reasons of the incident between vessels are the failure of the evaluation of risks for all vessels in the sailing area and unforeseen of the other vessel possible manoeuvres which were sailing in the outmost line of the separation.
• Inadequate Leadership and Supervision / Planning: Another root causes of the incident are improper planning of watch hours and not taking necessary measurements as a result of evaluations of inconveniences timely.
• External/Environmental Conditions: Sailing in narrow zone, congestion and poorness of the VTS stations in planning the traffic are the main effective reasons of the incident. To prevent such like incidents, environmental conditions must be evaluated sufficiently and proper speed and watch order must be planned according to congestion.
Some of those aspects are mainly linked to the management level (e.g. manoeuvres procedures, risk evaluation), but the ones coming from the evaluation of the operators’ status like the mental and physical fatigue can be monitored and measured via the NEUROID during the working activities.

In particular, along each phase of the considered scenario the neurophysiological measures can characterise the operators in terms of Mental Workload, Stress, Vigilance, and Engagement combination and those information can be consequently used to 1) interact and\or intervene in real-time on the system, for example depending on the Master's mental workload it could be possible to enable a warning during overload or low vigilance conditions to regain the proper operational status; 2) post-doc analyse the incident\accident or simulation by combining the previous tasks analysis, and the operators' mental states profile along the scenario, for example to find out how the Mental Workload, Stress, Vigilance, and Engagement of the Master and C/O were right before the collision or during a specific phase which brought to the collision, and finally understand if the configuration of those mental states was not appropriate (e.g. distracted?) or, on the contrary, the operators were under too high demanding and stressing conditions due to the adopted procedure or situation.

FS #B06:LOAT

 

Background

 

The Levels of Automation Taxonomy (LOAT) classifies and compares different kinds of automation. It was originally developed in the context of the SESAR Programme by analysing 26 different automated functionalities for the Air Traffic Control and the flight crew. The taxonomy is grounded on the seminal works of Sheridan and Verplanck (1978). A new model was developed in order to overcome limitations encountered when applying the theory in practical situations. This formed the basis to develop a set of Automation Design Principles. The LOAT has 2 main purposes: i) provide potential design solutions with lower/higher level of automation; ii) help classify automation examples and provide specific human centred suggestions.

 

 

 

 

Key Concepts

 

The LOAT brings into discussion important matters related to systems’ innovation.

• Automation is not only a matter of either automating a task entirely or not but deciding on the extent to which it should be automated.
• Introducing automation brings qualitative shifts in the way people practice rather than mere substitutions of pre-existing human tasks.
• Classification of different levels of automation, recognises various ways in which human performance can be supported by automation.
• The “optimal” level of automation in a specific task context is about matching the automation capabilities to a number of operational situations, while increasing the overall performance in efficient human-machine cooperation.
• It provides potential alternatives for designing the human-machine interaction by taking full benefit of the technical solution.

 

Benefits

 

The LOAT can be used to:

✓ Compare different design options and determine the optimal automation level in all operational contexts. The best level of automation depends on the capabilities of the automation itself and on the operational context in which it is included. You can use the LOAT to orient yourself in these choices.
✓ Understand the impact of automation, both positive and negative, on the system. The highest level of automation is not always the best one. Finding the right level helps to make sure that you are taking the full benefit from it, without being negatively affected by side effects, such as nuisance alerts or erroneous directions.
✓ Get insight on how to prevent human performance issues and take full benefit of available technical solutions in various domains. A common taxonomy for different domains will mean a shared approach to automation design. Developed in the Air Traffic Management domain but applicable to all domains in which automation plays an important role in sustaining human performance
✓ Classify existing implementations and derive lessons to improve automation design. Classifies the ‘nuances’ of automation enlightening the possibilities in design. Modifying existing tasks or introducing new ones involve the use of different psychomotor and cognitive functions, which implies the adoption of different automation solutions.

 

 

 

How It Works

 

The LOAT is organized as a matrix:

I. in horizontal direction: the 4 generic functions, derived from a four-stage model of the way humans process information:
1) Information acquisition: Acquisition and registration of multiple sources of information. Initial pre-processing of data prior to full perception and selective attention;
2) Information analysis: Conscious perception, manipulation of information in working memory. Cognitive operations including rehearsal, integration, inference;
3) Decision and action selection: Selection among decision alternatives, based on previous information analysis. Deciding on a particular (‘optimal’) option or strategy;
4) Action implementation: Implementation of a response or action consistent with the decision previously made. Carrying out the chosen option.
II. in vertical direction: each cognitive function groups several automation levels (between 5 and 8). All automation levels start with a default level ‘0’ - corresponding to manual task accomplishment - and increase to full automation.
 

The use of the LOAT incorporates 4 steps:
1) Identify the automated tool to be classified;
2) Determine which function is supported (can be more than one);
3) Determine the relevant cluster of automation levels;
4) Identify the specific automation level and consider the design principles associated to it. The main LOAT goals are:
• it comes in the aid of a human, increasing their performance through human-machine cooperation;
• it is a method by which we can efficiently identify the most suitable human-automation design;
• Based on the classification of already developed automation examples we can provide specific and relevant human factor recommendations.

 

 

Illustrative Example

 

Illustrative example: Remote towers

To see how the LOAT can be implemented in practice we should consider an example of human-machine interaction in industry.

ANALYSIS: Following the aforementioned steps, we can use the LOAT to identify the level of automation.
Step 1: Identify the automated tool to be classified. From the technologies of the RTC mentioned in the SCENARIO we choose the one we want to classify: e.g. Surveillance
Step 2: Determine which function is supported. We should then consider the main purposes of surveillance cameras: i.e. they collect images. Considering this, the function supported is: Information Acquisition (A).

 

Step 3: Determine the relevant cluster of automation levels. While the cameras are the ones collecting and generating the image to the user but only on the user’s demand, the relevant automation level is: supported by Automation.

 

Step 4: Identify the specific automation level. The user is still in control to decide whether the information received from the cameras is relevant or not, and can filter the information received, so the automation level is: with user filtering and highlighting of relevant info (A2)

 

In this way the LOAT can be used to decide upon the optimal level of automation suitable for the user and can help to establish whether or not it is a good choice to automate the system in the first place.

Would like to know more ?