We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners who may combine it with other information that you’ve provided to them or that they’ve collected from your use of their services. You consent to our cookies if you continue to use our website. Term of use & Privacy Policy

Products

FS #T01 Human HAZOP - Human Hazard and Operability Study

 

Background

HAZOP (Hazard and Operability Study) was developed in the early 1970s by Professor Trevor Kletz and his colleagues at the UK chemical company ICI, as a means of safety assurance of chemical installations. It is essentially a ‘what-if?’ approach, using design and operational experts to rigorously analyse a design or an operational procedure to determine what could go wrong, and how to prevent it going wrong in the future. HAZOP quickly spread to other industries as a robust means of checking a design for safety problems. In the 1990s it transitioned to the consideration of human error and the design of human machine interfaces, for example in the air traffic industry. 

The HAZOP process is exhaustive and has been shown many times to be effective. It does not place high demands on analytic expertise as certain other techniques do, instead offering a structured way of interrogating a design. It results in a table highlighting any potential vulnerabilities (these can be ranked or weighted in terms of severity if they were to occur), and design or operational remedies to mitigate the risk.

Human HAZOP focuses on safety, but it also identifies design issues related to productivity and system performance, and so can also lead to efficiency and effectiveness enhancements. This is reflected in the word ‘Operability’ in the title of the technique. Designers often find HAZOP approaches to be an excellent means of quality assurance for their designs.

 

 

Key Concepts

Human HAZOP is a Group method, rather than a single-analyst approach such as TRACER or STAMP.

It relies on ‘structured brainstorming’ by a small group (e.g. 6-8) design, operational and Human Factors experts, led by a HAZOP chairman

It requires a representation of the system, procedure or interface being evaluated. Typically design documents and drawings (or photos if operational) are present, along with a task analysis 
   (typically Hierarchical Task Analysis or Tabular Task Analysis which details how the human operators are intended to interact with the system.

The HAZOP study group proceeds through each step of the procedure or task analysis and considers potential deviations from the expected behaviour, prompted by HAZOP guidewords.

The consequences of deviations from the intended functioning of the system are considered, as well as existing safeguards & recovery means

Required safety recommendations are stated and recorded by the HAZOP secretary.

HAZOPs are formally recorded and logged.

 

 

 

 

Benefits

HAZOP has survived a long time, and has a long list of benefits:

Human HAZOP provides systematic and exhaustive design review, and can lead to the discovery of new hazards.

It requires limited technical training – it is an ‘intuitive’ method.

The use of a team gives a range of viewpoints.

It has a good track record in several industries.

It is versatile, and can be applied to all sorts of design formats.

It can be used early on (e.g. at the concept design stage).

It is good at finding credible vulnerabilities, including violations.

It has high acceptance by both designers and operators, as they both input to the HAZOP.

It creates a good ‘bridge’ between design and operations.

 

On the negative side, there are several well-known disadvantages:

HAZOP, especially if applied to a complete system, is resource intensive.

The GIGO (garbage in, garbage out) principle applies. If, for example, inexperienced operational people are used as experts, don’t expect too much.

HAZOP tends to concentrate on single deviations or failures – it does not look at how different failures might interact – only risk modelling does this.

 

 

How It Works

Human HAZOP works by applying a set of guidewords to the chosen design representation, be it design drawings and documentation, task analyses, or more dynamic visualisations (e.g. working prototypes) of the system under investigation.

The guidewords are as follows shown in the table below. As an example, the most commonly used guideword is ‘No’, meaning the human operator does not do something when required (e.g. failing to lower landing gear). Guidewords such as More or Less could be applied to entering correct weight into software used to manage balance (aviation) or ballast (maritime). Reverse is rarer but can still happen, as in the AF447 air crash where the co-pilot manoeuvred the controls in the wrong direction, possibly due to startle response and inexperience, putting the plane into an irrecoverable stall. In maritime it could mean setting thrusters in the wrong direction when docking. Early and Late could relate to use of flaps or deciding when to descend an aircraft, or when to turn to pass another vessel (maritime). The most interesting guideword is ‘Other than’, wherein a completely unintended action is the result. This requires operational experience rather than fanciful imagination, and typically in a HAZOP only one or two (or none) such possibilities are identified. 

In the HAZOP session, the HAZOP Chairman leads the group through each step of the activity associated with the interface or procedure, and each guideword is considered in turn. 

The results of a HAZOP are logged in a HAZOP table by the HAZOP secretary. Mitigations may not always be apparent, in which case actions are given to HAZOP team members or their departments to develop mitigations and report back to the group or design authority to ensure they are sufficient and mitigate the hazard.

 

 

Illustrative Example

Aviation domain
The new Air Traffic Controller systems shift the focus away from one primary form of information presentation and usage (paper) to a completely different presentation medium (computer screen).  Additionally, although both these media are visual in nature, the computerisation of flight progress strips paves the way for automated ‘up-linking’ of messages to the cockpit of aircraft, reducing the need for oral communication for a number of tasks. This transmission of information is known as ‘data-link’. Given the high performance and proven nature of the current system, it is sensible to evaluate the transition to a new system interface.  A HAZOP analysis was therefore conducted on the proposed HMI (Human Machine Interface) of the new (electronic) flight progress information system. 

The HAZOP team that assessed the implications of the new interface was made up of three designers, one air traffic controller and two human factors specialists. The study identified a number of ‘vulnerabilities’ in the prototype system and ‘opportunities’ for error that needed to be designed out or worked around (e.g. via procedures and training).  A total of 87 recommendations, covering changes to the HMI, improved feedback and training / procedure improvements were generated in the three HAZOP sessions.  

The HAZOP approach proved surprisingly useful and productive of agreed changes in interface design. Moreover, since the designers were not only present, but were actively involved, any design changes they thought necessary were simultaneously accepted for implementation. The HAZOP therefore had very fast and effective impact on the design process. To illustrate the meeting output, part of the study is shown in the table below. 

  

Maritime domain

In shipping, the techniques to represent the geophysical characteristics of the seabed have shifted the focus away from paper-based nautical charts to computerized electronic charts such as sonar charts. A sonar chart is an HD bathymetry map featuring extraordinary bottom contour detail for marine and lakes, excellent for increasing awareness of shallow waters and for locating fishing areas at any depth level.

Although both techniques are visual in nature, by computerising the seabed mapping, it is possible to increase the visualization details and to reduce the need for communication and coordination amongst crew members. Therefore, a Human HAZOP analysis can be conducted on the proposed human-machine interface of the new electronic system. The outcomes of this study were used to identify a number of ‘vulnerabilities’ in the proposed system and ‘opportunities’ for error that needed to be designed out or worked around (e.g. via training in the use of sonar chart).To illustrate the HAZOP output, an example is shown in the table below.

 

FS #T02: TRACEr

 

Background

TRACEr and Human Error Reduction in Air Traffic Management (HERA) are Human Error Identification (HEI) techniques developed specifically for use in Air Traffic Control (ATC). They provide means of classifying errors and their causes retrospectively, and to proactively predict error occurrence. TRACEr was developed in 1999 by National Air Traffic Services, UK. Similarly, HERA technique was then developed by EUROCONTROL in 2003 and used NATS as its main input. It aims to determine how and why human errors are contributing to incidents, and thus how to improve human reliability within a high-reliability system.

 

 

 

Key Concepts

The TRACEr technique is based on the human information processing paradigm which claims that the human mind is like a computer or information processor. It contains eight interrelated taxonomies that altogether describe:

The context of the incident – using:
1. Task Error – describing controller errors in terms of the task that was not performed satisfactorily.
2. Information – describing the subject matter or topic of the error.
3. Performance Shaping Factors (PSF) – classifying factors that have influenced/ could influence the controller’s performance, aggravating the occurrence of errors or assisting error recovery.

The cognitive background of the production of an error – using:
4. External Error Modes (EEMs) – classifying the external and observable manifestation of the actual or potential error.
5. Internal Error Modes (IEMs) – linked specifically to the functions of the cognitive domains and describing which of the cognitive functions failed/ could fail and in what way.
6. Psychological Error Mechanisms (PEMs) – describing the psychological nature of the IEMs within each cognitive domain.
7. Error detection – describing the error using specific keywords.

The incident recovery – using:
8. Error correction – describing how the error was mitigated.

 

 

 

 

Benefits

• The TRACEr technique provides feedback on organisational performance before and after unwanted events. 
   Its main strengths are comprehensiveness, structure, acceptability of results and usability.
• It requires moderate resources.
• It offers a possibility to derive error reduction measures.
• It helps to determine what errors could occur, their causes and their relative likelihood of recovery.
• It facilitates the identification and classification of human errors in relation to Human-Machine Interaction (HMI).

 

 

 

How It Works

TRACEr can be used in two different ways, for a retrospective analysis (i.e. following an incident or accident) or for a predictive analysis, in order to anticipate the possible errors, in the context of a risk analysis. For the sake of brevity only the retrospective use is here illustrated, but in two different instances: first the original one, developed in the air traffic control domain, then a modified one adapted to the maritime domain.

Aviation Application
The process for applying TRACER retrospectively in the ATC domain can be represented as a flowchart, and identifying six different steps.

 

Analyse incident into 'error events', identifying the task steps in which an error was produced. 
Task Error Classification, classifying the error with the Task Error taxonomy 
Internal Error Modes (IEMs) Classification, deciding which cognitive function failed.
Psychological Error Mechanisms (PEMs) Classification, identifying the psychological cause.
Performance Shaping Factors (PSFs) Classification, selecting any PSFs related to error under analysis.
Error detection and Error correction, identifying errors/ corrective actions by answering to four questions.

Once the analyst has completed step number six, the next error should be analysed. If there are no more “error events” then the analysis is finished.

Maritime Application
The TRACEr methodology was adapted to the maritime sector by the World Maritime University and named TRACEr-Mar, it consists of nine coding steps or classification schemes that can be divided into three main groups, which describe the context of the incident, the operator context and the recovery from the incident. The overall process can be summarized with a flowchart and table below provides a brief summary of the nine TRACEr steps.

 

 

Illustrative Example

 

Aviation domain

To see how the technique can be implemented in practice we may consider the following ATCO-Pilot communication scenario which resulted in a loss of separation minima.

SCENARIO:  
A very busy day, young and unexperienced ATCO, tired because of lack of sleep. ATCO issuing clearance for aircraft A: “ABC123 descend FL240”; Pilot A readback: “ABC123 descend FL220”; ATCO issuing clearance for aircraft B: “XYZ789 climb FL230”; Pilot B readback: “XYZ789 climb FL230”.

STCA starts (Short Term Conflict Alert – ground-based safety net intended to assist the controller in preventing collision between aircraft). ATCO: “ABC123 avoiding action turn right.”

A representation of this scenario is illustrated in the figure below.

ANALYSIS focused on the first 'error event' (ATCO’s error): 

Step 1: Develop the sequence of events and identify error events.
. Controller fails to notice error
. Controller is late to respond to STCA
. Controller gives indication in wrong direction

Step 2: Task Error classification 
. Controller Pilot communication error

Step 3: IEM classification
. Readback error

Step 4: PEM classification
. Expectation bias

Step 5: PSF classification
. Traffic complexity

Step 6: Error detection and Error correction

 

 

Maritime domain 

SCENARIO: On December 2012, the dry cargo vessel Beaumont ran aground on Cabo Negro on the north Spanish coast while on passage from Coruna to Aviles. At the time of the grounding she was proceeding at full speed, and the Officer Of the Watch (OOW) was asleep. An inspection of the vessel’s internal compartment quickly established that, despite being driven hard aground on a rocky ledge, there was no breach of the hull. The MAIB investigation identified that the OOW had fallen asleep soon after sending his night lookout off the bridge. Available bridge resources that could have alerted the crew and/or awoken a sleeping OOW were not used resulting in Beaumont steaming at 11.5 knots with no one in control on the bridge for over an hour. 

The table below presents an example of applying the TRACEr-MAR taxonomy to the task error: Officer Of the Watch fell asleep on the bridge.

 

FS #T03/B: HEART Human Error Assessment and Reduction Technique

Background

HEART is a first-generation technique developed by Williams for the nuclear industry, and used in the field of Human Reliability Assessment (HRA) for the purposes of evaluating the probability of human error occurring throughout the completion of a specific task. From such analyses measures can then be taken to reduce the likelihood of errors occurring within a system and therefore lead to an improvement in the overall levels of safety.

 

 

Key Concepts

HEART method is based upon the principle that every time a task is performed there is a possibility of failure and that the probability of this is affected by one or more Error Producing Conditions (EPCs) (e.g. distraction, tiredness, etc.) to varying degrees.

Within HEART, those factors which have a significant effect on performance are of greatest interest. These conditions can then be applied to a "best-case-scenario" estimate of the failure probability under ideal conditions to then obtain a final error chance.

By forcing consideration of the EPCs potentially affecting a given procedure, HEART also has the indirect effect of providing a range of suggestions as to how the reliability may therefore be improved and hence minimising risk.

Benefits

On the positive side, HEART presents a long list of benefits:

HEART is rapid and straightforward to use.
HEART has a small demand for resource usage.  
It has a good track record in several industries such as nuclear, healthcare, rail, maritime, offshore and aviation industries. 
It can be used early on (i.e. at the development stage).
Several validation studies have been conducted and concluded that this technique produce estimations that are equivalent to those obtained by applying more complicated methods.

On the negative side, HEART also presents several well-known disadvantages:

The EPC data has never been fully released and it is therefore not possible to fully review the validity of it.
HEART relies to a high extent on expert opinion, first in the point probabilities of human error, and also in the assessed proportion of EPC effect. The final Human Error Probabilities are therefore very sensitive.

 

 

How It Works

HEART is an Additive Factors Model (AFM) that proposes the application of a multiplicative impact on human reliability calculations. This technique works under the assumption that any estimated reliability of task performance may be modified according to identified EPCs.

The HEART methodology generally consists of two assessment process steps (a qualitative and a quantitative step).

The qualitative method is started by classifying the generic task based on the accidents data report.  After finding the general task, it then breaks it down to smaller parts called Error‐producing conditions (EPCs). 

Once this task description has been constructed, the second step of this methodology is the quantitative method, wherein Human Error Probability (HEP) is calculated , usually by consulting local experts. To obtain the HEP using the HEART methodology, first it is necessary to obtain the nominal human unreliability, which belongs to the generic task. When the EPCs are determined, the multiplication number of each EPC is obtained.

 

 

 

Illustrative Example

Aviation domain
This example is derived from an application of CARA (Controller Action Reliability Assessment), an adaptation of HEART specific for the Air Traffic Control domain). In this case CARA is used to asses human performance for anticipated landing clearances during low visibility.

During periods of low visibility, for aircraft landing at airports, a support system was being designed to enable controllers to be provided with information that runways were clear and available for aircraft to land on when they may not be visible from the tower. One key area around the runway in this process is the ‘Obstacle Free Zone’ (OFZ) for which consideration was being given as to the form of alert a controller would receive if the following hazard occurred “The aircraft which landed and is moving clear of the runway stops beyond the trigger line (which enables the next aircraft to start to approach the runway) but not clear of the OFZ with the landing aircraft less than 200 feet above threshold.”

Simple event trees were developed and focused on human error probabilities of two key ATC tasks:

1. Controller identifies the aircraft which has landed is stationary within the OFZ.
2. A warning is issued to the landing aircraft that the aircraft is stationary in the OFZ.

To populate the event tree, CARA values were calculated for these two tasks. The values were produced by two human factors experts and drew on human reliability analysis and event trees from previous hazard analysis work. For task 1, three potential designs were considered and values calculated separately:

1. Use of an audible and visual alert
2. Use of a visual alert only
3. Use of no alert, and relying on controller scanning patterns

Examples of calculations of the human reliability are presented below.

Issue warning to aircraft 2 (Aircraft approaching runway)
Task Description and Assumptions: having identified that aircraft one (the aircraft already landed) is stationary, the controller must identify and warn aircraft 2. The error under consideration is that the controller does not warn aircraft 2, having identified that AC 1 is stationary.

 

Results: Calculation  = 0.003 x [(11-1) x 0.2 + 1] = 0.003 x 3 = 0.009

The analysis provided the following quantified outcomes for the Hazard using the event trees for the three design scenarios:

Use of an audible and visual alert - 9.4E-03
Use of a visual alert only - 1.4E-01
Use of no alert, and relying on controller scanning patterns 3.1E-01

The analysis identified that much better reliability would be achieved using option A, a visual and audible alert, as there is not a reliance on the controller to scan the display to identify the hazard. It was beyond the scope of this work to validate and take these data forward in to the system design.

 

Maritime domain
In the illustrative example, the case of ship collision during passage through the waterway (narrow canal) is applied. Human based factors are analysed to systematically predict human error for each task. The collision occurred between a tanker ship and a livestock ship during the narrow canal passage. The tanker ship which were steering with 11,7 knot speed to the course of 244° in Marmara Sea in the direction of Dardanelles, have contacted with her starboard bow to port stern quarter of the livestock ship within 20 nm range of Dardanelles at 06:25 LT on the date of 01.10.2013. Development of the event was as follows; the tanker ship overtook the bulk carrier ship when she was steering close to the middle of the separation in the direction of Dardanelles. It has been observed that livestock ship was steering in the starboard bow of bulk carrier ship in direction of Dardanelles and close to outmost line of the separation where she was also in the range of about 0,6 NM with the tanker ship. It also has been observed that another ship (container ship) was steering in the starboard bow of the tanker ship within the range of 1 mile. 

At the result of investigation of ECDIS records and statements of the officer on watch, it has been perceived that; After taking over bulk carrier ship with the range of 0,3 NM and heading towards to container ship, the tanker ship has contacted with livestock ship which were steering outmost line of the separation. Even though small course alteration to port side that was made by the tanker ship, contact couldn’t be prevented due to sudden and big course alteration to the port side had been made by the livestock ship. The root cause is attributed to the mental fatigue, inadequate manning on the bridge, inadequate standards, lack of communication, poor risk evaluation, delayed collision avoidance manoeuvring, limited time, lack of knowledge and inadequate checking. 

The VTS (Vessel traffic services) was informed about the collision. The weather was partly cloudy and sea state was calm at the time of collision. The time was early morning. The officer (OOW) was keeping navigational watch at the time of collision. The Master of ship was informed just before of the collision. Helmsman was called by the Master before collision. The hierarchical task analysis was performed to understand relative activities/tasks during ship strait passage. Then, HEART is applied for quantification of human errors. Following table shows details including tasks, responsible person, GTT, EPC, APOE and calculated HEP for the case of ship collision during the narrow canal passage.

 

FS #T04/B CREAM - Cognitive Reliability and Error Analysis Method

 

Background

The CREAM was developed by Hollnagel (1998) to analyse Cognitive Human Error and Reliability. The method enables users to assess the probability of human errors during the completion of a specific tasks. The aims of the technique are to identify human errors, quantify human errors, and minimize human errors. In order to achieve these aims, the technique performs both retrospective and prospective analysis. The method comprises a basic and an extended version. The basic one performs initial screening of human interactions while the extended version performs comprehensive analysis for human interactions by building on the output of the basic version. 

The technique has four different control modes to ascertain human failure probabilities in various actions. The concepts of the control modes were derived from the Contextual Control Model (COCOM) whose aim is to yield a practical and conceptual basis to improve human performance (Hollnagel, 1993). The control modes are scrambled, opportunistic, tactical, and strategic controls respectively. While the strategic control mode represents the lowest human error probability, the scrambled control mode gives the highest HEP.

 

 

 

Key Concepts

The key concepts of the techniques are:

Task analysis: To define the activities to achieve the goal. 

Common Performance Conditions (CPC): Human cognition and action context are determined in accordance with CPCs which provide a well-structured and comprehensive basis for characterizing the circumstances under which performance is expected to occur.

Control modes: Scrambled, Opportunistic, Tactical, Strategic modes. The control modes are linked with different failure probability intervals representing human action failure probabilities.

Cognitive activities: These are relevant for work in process control applications and also provide a pragmatic definition for each. Communicate, compare, diagnose, execute, monitor, observe, etc. are some of the cognitive activities. 

Context Influence Index (CII): This value can be calculated by deducting the number of “reduced” CPCs (i.e. CPCs likely to have a negative impact on performance in the situation) from improved CPCs.

Performance Influence Index (PII): Specify proper weighting factors for cognitive functions such as execution, planning, observation, and interpretation.

Cognitive failure probability (CFP):  This is probability of failure for each cognitive failure type.

The main focus of CREAM technique is based on 2 elements:

human performance - how a system behaves over time and 
• cognition activity description of cognition activity in the system/process.

 

 

Benefits

The technique presents a quantitative approach to systematically predict human error for designated tasks and ascertain the desired safety control level. It offers a clear and systematic approach to quantification. The quantification problem is, in fact, considerably simplified because CREAM is focused on the level of the situation or working conditions rather than on the level of individual actions. 

The technique is based on a fundamental distinction between competence and control. A classification scheme clearly separates genotypes (causes) and phenotypes (manifestations), and furthermore proposes a non-hierarchical organisation of categories linked by means of the sub-categories.

 

 

 

How It Works

The technique includes both basic and extended versions. 

The purpose of the basic method is to produce an overall assessment of the performance reliability that may be expected for a task. The basic one consists of the following steps:

1. Scenario development: This step provides a variety of scenarios including time availability, condition of the work place, the environment, stress, noise level, collaboration of the crew, etc. The CPCs are determined according to the scenario.

2. Task analysis of the process/work: The objective of this step is to identify the tasks and sub-tasks of the process according to the hierarchical task analysis (HTA).

3. Determine CPC: These are used to characterise the overall nature of the task. The combined CPC scores are needed to figure out human performance. There are nine CPCs defined in the CREAM: Adequacy of organisation, working condition, adequacy of the man-machine interface, availability of procedures/plans, number of simultaneous goals, available time, time of day, adequacy of training and experience, quality of crew collaboration. For each task, a CPC level is determined.

4. Determine control mode: The control mode is determined from the combined CPC score. The control mode corresponds to a region or interval of action failure probability.

The purpose of the extended version of CREAM is to produce specific action failure probabilities. The following steps are used:

1. Scenario development

2. Task analysis of the process/work

3. Determine CPC

4. Identify context influence index (CII): The CII is used to quantify CREAM, in particular CPCs, in order to simplify calculation. This value can be calculated by deducting the number of reduced CPCs from improved CPCs (Akyuz, 2015; He et al., 2008).

5. Determine performance influence index (PII): This step provides PII values which were generated to specify proper weighting factors for entire cognitive functions such as execution, planning, observation, and interpretation. Each CPC has a different PII value which means there are different weighing factors (He et al., 2008)

6. Calculate cognitive failure probability (CFP): The CFP specifies human failure probabilities for each cognitive failure type in order to calculate HEP value. Once the nominal cognitive failure probability (CFP0) has been nominated for each sub-task, the CFP value (HEP) can be calculated respectively (Akyuz, 2015). The following equation is used.

CFP=CFP0 x 100.26. CII     

 

 

 

Illustrative Example

Maritime domain
In the illustrative example, the case of ship collision during passage through a strait is handled. Human factors are analysed to systematically predict human error for each task. The extended version of CREAM technique is used.  

Step 1 Scenario development
The collision occurred between a tanker ship and a livestock ship during the Dardanelle Strait passage at sea. The collision occurred during overtaking. The VTS (Vessel traffic services) was informed about the collision since it occurred in the Dardanelle Strait passage. The weather was partly cloudy and sea state was calm at the time of collision. The time was early morning. The officer was keeping navigational watch at the time of collision. 

Step 2 Task analysis: 
Hierarchical task analysis was created covering potential activities/works during ship strait passage.   

Step 3 Determine CPC
Assessor determined appropriate CPCs for each task according to the scenario and crew condition.

Step 4 Identify context influence index (CII): 
Assessor determined CII values for each task

Step 5 Determine performance influence index (PII): 
Assessor determined PII values for each task. 

Step 6: Equation was used to calculate the human error probability (CFP) 
for each task.

Following table shows task analysis including responsible person, relevant cognitive activity, cognitive function and calculated CFP for the case of ship collision during the strait passage (He at al., 2008; Apostolakis et al., 1988).

Step 7: Identify significant human error: 
Analyse which of the sub-task of the ship strait passage need further details and if there is any control measure to mitigate human error. 

In the view of analysis, sub-task 2.2 (Proceed inside the separation line) has the highest cognitive error rate where OOW (Officer on Watch) and helmsman failed to observe monitoring ship position. Likewise, sub-task 3.3 (Increase ship manoeuvring ability by consuming diesel oil and steering engines) is another critical activity where human error increased. The Master failed to enhance the ability of ship manoeuvring by consuming fuel oil (instead of diesel oil) during strait passage.  

 

 

FS #A14: Eye Tracking

 

Background

Experts can tell us a great deal about the job they do, and how they do it; to a certain degree. They can talk us through situations, or past experiences and walk us through exactly what they did and how they did it. But this inevitably misses half the picture. It focusses on the procedural, rule following aspects of their task. It misses the short glances at pieces of information, it misses the bits of incomplete information that was gathered minutes, or hours before that only arranged itself into a complete picture at the precise moment the operator needed it.

When it gets to the point that someone is so skilled and practised at a task that they don’t realise they are doing a particular action or looking at something for information, it becomes impossible for them to verbalise it. But these are the important information aspects we need to capture to fully understand the nuances of that task and expert performance. These are the elements that make up that ‘expert’. This is where eye tracking comes into its own. Eye tracking can be an extremely useful tool in initial skills capture, and also to assist in measuring task performance and mental workload as it can provide objectivity. It can help us to build that complete picture and provide insights that are vital to fully understanding the complexities of tasks, actions and behaviours. For a designer, it completes the picture of how an operator is truly using an interface – what they find useful and what they do not, in a range of situations.

 

 

 

Key Concepts

The key concept of eye tracking is to be able to identify exactly what someone was looking at, when, and for how long. Eye tracking can be used in isolation, or in addition to other methods and techniques such as simulation or as part of a task analysis.

 

 

 

Benefits

• Eye tracking outputs can be particularly powerful when communicating results. This is especially true when working on multi-disciplinary projects. A diagram clearly showing how often someone looked at something or the scan path they followed in order to make a decision can paint a very clear picture to non-specialists. As already stated, eye tracking outputs can be used to give us an insight into peoples’ cognitive processes which they would otherwise be unable to communicate.

• Eye tracking outputs, specifically video outputs, can be used during interviews as an aid to the interviewee: “can you explain what you were doing then?” or “what was going through your mind during this action?” The prompt of seeing their actions again ‘through their own eyes’ can often trigger the interviewee to remember specific thoughts or feelings and allow a deeper insight into the actions at that time.

The main downside of eye tracking is the requirement of specialist equipment, and the need acquire a specific product for analysing the data depending on the eye tracker product. This is a substantial outlay, but they are also available for hire via short-term licenses. The actual use of the eye tracking equipment also has some downsides. Depending on the product used, some of the head mounted glasses-type eye trackers can obstruct peripheral vision as the frames can be quite thick. Some newer models are frameless for this reason. Also, the equipment requires calibration before use, and can easily de-calibrate if the user touches the equipment or moves suddenly.

• Another point for consideration is the analysis of data. Eye tracking produces large quantities of data that quite often requires substantial ‘cleaning’ before they can be analysed. This takes time, but when this is done the actual analysis is fairly straightforward and most of the software packages are user friendly.

 

 

How It Works

Eye tracking requires specialist software to both gather and analyse data. The good news is that there are now some extremely good quality, affordable options, which has opened up this area of analysis, which in turn has led to more research in the area and an increased understanding of outputs and metrics.

Typically there are two types of eye tracking device: head-mounted, or screen / panel mounted. 

Screen or panel mounted devices are typically used where the operator is in a fixed position and normally looking in one direction. These devices are used a lot for User Experience (UX) and web design applications to elicit where a user is looking in order to design webpages more effectively. These devices are also sometimes used in activities such as car driving, as the user is sat in one position and typically looking in one direction. 

Head mounted eye trackers are worn by the operator and are a lot more versatile as the operator can move freely. These head mounted trackers can resemble a pair of glasses, and have multiple cameras tracking the operator’s pupils and eye movements, and also filming what they are looking at. Both types of device can be used in lab-based or applied settings since they are relatively non-intrusive. 


Gathering data is fairly straightforward and depending on the device, can produce many different metrics. The most common eye tracking metrics are based on fixations and / or saccades. Fixations are discrete stable points of where the eye is looking for at least 300ms (and therefore visual information is processed by the brain), and saccades are defined as the eye movements between these fixations (where visual information is not processed). Typically, the first stage of analysis would be to separate the area the operator is working in or looking at into ‘areas of interest’. These then allow you to focus your analysis. Fixation metrics can include number of fixations, fixation durations, total number of fixations in one area of interest, fixation density, and repeat fixations. Fixation metrics can tell us a lot about the operator’s engagement with certain tasks or information. For example, if an operator is focussing on an area for a long time, it could indicate a higher cognitive effort. If an operator is fixating on many different points, it could indicate that the information the operator requires is scattered around, and we should aim to work out why, and if the presentation of information could be improved.

Saccade metrics can provide us with information regarding decision-making, but typically saccades and fixations are used in combination to provide scan-path data. Scan-paths can give us insight into how people go about looking for information, and allow us to analyse how information is gathered and used in certain situations. Scan-paths can be compared between operators, and between novice and expert users to understand the differences between approaches and the impacts these may have. We may be able to identify certain scan-paths that are always straight, and quick, indicating that this task is efficient. Conversely, we may be able to identify scan-paths which are erratic which may help us to understand any aspects of the task or information provision which isn’t currently working as well as it should. These metrics can provide us with some powerful insights into how work is carried out and can greatly assist with redesign of interfaces, displays, and tasks.

 

 

Illustrative Example

Aviation Domain:
As part of a larger project, we wanted to understand how pilots behaved in a complex situation containing multiple incidents. This was carried out in a full motion simulator, and a complex scenario was played out which resulted in the co-pilot flying the plane, a number of go-arounds, a runway change, and low fuel throughout the scenario. From the other data (cockpit data, and voice recordings) and through expert analysis, it was clear there were pilots who dealt with the scenario more successfully than others. By using eye tracking outputs, we were able to develop theories as to why some pilots performed better than others, and attach metrics to the data, such as dwell times on specific instruments, or scan-paths which were repeated time and time again. We were then able to triangulate this with the cockpit transcripts and understand why certain things were looked at certain times, or indeed, allow us to explore later during interview why they weren’t.This information helped to inform more advanced cockpit instrumentation designs, as well as informing emergency training requirements. 


As a specific example of the insights afforded by Eye Movement Tracking (EMT), it was possible to see how the Captain and First Officer were cross-checking each other during the escalating emergency scenario, and how they supported each other. Typically, the FO, whose main fixation was the PFD (Primary Flight Display)  was able to ‘offload’ a degree of the situation awareness to the Captain so that he could focus on diagnosing the problems affecting their flight. EMT was also able to clearly pinpoint the moment when each crew realised they were low on fuel, and how they used various cockpit instruments to analyse an electrical failure. In some areas instruments such as the PFD were fully supporting the flight crew, whereas in others there was clearly room for improvement.

Maritime Domain:
In this study, navigator’s eye movement on the navigation bridge simulator was analysed during passage through the Istanbul Strait. A measurement device called “EMR-8” was used for eye movement tracking. In the application, three examinee groups were selected and they divided according to their onboard experience level. Each group has 4 examinees in real-time bridge simulator. Group 1 consisted of deck department students who had up to 2.5 months sea experience onboard ship as a cadet. Group 2 included junior officers who had already completed 12 months training on-board ship. Group 3 consisted of oceangoing masters who had wide experiences at sea. 

The scenario about passing Istanbul Strait was given to each examinee. The visual field was divided into three parts; inside, outside and others. The inside part has three components; instruments, indicators and an engine telegraph. The outside part has three components; the sea condition, navigational equipment and target ships. The scenario was recorded by eye movement tracking. In the light of findings, Group1 (beginners), gave minimum attention to navigation euqipments, target ships, regulations and sailing routes. Group 2, the examinees had intermediate characteristics as a navigation officer between professionals and beginners. They had enough knowledge and ability to use navigational equipments, target ships, regulations and routes. They had a longer duration on the bridge compared to Group 1. On the other hand, Group 3 had high level experience and knowledge. The examinees gave utmost attention to navigational equipments, sailing routes, target ships and regulations. Group 3 could stand on the bridge for the longest time without losing concentration. 

 

 

FS #T15/B: NEUROID Neurophysiological Indicators

 

Background

NEUROID is a technique that employ signals gathered from the operator to characterize Human Factors like Mental Workload, Stress, Vigilance, and Engagement from a neurophysiological perspective. NEUROID aims at providing information linked to the insights of the operators while interacting with the surrounding environments (e.g. aircraft cockpit, ship commands) and executing operative tasks without interfering with them, and it allows to adapting the system itself depending on the operators’ psychophysical states. 

The NEUROID was developed by UNISAP and validated together with DBL and ENAC in several operational contexts like Automotive, Aviation, and ATM, to determine how the combination of the considered HFs can be used to define the Human Performance Envelope (HPE) of the operator, and consequently employed to monitor the operator while interacting with high-automated system and unexpected malfunctions, or under challenging and demanding situations.

image courtesy Stress project

 

 

Key Concepts

The NEUROID is based on data-mining and machine-learning algorithms to make the HFs neurophysiological models fitting each operator and minimize inter-user variability, therefore obtaining reliable operator’s assessment. More than 88% of all general aviation accidents are attributed to human error. Therefore, NEUROID generates an innovative and systematic approach to quantify and objectively measure HFs by taking into account, at the same time, the behaviours, emotions, and the mental reactions of the operators themselves, and integrating them with the data related to accidents and incidents investigations. 

The general concept at the base of NEUROID is that brain, body, and operator’s experience are reciprocally coupled, and that accurate assessment of the operator’s can only be achieved through a well-defined combination of all the available data. Each biological activity is regulated by the human Nervous System, therefore variations of such biological activities correspond to internal reactions because of modification of external (environment) and internal (mental, motivations, emotions, etc.) factors. There are numerous types of neurophysiological measures, such as Electroencephalogram (EEG, related to brain activity), Electrocardiogram (ECG, hearth activity), Electrooculogram (EOG, ocular activity), Galvanic Skin Response (GSR, skin sweating), and so on. Such neurophysiological measures can be seen as the physical interface of the NEUROID technique that will enable to gather insights about all the aspects relating to HFs of the operator like Mental Workload, Stress, Vigilance, and Engagement. The key concept of NEUROID is the Human Performance Envelope (HPE) taxonomy: instead of considering one or two single human factors, the HPE investigates a set of interdependent factors, working alone or in combination, which allows to completely characterize the operator. In other words, these concepts are proposed as performance shaping factors, which can differentially and interactively affect successful completion of a task. The HPE theory explicitly declares that boundaries exist where performance can degrade in line with the theoretical underpinnings for the considered HFs.

 

 

Benefits

Traditional methods to catch information about the operators’ HFs are usually based on self-reports and interviews. However, it has been widely demonstrated how such kind of measures could suffer of poor resolution due to a high intra- and inter-user variability depending on the nature of the measure itself (i.e. subjective). In addition, the main limitation in using subjective and behavioural measures alone is due to the impossibility of quantifying ‘‘unconscious’’ phenomena and feelings underlying human behaviours, and most importantly it is not possible to measure such unconscious reactions experienced by operators while performing working activities. In fact, the execution of a task is generally interrupted to collect subjective evaluations. 

On the contrary, neurophysiological measures exhibit an unobtrusive and implicit way to determine the operator’s affective and cognitive mental states on the basis of mind-body relations. Therefore, the benefit and advantage of the NEUROID technique is to use neurophysiological measures and machine-learning algorithms to overcome such limitations, especially to (1) objectively assess the operator’s mental states while dealing with operative tasks; (2) identify the most critical and complex conditions and correlated them with accident and incident investigations; and (3) create a closed-loop between the systems and operators (i.e. Joint Human Machine Cognitive system) to continuously and non-invasively monitor the operators themselves.

Finally, modern wearable technology can help in overcome the invasiveness (e.g. many cables, uncomfortable) of the equipment necessary to collect the neurophysiological signals from the operators.

 

 

 

How It Works

• NEUROID consists in 3 main phases as reported in the figure. 

The neurophysiological signals are gather from the operators by wearable technology while dealing with working tasks. 

Afterward, the neurophysiological data are processed through a series mathematical steps towards the machine-learning phase, in which each HF is classified. 

Finally, the considered HFs are combined to define the Operator’s HPE, and integrated with the operator’s behaviour, and data related to accidents and incidents (e.g. SHIELD database and HURID framework) for 

  a comprehensive operator’s assessment which can be employed for different applications, for example real-time mental states assessment, or triggering adaptive automations.

 

Illustrative Example

Aviation Domain
To see how the NEUROID technique can be implemented in practice we can consider the following scenario where an Air Traffic Controller (ATCO) is managing the air traffic and at the same time we are acquiring the ATCO’s EEG, ECG, and GSR neurophysiological signals. In the first phase of the scenario, the ATCO can rely on the support of high-automated systems to manage high traffic situation (High). Then, the traffic demand returns to a normal condition (Baseline), but at a certain moment the automations crash (Malfunction) and the ATCO will have to keep managing the traffic, and find out what is wrong with the automations.

In this context, we can use the NEUROID to measure how the ATCO’s HPE changes. In this regard, the neurophysiological data will be employed to estimate the ATCO’s Mental Workload, Stress, Attention, Vigilance, And Cognitive Control Behaviour based on the S-R-K model. Finally, those HFs will be combined to define the ATCO’s HPE on the three operational conditions as shown in the following figure. In particular, the values of the considered mental states have been normalized within the [0 ÷ 1] range: “0” means Low, while “1” mean High. Concerning the S-R-K aspect, “0” represent the S (Skill), “0.5” the R (Rule), and “1” the K (Knowledge) level.
 

 

The analysis of the HPE in the different scenario conditions exhibits interesting variations on the configuration of the mental states. In fact, the failure of the automations (red line) induced higher Vigilance, lower Attention, and shift from Skill to Rule level in the Cognitive Control Behaviour with respect to the HIGH (orange line) and BASELINE (blue line) conditions. Furthermore, considering the HPE changes over time throughout the ATM scenario allows to identify those moments, thus situations, corresponding to high (RICH) or low (POOR) performance. For example, for the considered ATCO, the HPE corresponding to RICH (green plot) and POOR (red plot) performance are reported in the following figure. Generally, in correspondence of RICH performance condition, the vigilance is higher and the stress is lower than during POOR performance. In conclusion, the NEUROID could be used, for example, to monitor the operators and warn them when they are working under low vigilance and high stress condition.

 

 

Maritime Domain
To see how the NEUROID technique can be implemented in Maritime practice we can consider the following scenario where the collision occurred between tanker ship and livestock ship during Dardanelle Strait passage at sea. The collision occurred during overtaking. The VTS (Vessel traffic services) was informed about the collision since it occurred during Dardanelle Strait passage. The weather was partly cloudy and sea state was calm at the time of collision. The time was early morning. The officer was keeping the navigational watch at the time of collision. It was early morning at the time of collision.

The HF analysis of the ship collision during Strait passage would highlight the following potential root causes: 

Mental Fatigue: Rather than physical fatigue, over alertness and overload (decision making, concentration, mental load). Watchkeeper Officers continuously working load can affect his/her decisions.

Physical Fatigue: Rather than mental fatigue, physical fatigue can be seen due to intensive work load on-board ship. 

Inadequate Manning on Bridge: According to sailing zone, increase the number of watchmen on bridge may help perception of the dangers earlier.

Inadequate Policy, Standards and Application: Increase of the inspections due to sailing rules and minimum CPA-TCPA values and activation of the existing rules may help avoiding dangerous situations and provide clearance from the plotted vessels.

Inadequate / Lack of Communication: Necessary manoeuvre could not be applied due to inadequate communications between vessels. Lack of communications between vessels and VTS also caused vessels to sail in close range. Every communication methods on board must be managed in most efficient manner.

Poor Risk Evaluation: Main reasons of the incident between vessels are the failure of the evaluation of risks for all vessels in the sailing area and unforeseen of the other vessel possible manoeuvres which were sailing in the outmost line of the separation. 

Inadequate Leadership and Supervision / Planning: Another root causes of the incident are improper planning of watch hours and not taking necessary measurements as a result of evaluations of inconveniences timely.

External/Environmental Conditions: Sailing in narrow zone, congestion and poorness of the VTS stations in planning the traffic are the main effective reasons of the incident. To prevent such like incidents, environmental conditions must be evaluated sufficiently and proper speed and watch order must be planned according to congestion.

Some of those aspects are mainly linked to the management level (e.g. manoeuvres procedures, risk evaluation), but the ones coming from the evaluation of the operators’ status like the mental and physical fatigue can be monitored and measured via the NEUROID during the working activities. 

In particular, along each phase of the considered scenario the neurophysiological measures can characterise the operators in terms of Mental Workload, Stress, Vigilance, and Engagement combination and those information can be consequently used to:

interact and\or intervene in real-time on the system, for example depending on the Master's mental workload it could be possible to enable a warning during overload or low vigilance conditions to regain the proper operational status;

post-doc analyse the incident\accident or simulation by combining the previous tasks analysis, and the operators' mental states profile along the scenario, for example to find out how the Mental Workload, Stress, Vigilance, and Engagement of the Master and C/O were right before the collision or during a specific phase which brought to the collision, and finally understand if the configuration of those mental states was not appropriate (e.g. distracted?) or, on the contrary, the operators were under too high demanding and stressing conditions due to the adopted procedure or situation.

 

 

FS #T19: Safety Culture

 

Background

The roots of Safety Culture are spread across several industries (oil and gas, nuclear power, space, air transport, rail, medical and maritime), being first mentioned officially in a report of the Chernobyl nuclear power accident in 1986. The International Atomic Energy Agency (IAEA) used safety culture to explain the organizational conditions that led to the violations of the front-line operators that created the path to disaster. A weak safety culture is seen as a strong contributory factor in various important accidents such as the King’s Cross underground station fire (Fennell, 1998), the Herald of Free Enterprise passenger ferry sinking (Sheen, 1987), the Clapham Junction passenger train crash (Hidden, 1989), the Dryden air crash (Maurino et al. 1995), the Uberlingen mid-air collision (2002), and – more controversially – the two recent Boeing 737-Max air crashes where poor safety culture appeared to apply more to the design, development, validation and oversight phases.

Even though there is no single definition of safety culture, various authors such as the International Nuclear Safety Agency Group (INSAG-4, 1991), Cox and Cox (1991), the UK Health and Safety Commission (HSC 1993) and Guldenmund (2000) agree on what safety culture represents. In broad terms, they all endorse the idea that safety culture embodies the practices, attitudes, beliefs, norms, perceptions and/or values that employees or groups of employees in a company share in relation to managing risks and overall safety. In addition, EUROCONTROL (2008) specified that safety culture can be described as “the way safety is done around here”, suggesting that it needs a practical approach.

 


Traditionally, safety culture is applied to operational organizations such as air traffic organizations (most in Europe undertake periodic safety culture surveys, certain airlines, and more recently, airports. However, the approach has been applied to one research and development centre where the focus was on designing new operational concepts for air traffic management (Gordon et al, 2007). While it is difficult to directly link equipment design and safety culture, and it is perhaps impossible for a designer to predict the safety culture that may evolve in the future when their design goes into operation, there are some salient points that designers need to consider:

Safety culture surveys sometimes do raise issues concerning design and the need for design improvement.
• Design of equipment, workplace or interfaces that are cumbersome, difficult to use or complex/confusing may lead human operators to perform ‘workarounds’ that are easier in practice, but may possibly be less safe in certain circumstances.
• Whenever design choices are being made, and there is a safety element, and especially when there is a potential safety-efficiency trade-off, the designer should favour safety. As one senior project manager put it, safety at any cost doesn’t make sense, but safety as a starting point does.
In stressful or emergency scenarios, where the operators have to make judgement calls between safety and productivity, if indications in the interface are not clear and integrated to help rapid decision-making that errs on the side of safety and caution, then a different outcome is likely to arise. Designers must ensure that, in the heat of the moment, the required safe action will be as clear as it can be.
• Designers must be clear as to which equipment and interface elements are safety critical.Moreover, the chief designer must have a holistic (rather than piecemeal or fragmented) view of the system and the safety critical role of the human operators, as well as a clear idea of the competence of those intended operators.
• The designer cannot ‘leave safety to the safety assessor’, or assume that oversight authorities can pick things up later. This is poor design culture, and, frankly, no one understands design intent better than the designer. The safety role of the human operator must be considered from the outset by the designer and be ‘built-in’.
• The designer cannot assume something is fit for purpose based on a few prototyping sessions with a few operational experts. Proper validation with a range of typical end users and abnormal as well as normal scenarios needs to occur.
• Designers should not underestimate the positive impact of design on safety culture, particularly where new and better equipment – user-friendly, making use of human aptitudes – enters a workplace which has hitherto been dogged by equipment problems.
• Designers should read accident and incident reports. This is the only way to look ahead, and to see how well-intended designs can end up in accidents.
• Designers should stay close to real operations and operators as much as possible, and their design organizations should encourage and enable such contact. This is the best way for designers to see around the corner, and to understand the user and their working context and culture as it evolves.

More generally, safety culture (or lack of it) can of course be felt by those working in design organizations. If for example, as a designer or engineer you think you are being pressurized to produce or accept a poorer design because of commercial pressures, who are you going to tell, and will they take you seriously and support your safety concern, or will they bow to the commercial pressure?

 

 

Key Concepts

Safety culture has a number of dimensions, such as the following (Kirwan et al, 2016):

Management commitment to safety (extent to which management prioritize safety);
Communication (extent to which safety-related issues are communicated to staff, and people ‘speak up for safety’);
Collaboration and involvement (group involvement and attitudes for safety management);
Learning culture (extent to which staff are able to learn about safety-related incidents);
• Just culture and reporting (extent to which respondents feel they are treated in a just and fair manner when reporting incidents);
Risk handling (how risk is handled in the organisation);
Colleague commitment to safety (beliefs about the reliability of colleagues’ safety behaviour);
Staff and equipment (extent to which the available staff and equipment are sufficient for the safe development of work);
Procedures and training (extent to which the available procedures and training are sufficient for the safe development of work).

Safety culture is typically evaluated using an online questionnaire of validated questions that is distributed across the organisation and filled in anonymously (usually taking about 10 minutes). Based on the responses, analysts then identify the safety culture ‘hotspots’, which are then discussed in confidential workshops, leading to recommendations for the organisation to improve its safety culture. At a macro level, typically a ‘spider diagram’ is produced based on the results, highlighting which safety culture dimensions need most attention. In the example figure to the right, ‘Colleague commitment to safety’ is doing well in safety culture terms, whereas ‘Staffing and Equipment’ requires attention.

 

 

 

Benefits

Safety culture surveys, particularly when anonymous and carried out by an independent organisation, are often seen as one of the only ways to find out all your risks and what people really think about the state of safety and safety culture in an organisation. They can pinpoint both current issues that can be corrected (so-called ‘quick wins’) without too much trouble, as well as deeper problems that may be entrenched in the organisational culture. Most safety culture surveys do identify the safety culture strengths of an organisation as well as vulnerabilities, so there is always some ‘good news’ as well as areas for improvement. Because the approach uses workshops of operational personnel, usually the recommendations that arise from surveys are useful and practicable, since they come from the organisation itself. 

The survey process is long, however, often taking 9 months from the first decision to have a survey, to having a report with recommendations. The management need to be committed to doing the process and realise that it is not a quick fix, and there will be work to do afterwards. If management are doing the survey just to gain ‘a tick in the box’, then it may be better not to do one at all.

 

 

How It Works

In many application of the aviation domain, there are two main phases used to evaluate the safety culture in a safety culture survey:

Phase 1: Questionnaire 
The safety culture survey uses an electronic version of the Safety Culture Questionnaire, over a period of typically 3 weeks. The confidential data are anonymised by The London School of Economics and then independently analysed by EUROCONTROL

Phase 2: (Optional) Workshops 
After an initial analysis of the questionnaire results, there is a visit to the organisation’s premises to hold a number of confidential workshops with front-line staff (without managers present) to clarify and explore further certain aspects emerging from the questionnaires. An example of what might be discussed in such a workshop is indicated by the figure below, which shows results for an organisation in the safety culture dimension ‘Staff and Equipment’. With respect to the questionnaire statement ‘We have the equipment needed to do our work safely’, whilst more than 70% were in agreement (the green bar), the rest were either unsure or in disagreement. The workshop would explore equipment issues or concerns people have, which are underlying this response.

 

 

Illustrative Example

Aviation domain
It should be noted that the example reported was seen as an exploratory survey and were focused on a single particular segment of the organization – Airbus Design rather than being company-wide (see Kirwan et al, 2019).

For the Airbus survey, the focus was on the Design part of the organization, which involves around 2,000 of the 70,000 employees. This survey required considerable tailoring of the questionnaire, as the job of design and systems engineering is somewhat different from airline operations, even if most underlying issues, except fatigue, and dimensions remain relevant. But the tailoring worked, with the same issues raised during workshops as via the questionnaires. Participation in the workshops by designers from all three of Airbus’ main design locations (France, UK, and Germany) was intense. The general principle espoused in each workshop was that a safety defect in a design of an aircraft could simply not be allowed to happen; therefore, any safety issues had to be resolved carefully and thoroughly before continuing, even if resolution efforts delayed progress toward production. One observation from the study team was that the managers in the management workshop were connected with safety and detailed safety concerns, more so than is often the case for middle managers. This is because many of the managers had held operational positions or positions ‘close to operations,’, so that they understood the operational pressures that could affect airlines and pilots. Another had been to an accident site after an air crash, and noted that after such an experience ‘safety never leaves you.’

The workshops focused a lot on internal communications between the different design departments, as well as the resolution of safety issues that could delay production, and the importance of middle manager support during such resolution periods. 

A number of constructive recommendations were made to Airbus in the final survey report.
 


Maritime domain
A safety culture assessment and an implementation framework were developed by the University of Strathclyde to enhance maritime safety. The focus was on the development of a safety culture assessment tool which covers all of the safety related aspects in a shipping company and measures will be taken proactively and reactively in order to enhance safety of the shipping industry.

Questionnaires were developed with an interdisciplinary group to ensure that they capture the fight information to conduct a comprehensive analysis. Each safety factor has specific questions which try to address the employee’s attitudes and perceptions amongst crew members and shore staff. There are a total of 85 questions which are asked to the employees, under the headings of communication, employer-employee trust, feedback, involvement, problem identification, promotion of safety, responsiveness, safety awareness and training and competence. 

The aim of the surveys is to analyse strengths and weaknesses in the company and perform benchmarking between crew and shore staff’s attitudes towards safety. Participants of the surveys were asked questions about demography, safety factors and open-ended questions about company policies. The safety culture assessment framework requires commitment from all of the bodies in a shipping company for successful implementation. 

Based on the findings from the questionnaires a set of recommendations were provided to the shipping company to improve communication, employer-employee trust, feedback, involvement, mutual trust, problem identification, responsiveness, safety awareness, training and competence, and the promotion of safety.

 

 

FS #M03/B: LOAT Levels of Automation Taxonomy

 

Background

The Levels of Automation Taxonomy (LOAT) classifies and compares different kinds of automation. It was originally developed in the context of the SESAR Programme by analysing 26 different automated functionalities for the Air Traffic Control and the flight crew. The taxonomy is grounded on the seminal works of Sheridan and Verplanck (1978). A new model was developed in order to overcome limitations encountered when applying the theory in practical situations. This formed the basis to develop a set of Automation Design Principles. The LOAT has 2 main purposes: 

provide potential design solutions with lower/higher level of automation; 

help classify automation examples and provide specific human centred suggestions.

 

 

 

Key Concepts

The LOAT brings into discussion important matters related to systems’ innovation.

Automation is not only a matter of either automating a task entirely or not but deciding on the extent to which it should be automated.

Introducing automation brings qualitative shifts in the way people practice rather than mere substitutions of pre-existing human tasks.

Classification of different levels of automation, recognises various ways in which human performance can be supported by automation.

The “optimal” level of automation in a specific task context is about matching the automation capabilities to a number of operational situations, while increasing the overall performance in efficient human-machine cooperation.

It provides potential alternatives for designing the human-machine interaction by taking full benefit of the technical solution.

 

 

 

Benefits

The LOAT can be used to:

Compare different design options and determine the optimal automation level in all operational contexts. The best level of automation depends on the capabilities of the automation itself and on the operational context in which it is included. You can use the LOAT to orient yourself in these choices.


Understand the impact of automation, both positive and negative, on the system. The highest level of automation is not always the best one. Finding the right level helps to make sure that you are taking the full benefit from it, without being negatively affected by side effects, such as nuisance alerts or erroneous directions.


Get insight on how to prevent human performance issues and take full benefit of available technical solutions in various domains. A common taxonomy for different domains will mean a shared approach to automation design. Developed in the Air Traffic Management domain but applicable to all domains in which automation plays an important role in sustaining human performance.


Classify existing implementations and derive lessons to improve automation design. Classifies the ‘nuances’ of automation enlightening the possibilities in design. Modifying existing tasks or introducing new ones involve the use of different psychomotor and cognitive functions, which implies the adoption of different automation solutions.

 

 

 

How It Works

The LOAT is organized as a matrix:

I. in horizontal direction: the 4 generic functions, derived from a four-stage model of the way humans process information:
1) Information acquisition: Acquisition and registration of multiple sources of information. Initial pre-processing of data prior to full perception and selective attention;
2) Information analysis: Conscious perception, manipulation of information in working memory. Cognitive operations including rehearsal, integration, inference;
3) Decision and action selection: Selection among decision alternatives, based on previous information analysis. Deciding on a particular (‘optimal’) option or strategy;
4) Action implementation: Implementation of a response or action consistent with the decision previously made. Carrying out the chosen option.


II. in vertical direction: each cognitive function groups several automation levels (between 5 and 8). All automation levels start with a default level ‘0’ - corresponding to manual task accomplishment - and increase to full automation.
 

The initial methodology that has been defined for the LOAT incorporates 4 steps: 1) Identify the automated tool to be classified; 2) Determine which function is supported (can be more than one); 3) Determine the relevant cluster of automation levels; 4) Identify the specific automation level and consider the design principles associated to it. The main LOAT goals are:

it is a method by which we can efficiently identify the most suitable human-automation design;
based on the classification of already developed automation examples we can provide specific and relevant human factor recommendations.

 

 

Illustrative Example

Aviation Domain
To see how the LOAT can be implemented in practice we should consider an example of human-machine interaction in industry.

SCENARIO: The Remote (or virtual) Tower uses an array of sensing and communication technologies on the airfield and the surrounding airspace, while providing the actual control functions (using certified controllers). In matter of automation the remote towers come with the following technology:

surveillance is generally provided by an array of video cameras, both conventional and infrared, to replace and improve upon the controller’s OTW view.

communications between controllers at the RTC and aircraft that use the airport are primarily via radio.

displays typically include an “out-the window” display of the airfield, data from radar (if available) and any other surveillance method. They may also include equipment indicating weather conditions (such as a wind rose), a compass rose, and geographical overlays.

controls would include microphones lighting controls, camera controls, etc.

other technologies include object tracking and alerting (in the air and on the ground) and features such as automatic runway intrusion alerts.

As in a conventional tower, all surveillance and communications information is transmitted to one or more controller positions in the Remote Tower Center (RTC).

ANALYSIS: Following the aforementioned steps, we can use the LOAT to identify the level of automation.

Step 1: Identify the automated tool to be classified. From the technologies of the RTC mentioned in the SCENARIO we choose the one we want to classify: e.g. Surveillance

Step 2: Determine which function is supported. We should then consider the main purposes of surveillance cameras: i.e. they collect images. Considering this, the function supported is: Information Acquisition (A).

Step 3: Determine the relevant cluster of automation levels. While the cameras are the ones collecting and generating the image to the user but only on the user’s demand, the relevant automation level is: supported by Automation. 

Step 4: Identify the specific automation level. The user is still in control to decide whether the information received from the cameras is relevant or not, and can filter the information received, so the automation level is: with user filtering and highlighting of relevant info (A2) 

In this way the LOAT can be used to decide upon the optimal level of automation suitable for the user and can help to establish whether or not it is a good choice to automate the system in the first place.

 
 

Would like to know more ?