Abstract/Summary

Control room environments are present throughout many areas of commercial and industrial infrastructure, from air traffic control and harbour masters to chemical plant control and military operations. These control rooms work using a human-in-the-loop system in which a human operator monitors the data from sensors in the environment they are responsible for and make decisions and take action to maintain efficiency and safety. Though humans possess a natural aptitude for spotting patterns and anomalies in complex data, the majority of safety and process control errors are still human errors. These errors are most often as a result of ‘cognitive overload’ – a state in which the operator is presented with more information than they can effectively process cognitively in real time. Machine learning is often employed to automate some of the tasks of the human operator to reduce their cognitive workload to reduce errors. The performance of machine learning systems relies on obtaining large volumes of labelled data; which is an expensive and time consuming process that is currently performed by experts manually reviewing complex data and providing labels. The work presented here focusses on automating the labelling process by assessing the cognitive state of the operator using objective measures and using these measures to detect when events occurred in complex data in order to provide labels. The research assesses pupil diameter, echocardiogram (ECG) and novel mouse movement metrics to determine their suitability as classifiers that can automatically detect when events occurred. Results demonstrate that pupil diameter performs better than ECG as a physiological classifier. When event types are analysed separately, results demonstrate that all measures developed correctly recall between 75% - 95% of longer event types whereas performance for shorter events types correctly recalls between 14% for mouse measures, 31% ECG measures and 78% for pupil measures.