Study finds eye pupil as an indicator of effective decision making

reviewed by Emily Henderson, B.Sc.Apr 27, 2020

A team of military researchers and academics are investigating how changes in the size of the eye’s pupil can indicate a person’s cognitive status as a means of enabling teamwork with autonomous agents.

The Army’s future battle space will require humans and AI agents to effectively team up to achieve mission-critical objectives. Although AI agents can fill gaps in human performance, they are rigid and lack the flexibility inherent in human behavior, which could interfere with teamwork.

Human brains are amazing and adaptive systems that automatically apply the correct cognitive processes to complete a task and start each process at the correct time. However, the resources of our brain are limited. Being able to predict a soldier’s state of mind before resources run out is an opportunity for a self-employed agent to deploy capabilities to assist the soldier. To advance the enablement of this technology, we wanted to better understand how physiological cues, such as changes in pupil size, are related to performance and cognitive states. “

Dr. Russell Cohen Hoffing, Scientist, US Army Combat Capability Development Command Army Research Laboratory. USA

A joint effort between Army researchers and the University of California, Santa Barbara Institute for Collaborative Biotechnologies, the Cognitive Resilience and Sleep History project, or CRASH, seeks to understand how variations in state (measured by physiological sensors) influence performance. MORE ONE published the team’s research on a subset of the data set, “Dissociable Mappings of Tonic and Phase Pupillary Characteristics in Cognitive Processes Involved in Mental Arithmetic.”

In this research, the team sought to understand the cognitive processes that affect changes in pupil size, and the reliability of these relationships, as a basis for estimating how human cognitive processes and performance can vary in real-world tasks, Cognitively challenging, Cohen Hoffing said. The pupil is a unique data source, as it is the only internal organ of the body that the brain networks directly modulate and is visible to the outside world.

“The potential of this research is exciting because eye-tracking technologies are becoming universal in commercial and military contexts,” said Cohen Hoffing. “Inherent in eye tracking algorithms, pupil size is estimated but rarely used for analysis. Our research program aims to generate knowledge products that improve the usability of this type of data to gain a better understanding of cognitive processes such as attention and decision making. “

The researchers collected repeated measurements from the participants on eight separate occasions over four months. The data provided insight into the consistency of student response and relationships to behavior both within and between individuals, with a single view of cognitive processes over time rather than in single-session studies, Cohen said. Hoffing.

The study findings demonstrated that researchers can use pupil characteristics to index both time-varying and static aspects of cognition to understand how cognitive processes influence performance. The results indicated that, at the test level, the time each participant took to answer a mental calculation question was correlated with the time to maximum pupil dilation and pupil size. The relationship between student performance and rapid characteristics indicated that a latent student response was correlated with the process of arriving at an answer, while the student response increases correlated with the amount of attention applied to provide a response. In contrast, average student size was correlated with variability in how quickly participants completed questions, suggesting that average student size indicates that they are ready to complete the mental computation task.

The study results confirm and extend previous research, showing that cognition reliably influences the student in at least two time courses: a fast, transitory influence and a sustained influence of longer duration.

“These findings allow us to better understand where student data may be useful to the human and agent team,” said Dr. Steven Thurman, an Army scientist and lead author of the manuscript. “For example, it may be the case that pupil size is more reliable in complex real-world contexts only when data is averaged over the course of several seconds or minutes. Such a case would allow the ability to track time scale changes longer in mental states, such as vigilance, workload, or fatigue, but potentially limit their use for moment-to-moment decision tracking.It is important to use longitudinal studies like this to understand the usefulness of student data on these variable timescales. “

Future studies will examine how to apply this research in real-world contexts, such as the use of virtual reality to assess whether the characteristics of student size can be exploited in dynamic contexts. This will be a springboard to test the efficacy of adaptive autonomic agents that use pupil size as an effective measure of hidden human states.