Description Mental workload, also known as cognitive5 (work)load (or simply as driver workload in the driving context), is one of the most important variables in psychology, ergonomics, and human factors for understanding performance. This psychological state is, however, challenging to monitor continuously (131).
A commonly-used definition of mental workload is the one proposed by Hart and Staveland (76). They define mental workload as the cost incurred by a person to achieve a particular level of performance in the execution of a task. It is thus the portion of an individual’s mental capacity—necessarily limited—that is required by the demands of this task (23; 161), i.e., the ratio between the resources required to perform it and the available resources of the person doing it (188; 232).
In the literature on mental workload, one often finds references to another state called cognitive distraction. Mental workload and cognitive distraction are two different concepts, even if they can be linked when a driver performs secondary tasks while driving. Cognitive distraction increases the mental workload of a driver. An increase in mental workload is, however, not in itself an indication of cognitive distraction. First, mental workload can increase in the absence of distraction, e.g., when a driver is focusing to execute the primary task of driving correctly and safely. Second, mental workload can increase significantly with an increasing complexity of the driving environment (191). Cognitive distraction is further considered later as a particular category of (the state of) distraction.
Mental workload and stress are also linked since an increasing mental workload usually induces some stress in the driver.
Indicators In the driving context, visual tasks and mental tasks are closely linked. Indeed, while driving, a driver is constantly perceiving his/her driving environment and analyzing what he/she see in order to make the right decisions whenever required, e.g., scanning a crossroad and simultaneously judging the time and space relationships of other road users to decide when it is safe to cross an intersection. Therefore, it is logical that many researchers use eye-related parameters (e.g., blinks, fixations, and pupil diameter) to assess the mental workload of a driver (132).
Among the driver-based, physiological indicators, EDA is an indicator of mental workload (94). Similar to EDA, HR (61) and HRV (169) are, in automotive research, popular indicators of mental workload. HR increases as a task gets more difficult (182) or if other tasks are added (54). EEG is a valuable indicator for studying mental workload because it records the electrical activity of the brain itself, but it is complex to analyze (101). The pupil diameter is considered to be a valid indicator of mental workload (61; 105; 173). Indeed, Yokoyama et al. (241) indicate that the mental workload of a driver may be predicted from the slow fluctuations of the pupil diameter in daylight driving. These physiological parameters are, however, also influenced by other aspects of the mental and physical states of the driver (e.g., drowsiness and TR fatigue) and by environmental variables (e.g., illumination and temperature).
Among the driver-based, behavioral indicators, Fridman et al. (59) have shown that the visual scanning by a driver decreases with an increasing mental workload. Furthermore, since the interval of time between saccades has been shown to decrease as the task complexity increases, saccades may be a valuable indicator of mental workload (122; 140).
Subjective measures of mental workload exist, like the NASA task load index (NASA TLX) (76), which is a workload questionnaire for self-report, and the rating scale mental effort (RSME).
Driving performance can diminish as a result of an increase in mental workload. The vehicle-based indicators which are the most sensitive to such an increase are SDLP and SWM (191).
Palasek et al. (166) use the driving environment to estimate the attentional demand required from the driver to drive. The features extracted from the analysis of the driving environment are thus indicators of the mental workload of the driver.
The above information allows one to fill, in Table 4, the relevant cells of the “Mental workload” column.
Sensors Cameras are often used in the literature to characterize mental workload as they are particularly well suited to extract driver-based, behavioral indicators and are non-invasive.
Fridman et al. (59) describe a system for characterizing, non-invasively, via a camera facing the driver, what they call his/her cognitive load (CL). The system exploits the well-documented, experimental observation that the angular distribution of gaze direction (often characterized by the 2D pupil position) tends to become more concentrated, especially vertically, when the CL increases. Using video imagery, the system classifies the CL of the driver into one of the three CL levels (low, medium, high), as he/she engages in activities other than the primary task of driving, such as a conversation or the adjustment of the infotainment system. The system extracts, from a 90-frame, 6-second video clip, via computer vision, the face and the region of one eye of the driver. It then uses one of two methods: (1) mainly active appearance models (AAMs) for the face, eyelids, and pupil (when visible) to produce a sequence of pupil 2D positions, and (2) one hidden Markov model (HMM) for each of the three CL levels. The second one uses a single 3D CNN with three output classes corresponding to these levels. The two methods thus rely on a sequence of pupil positions and on a sequence of eye images, respectively. The output of the system is one of the three CL levels.
In order to develop this system, the authors first acquired training data in real-driving conditions while imposing on the driver a secondary task of a given CL level. This imposition of a given CL level while performing a primary task (here driving) is commonly achieved in the literature through the standard “n-back” task, where the three values of n, i.e., n = 0, 1, and 2, are viewed as corresponding to low, medium, and high CL. For the n-back task, a sequence of numbers is dictated to the subject, who is asked, for each number, whether it matches the one dictated n positions earlier in the sequence. For example, for n = 2, the subject must indicate whether the current number is the same as the one he/she heard 2 steps before, all this while he/she performs the primary task, here driving.
The authors indicate (1) that the differences in cognitive loading for the three levels have been validated using, among others, physiological measurements (e.g., HR, EDA, and pupil diameter), self-report ratings, and detection-response tasks, and (2) that these levels have been found to cover the usual range of secondary tasks while driving, such as manipulating a radio or a navigation system.
It is noteworthy that the data used for building the system was acquired through real driving, during which the driver repeatedly performed n-back tasks, while a camera was recording his/her face and surrounding area, this by contrast with the many other developments made using a driving simulator, in highly controlled conditions, and difficult to implement in real-life conditions.
The authors indicate that, while they use the term “cognitive load”, the literature often uses synonyms like “cognitive workload”, “driver workload”, and “workload”.
Musabini and Chetitah (152) describe another system that is also based on eye-gaze dispersion. They use a camera facing the driver, produce a heatmap representing the gaze activity, and train an SVM classifier to estimate the mental workload based on the features extracted from this representation.
Le et al. (110) characterize the mental workload based on the involuntary eye movements of the driver, resulting from head vibrations due to changing road conditions. They report that, as the mental workload increases, these involuntarily eye movements become abnormal, resulting in a mismatch between the actual eye movements measured via an eye-tracking device and the predicted eye movements resulting from a VOR+OKR model6 . For each driver, the VOR parameters are estimated during the first 10 sec of driving in condition of normal mental workload, whereas the parameter in the OKR model is fixed. The hypothesis of abnormal eye movements while driving under mental workload was validated using a t-test analysis. Different levels of mental workload were induced in a driving simulator using the n-back task.
Palasek et al. (166) use an external camera recording the driving environment to estimate the attentional demand using attentive-driving models. Indeed, the task of driving can sometimes require the processing of large amounts of visual information from the driving environment, resulting in an overload of the perceptual systems of a human being. Furthermore, traffic density is known to increase the mental workload (71), so that urban environments lead to a higher mental workload than rural and highway environments do (243), all other conditions being equal.
The above information allows one to fill the relevant cells of Table 5.