Our survey of the field of DM and DMSs led us to the idea of synthesizing this field in terms of the three key components of states, indicators, and sensors. The next two subsections discuss the first two components, and the third subsection brings all three components into a system block diagram (BD).
Our survey convinced us that the (global) state of a driver should be characterized along at least the five dimensions—called here states—of drowsiness, mental workload, distraction, emotions, and under the influence.
One goal of a DMS is to determine the levels of one or more of these states in real time, nearly continuously, and, preferably, in a non-invasive way. We use “level” in a very general sense. The level can take several forms, such as a numerical value or a label. The numerical value can be on a continuous scale or on a discrete scale. A label can be the most likely (output) class of a classifier together with its probability, likelihood, or equivalent. A level can be binary, e.g., 0 and 1, or “alert” and “drowsy”. The levels of one or more of the five states can then be used to issue alerts or take safety actions; this is, however, not the object of this paper.
The first four states present a formidable challenge in that they are not defined in a precise way and cannot be measured directly, by contrast with, say, physical quantities such as voltage and power. The fifth state can be defined precisely, at least in the case of alcohol, but the measurement of its level requires asking the driver to blow in a breathalyzer and/or to submit to a blood test, both of which can be performed in real time nor non-invasively. In short, for all practical purposes, one cannot directly measure or obtain the level of any of the five states in any simple way. This is the reason for having recourse to “indicators” of each of these states.
While one may have an intuitive idea of what an indicator is, it is useful to define, as precisely as possible, what it is. In a nutshell, an indicator must be well defined, and there must be a clear procedure for computing its values (at a succession of time instants) based on input data provided by one or more sensors.
For the purpose of this paper, a “quantity” or “item” is called an indicator for a given (sub)state if it satisfies all of the following conditions:
As an illustration, the eye-blink rate (i.e., the blink rate of the left or right pair of eyelids) is scientifically recognized as being indicative of drowsiness. This parameter obeys all conditions above, and is thus an indicator of drowsiness.
Similarly to the level of a state, we talk about the value of an indicator. We use both “value” and “level” simply as a way to implicitly communicate wether one is talking about an indicator or a state. Ultimately, a set of values of the indicators of a state must be converted into a level of this state. The conversion may require the use of an advanced, validated algorithm.
Indicators are generally imperfect. In most cases, an indicator cannot be guaranteed to be fully correlated with a related state. Due to the presence of complex interrelationships between each state and its indicators, it is important to use as many indicators as possible to promote valid and reliable interpretation of the state of the driver. An example follows. The heart rate (HR) is known to be an indicator of drowsiness. But imagine that one relies solely on the HR to monitor drowsiness, and that the driver must suddenly brake to avoid an accident. Inevitably, this will cause his/her HR to undergo important variations. These particular variations have, however, no direct link with his/her level of drowsiness. Thus, while it is true that the HR is an indicator of drowsiness, one cannot rely on it alone to provide a reliable level of drowsiness. The environment, among other things, needs to be considered.
The values of indicators are obtained through algorithms applied to data collected via sensors.
Figure 2 shows a system BD that uses the terminology introduced above, i.e., sensors, indicators (and values thereof), and states (and levels thereof). The BD is drawn for a single, generic state, and one must specialize it for each of the five states of interest (or others).
The BD is self-explanatory. The input is the situation of interest (with the driver, vehicle, and environment). One or more sensors acquire data, typically signals and images. Algorithms extract the values of the indicators that are deemed relevant for the state of interest. Other algorithms convert these values into a level of the state. The three successive subsystems are labelled with the operation they perform, i.e., acquire, extract, and convert. The input and output of each subsystem should ideally be viewed as being functions of time.
If several states are used simultaneously, the value of a given indicator can be used to compute the level of any state that this indicator relates to.