# US 2025/0145162 A1 · Driver abnormality sign detection device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0145162 A1 - Document ID: US20250145162A1 - Status: Published application - Kind: A1 - Application number: 18/940,850 - Filed: 2024-11-08 - Published: 2025-05-08 - Earliest claimed: 2023-11-08 - Assignees: Mazda Motor Corporation - Applicants: Mazda Motor Corporation - Inventors: Junichiro KUWAHARA; Dongmei HUANG; Ryoma SASAKI - Web page: https://patentgrep.com/patent/US20250145162A1 ## CPC - B60W 40/09: Driving style or behaviour - B60W 40/08: related to drivers or passengers - B60W 50/0097: {Predicting future conditions} - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - G06V 40/18: Eye characteristics, e.g. of the iris - B60W 2040/0818: {Inactivity or incapacity of driver} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2540/10: Accelerator pedal position - B60W 2540/106: Rate of change - B60W 2540/12: Brake pedal position - B60W 2540/18: Steering angle - B60W 2540/225: Direction of gaze - B60W 2540/229: Attention level, e.g. attentive to driving, reading or sleeping - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A driver abnormality detection device is provided that detects an abnormality sign of a driver at a sufficiently earlier stage before driving actually becomes difficult. The detection device includes a control circuit configured to detect an abnormality sign of a driver based on visual line information from a visual line detector, driving operation information from a driving operation detector, and traveling environment information from a traveling environment information detector. The control circuit is configured to calculate a visual line abnormality degree, a driving operation abnormality degree, and a traveling risk of a vehicle. The control circuit is configured to combine the visual line abnormality degree, the driving operation abnormality degree, and the traveling risk to generate a comprehensive abnormality degree, and detect the abnormality sign of the driver when the comprehensive abnormality degree is a predetermined threshold value or greater. ## Claims ### Claim 1 A driver abnormality sign detection device detecting an abnormality sign of a driver driving a vehicle, the driver abnormality sign detection device comprising: a control circuit configured to detect the abnormality sign of the driver based on visual line information acquired from a visual line detector, driving operation information acquired from a driving operation detector, and traveling environment information acquired from a traveling environment detector, wherein the control circuit is configured to acquire a predicted visual line based on the traveling environment information, acquire an actual visual line based on the visual line information, calculate a visual line abnormality degree which represents an extent that the actual visual line differs from the predicted visual line, acquire a predicted driving operation based on the traveling environment information, acquire an actual driving operation based on the driving operation information, calculate a driving operation abnormality degree which represents an extent that the actual driving operation differs from the predicted driving operation, calculate a traveling risk of the vehicle based on the traveling environment information, calculate a comprehensive abnormality degree by combining the visual line abnormality degree, the driving operation abnormality degree, and the traveling risk, and on condition that the comprehensive abnormality degree is equal to or greater than a predetermined threshold value, detect the abnormality sign of the driver on condition. ### Claim 2 (depends on 1) The driver abnormality sign detection device according to claim 1, wherein the control circuit is configured to: determine a maximum visual line abnormality degree and a maximum driving operation abnormality degree within a predetermined time; and calculate a comprehensive abnormality degree by combining the maximum visual line abnormality degree, the maximum driving operation abnormality degree, and the traveling risk. ### Claim 3 (depends on 2) The driver abnormality sign detection device according to claim 2, wherein the control circuit is configured to: calculate an average traveling risk in the predetermined time; and calculate a comprehensive abnormality degree by combining the maximum visual line abnormality degree, the maximum driving operation abnormality degree, and the average traveling risk. ### Claim 4 (depends on 3) The driver abnormality sign detection device according to claim 3, wherein, on condition that the comprehensive abnormality degree is equal to or greater than the predetermined threshold value and the traveling risk is increasing, the control circuit is configured to detect the abnormality sign of the driver. ### Claim 5 (depends on 4) The driver abnormality sign detection device according to claim 4, wherein the control circuit is configured to determine that the traveling risk is increasing based on an average of the traveling risk in the predetermined time. ### Claim 6 (depends on 3) The driver abnormality sign detection device according to claim 3, wherein the control circuit is configured to: acquire a predicted saccade of the driver based on the traveling environment information, acquire an actual saccade of the driver based on the visual line information, and calculate, as the visual line abnormality degree, a difference between the actual saccade and the predicted saccade. ### Claim 7 (depends on 3) The driver abnormality sign detection device according to claim 3, wherein the control circuit is configured to: acquire prediction operation amounts of a steering wheel, an accelerator pedal, and/or a brake pedal based on the traveling environment information, acquire actual operation amounts based on the driving operation information, and calculate, as the driving operation abnormality degree, a difference between the actual operation amounts and the predicted operation amounts. ### Claim 8 (depends on 1) The driver abnormality sign detection device according to claim 1, wherein the control circuit is configured to calculate the visual line abnormality degree, the driving operation abnormality degree, the traveling risk, and the comprehensive abnormality degree in a predetermined cycle. ### Claim 9 (depends on 1) The driver abnormality sign detection device according to claim 1, wherein the control circuit is configured to calculate the visual line abnormality degree, the driving operation abnormality degree, and the traveling risk, in parallel. ### Claim 10 (depends on 1) The driver abnormality sign detection device according to claim 1, wherein, on condition that the comprehensive abnormality degree is equal to or greater than the predetermined threshold value and the traveling risk is increasing, the control circuit is configured to detect the abnormality sign of the driver. ### Claim 11 (depends on 10) The driver abnormality sign detection device according to claim 10, wherein the control circuit is configured to determine that the traveling risk is increasing based on an average of the traveling risk in a predetermined time. ### Claim 12 (depends on 1) The driver abnormality sign detection device according to claim 1, wherein the control circuit is configured to: acquire a predicted saccade of the driver based on the traveling environment information, acquire an actual saccade of the driver based on the visual line information, and calculate, as the visual line abnormality degree, a difference between the actual saccade and the predicted saccade. ### Claim 13 (depends on 1) The driver abnormality sign detection device according to claim 1, wherein the control circuit is configured to: acquire prediction operation amounts of a steering wheel, an accelerator pedal, and/or a brake pedal based on the traveling environment information, acquire actual operation amounts based on the driving operation information, and calculate, as the driving operation abnormality degree, a difference between the actual operation amounts and the predicted operation amounts. ### Claim 14 A driver abnormality sign detection device detecting an abnormality sign of a driver driving a vehicle, the driver abnormality sign detection device comprising: a control circuit configured to detect the abnormality sign of the driver based on visual line information acquired from a visual line detector, driving operation information acquired from a driving operation detector, and traveling environment information acquired from a traveling environment detector, wherein the control circuit is configured to run a visual line model, using driving operation and traveling environment as inputs, that outputs a predicted visual line of the driver in a healthy state; run a driving operation prediction model, using the traveling environment as an input, that outputs a predicted driving operation value of the driver in the healthy state; acquire an actual visual line based on the visual line information, calculate a visual line abnormality degree which represents a difference between the actual visual line and the prediction visual line; acquire an actual driving operation based on the driving operation information, calculate a driving operation abnormality degree that represents a difference between the actual driving operation and the prediction driving operation; calculate a traveling risk of the vehicle based on the traveling environment information; calculate a comprehensive abnormality degree by combining the visual line abnormality degree, the driving operation abnormality degree, and the traveling risk; and on condition that the comprehensive abnormality degree is a predetermined threshold value or greater, detect the abnormality sign of the driver. ### Claim 15 (depends on 14) The driver abnormality sign detection device according to claim 14, wherein the control circuit is configured to: determine a maximum visual line abnormality degree and a maximum driving operation abnormality degree within a predetermined time; and calculate a comprehensive abnormality degree by combining the maximum visual line abnormality degree, the maximum driving operation abnormality degree, and the traveling risk. ### Claim 16 (depends on 14) The driver abnormality sign detection device according to claim 14, wherein the control circuit is configured to: calculate an average traveling risk in a predetermined time; and calculate a comprehensive abnormality degree by combining the maximum visual line abnormality degree, the maximum driving operation abnormality degree, and the average traveling risk. ### Claim 17 (depends on 14) The driver abnormality sign detection device according to claim 14, wherein, on condition that the comprehensive abnormality degree is equal to or greater than the predetermined threshold value and the traveling risk is increasing, the control circuit is configured to detect the abnormality sign of the driver. ### Claim 18 A non-transitory computer readable storage device having computer readable instructions that when executed by circuitry cause the circuitry to: acquire a predicted visual line based on traveling environment information of a vehicle, acquire an actual visual line of a driver of the vehicle based on visual line information, calculate a visual line abnormality degree based on a difference between the actual visual line and the predicted visual line, acquire a predicted driving operation based on the traveling environment information, acquire an actual driving operation based on driving operation information, calculate a driving operation abnormality degree based on a difference between the actual driving operation differs from the predicted driving operation, calculate a traveling risk of based on the traveling environment information, calculate a comprehensive abnormality degree by combining the visual line abnormality degree, the driving operation abnormality degree, and the traveling risk, and on condition that the comprehensive abnormality degree is equal to or greater than a predetermined threshold value, detect an abnormality sign of the driver on condition.