# US 12,005,906 B2 · Using driver assistance to detect and address aberrant driver behavior > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,005,906 B2 - Document ID: US12005906B2 - Status: Granted - Kind: B2 - Application number: 17/065,722 - Filed: 2020-10-08 - Published: 2021-04-15 - Granted: 2024-06-11 - Earliest claimed: 2019-10-15 - Assignees: Waymo LLC - Applicants: Waymo LLC - Inventors: David Harrison Silver - Web page: https://patentgrep.com/patent/US12005906B2 ## CPC - B60W 40/09: Driving style or behaviour - G01C 21/3658: {Lane guidance} - B60Q 1/46: for giving flashing caution signals during drive, other than signalling change of direction, e.g. flashing the headlights {or hazard lights} - B60Q 1/525: {automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data} - B60Q 9/008: {for anti-collision purposes} - B60W 50/0205: {Diagnosing or detecting failures; Failure detection models} - B60W 50/0225: {Failure correction strategy} - B60W 50/06: Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - B60W 60/0015: {specially adapted for safety} - B60W 60/0051: {from occupants to vehicle} - G01C 21/30: Map- or contour-matching - G01C 21/3641: {Personalized guidance, e.g. limited guidance on previously travelled routes} - G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes} - B60W 30/18163: {Lane change; Overtaking manoeuvres} - B60W 2040/0818: {Inactivity or incapacity of driver} - B60W 2050/0029: {Mathematical model of the driver} - B60W 2050/0071: {Controller overrides driver automatically} - B60W 2050/0215: {Sensor drifts or sensor failures} - B60W 2520/10: Longitudinal speed - B60W 2520/105: Longitudinal acceleration - B60W 2540/30: Driving style - B60W 2552/05: Type of road, e.g. motorways, local streets, paved or unpaved roads - B60W 2552/53: Road markings, e.g. lane marker or crosswalk - B60W 2554/406: Traffic density - B60W 2555/60: Traffic rules, e.g. speed limits or right of way - B60W 2556/10: Historical data - B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data - B60W 2556/65: Data transmitted between vehicles - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract The technology relates to identifying and addressing aberrant driver behavior. Various driving operations may be evaluated over different time scales and driving distances. The system can detect driving errors and suboptimal maneuvering, which are evaluated by an onboard driver assistance system and compared against a model of expected driver behavior. The result of this comparison can be used to alert the driver or take immediate corrective driving action. It may also be used for real-time or offline training or sensor calibration purposes. The behavior model may be driver-specific, or may be a nominal driver model based on aggregated information from many drivers. These approaches can be employed with drivers of passenger vehicles, busses, cargo trucks and other vehicles. ## Claims ### Claim 1 A method comprising: performing, by one or more processors, state tracking of driving actions performed by a driver of a vehicle while the vehicle travels over a given distance; receiving, by the one or more processors, sensor information from a perception system of the vehicle, the sensor information including information associated with an external environment of the vehicle; maintaining, by the one or more processors, a driving log based on the state tracking of driving actions and the sensor information, the driving log indicating each time one of the driving actions performed by the driver falls outside of a threshold range and each time the driver fails to perform a driving action; determining, by the one or more processors based on the driving log, whether the driver's behavior is aberrant; and when the driver's behavior is determined to be aberrant, controlling, by the one or more processors, the vehicle to perform one or more actions while operating in a semi-autonomous driving mode. ### Claim 2 (depends on 1) The method of claim 1, wherein the determination of whether the driver's behavior is aberrant includes comparing the driving actions against a model of expected driver behavior, the model being an aggregated model based on information from multiple drivers other than the driver of the vehicle. ### Claim 3 (depends on 1) The method of claim 1, wherein the determination of whether the driver's behavior is aberrant includes comparing the driving actions against a model, the model being specific to the driver of the vehicle based on his or her past driving history. ### Claim 4 (depends on 1) The method of claim 1, wherein the determination of whether the driver's behavior is aberrant includes evaluating at least one of a number of sensors of the perception system, an accuracy of each sensor of the perception system, a sensor field of view, or an arrangement of sensors around the vehicle. ### Claim 5 (depends on 1) The method of claim 1, wherein the determination of whether the driver's behavior is aberrant includes evaluating at least one of lane departures, weaving within a lane, a speed profile, a braking profile, or a turning profile. ### Claim 6 (depends on 1) The method of claim 1, wherein the determination of whether the driver's behavior is aberrant includes comparing the state tracking of the driving actions with at least one of map information and the received sensor information. ### Claim 7 (depends on 1) The method of claim 1, wherein the one or more actions include providing haptic feedback to the driver in order to warn the driver that the driver's behavior is aberrant. ### Claim 8 (depends on 1) The method of claim 1, further comprising: performing, by the one or more processors, calibration of one or more sensors of the perception system based on the state tracking. ### Claim 9 (depends on 1) The method of claim 1, wherein the one or more actions include changing an orientation or a pattern of one or more headlights of the vehicle in order to alert other vehicles that the driver's behavior is aberrant. ### Claim 10 (depends on 1) The method of claim 1, wherein the external environment includes at least one of (i) a road condition or (ii) at least one other vehicle, and the road condition includes construction that resulted in a narrowing of one or more lanes or a loss of an adjacent shoulder. ### Claim 11 (depends on 1) The method of claim 1, wherein the driver's behavior is aberrant due to the driver being at least one of inebriated, drowsy or inattentive. ### Claim 12 (depends on 1) The method of claim 1, wherein the driver's behavior is aberrant due to the driver being inexperienced. ### Claim 13 (depends on 1) The method of claim 1, wherein the sensor information further includes information associated with at least one of an object or a condition within the vehicle. ### Claim 14 (depends on 13) The method of claim 13, wherein the sensor information indicates at least one of a position of the driver, whether the driver's hands are on a steering wheel of the vehicle, eye gaze direction associated with the driver, or one or more factors associated with equipment of the vehicle. ### Claim 15 (depends on 1) The method of claim 1, wherein the one or more actions include limiting or reducing a speed of the vehicle. ### Claim 16 A control system for a vehicle, the control system comprising: memory configured to store a behavior model; and one or more processors operatively coupled to the memory, the one or more processors being configured to: perform state tracking of driving actions performed by a driver of the vehicle while the vehicle travels over a given distance; receive sensor information from a perception system, the sensor information including information associated with an external environment of the vehicle; maintain a driving log based on the state tracking of driving actions and the sensor information, the driving log indicating each time one of the driving actions performed by the driver falls outside of a threshold range and each time the driver fails to perform a driving action; determine, based on the driving log, whether the driver's behavior is aberrant; and when the driver's behavior is determined to be aberrant, control the vehicle to perform one or more actions while operating in a semi-autonomous driving mode. ### Claim 17 (depends on 16) The control system of claim 16, wherein the determination of whether the driver's behavior is aberrant includes comparing the driving actions against a model of expected driver behavior, the model being an aggregated model based on information from multiple drivers other than the driver of the vehicle. ### Claim 18 (depends on 16) The control system of claim 16, wherein the determination of whether the driver's behavior is aberrant includes comparing the driving actions against a model of expected driver behavior, the model being specific to the driver of the vehicle based on his or her past driving history. ### Claim 19 (depends on 16) The control system of claim 16, wherein the determination of whether the driver's behavior is aberrant includes an evaluation of at least one of a number of sensors of the perception system, an accuracy of each sensor of the perception system, a sensor field of view, or an arrangement of sensors around the vehicle. ### Claim 20 (depends on 16) The control system of claim 16, wherein the determination of whether the driver's behavior is aberrant includes an evaluation of at least one of lane departures, weaving within a lane, a speed profile, a braking profile, or a turning profile. ### Claim 21 (depends on 16) The control system of claim 16, wherein the determination of whether the driver's behavior is aberrant includes a comparison of the state tracking of the driving actions with at least one of map information and the received sensor information. ### Claim 22 (depends on 16) The control system of claim 16, wherein the one or more actions include providing haptic feedback to the driver in order to warn the driver that the driver's behavior is aberrant. ### Claim 23 (depends on 16) The control system of claim 16, wherein the one or more processors are further configured to perform calibration of one or more sensors of the perception system based on the state tracking. ### Claim 24 (depends on 16) The control system of claim 16, wherein the one or more actions include changing an orientation or a pattern of one or more headlights of the vehicle in order to alert other vehicles that the driver's behavior is aberrant. ### Claim 25 (depends on 16) A vehicle configured to evaluate aberrant driving actions of a driver while operating in the semi-autonomous driving mode, the vehicle including the control system of claim 16, and a driving system and a perception system operatively coupled to the control system.