# US 2009/0261979 A1 · Driver Fatigue Monitoring System and Method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2009/0261979 A1
- Document ID: US20090261979A1
- Status: Published application
- Kind: A1
- Application number: 12/341,559
- Filed: 2008-12-22
- Published: 2009-10-22
- Earliest claimed: 1992-05-05
- Assignees: American Vehicular Sciences LLC
- Inventors: David S. Breed; Wendell C. Johnson; Wilbur E. DuVall
- Web page: https://patentgrep.com/patent/US20090261979A1

## CPC

- B60R 21/01538: {for image processing, e.g. cameras or sensor arrays}
- B60J 10/00: Sealing arrangements
- B60N 2/0022: {for sensing anthropometric parameters, e.g. heart rate or body temperature}
- B60N 2/0025: {by using weight measurement}
- B60N 2/0026: {for distinguishing between humans, animals or objects}
- B60N 2/0028: {of a body part, e.g. of an arm or a leg}
- B60N 2/0029: {for detecting the motion of the occupant}
- B60N 2/0035: {characterised by the sensor data transmission, e.g. wired connections or wireless transmitters therefor; characterised by the sensor data processing, e.g. seat sensor signal amplification or electric circuits for providing seat sensor information}
- B60N 2/02246: {Electric motors therefor}
- B60N 2/0248: {with memory of positions}
- B60N 2/0273: {taking into account user data, e.g. knee height or physical state}
- B60N 2/0276: {reaction to emergency situations, e.g. crash}
- B60N 2/2806: {for securing the child seat to the vehicle}
- B60N 2/2863: {backward facing}
- B60N 2/853: characterised by their adjusting mechanisms, e.g. electric motors
- B60N 2/888: with arrangements for protecting against abnormal g-forces, e.g. by displacement of the head-rest
- B60R 1/088: {using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors}
- B60R 11/0241: {for telephones}
- B60R 16/037: for occupant comfort {, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel}
- B60R 21/0132: responsive to vehicle motion parameters {, e.g. to vehicle longitudinal or transversal deceleration or speed value}
- B60R 21/01516: {using force or pressure sensing means}
- B60R 21/01534: {using electromagneticwaves, e.g. infrared}
- B60R 21/01536: {using ultrasonic waves}
- B60R 21/01542: {detecting passenger motion}
- B60R 21/01552: {detecting position of specific human body parts, e.g. face, eyes or hands}
- B60R 21/01554: {Seat position sensors}
- B60R 25/25: using biometry
- B60R 25/255: {Eye recognition}
- B60R 25/257: {Voice recognition}
- E05F 15/43: responsive to disruption of energy beams, e.g. light or sound
- E05F 15/431: {specially adapted for vehicle windows or roofs}
- G01F 23/20: by measurement of weight, e.g. to determine the level of stored liquefied gas
- G01F 23/2962: Measuring transit time of reflected waves
- G01F 23/804: {containing circuits handling parameters other than liquid level}
- G01S 7/417: {involving the use of neural networks}
- G01S 7/4802: {using analysis of echo signal for target characterisation; Target signature; Target cross-section}
- G01S 7/539: using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S 15/523: {for presence detection (burglar, theft or intruder alarms G08B13/00, e.g. G08B13/16)}
- G01S 15/87: Combinations of sonar systems
- G01S 15/88: Sonar systems specially adapted for specific applications (seismic or acoustic prospecting or detecting G01V1/00)
- G01S 17/88: Lidar systems specially adapted for specific applications
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- B60N 2/0268: {using sensors or detectors for adapting the seat or seat part, e.g. to the position of an occupant}
- B60N 2/0272: {using sensors or detectors for detecting the position of seat parts}
- B60N 2210/12: Capacitive; Electric field
- B60N 2210/16: Electromagnetic waves
- B60N 2210/18: Infrared
- B60N 2210/20: Radar
- B60N 2210/24: Cameras
- B60N 2210/26: Ultrasonic, e.g. sonar
- B60N 2210/42: Strain gauges
- B60N 2220/10: using a database
- B60N 2220/30: using a neural network
- B60N 2230/30: Signal processing of sensor data
- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- B60R 21/0134: responsive to imminent contact with an obstacle {, e.g. using radar systems}
- B60R 21/0136: responsive to actual contact with an obstacle {, e.g. to vehicle deformation, bumper displacement or bumper velocity relative to the vehicle (monitoring crash strength in combination with passenger seat detection B60R21/01558)}
- B60R 21/01544: {detecting seat belt parameters, e.g. length, tension or height-adjustment}
- B60R 21/01548: {sensing the amount of belt winded on retractor}
- B60R 2001/1223: {with sensors or transducers}
- B60R 2001/1253: {with cameras, video cameras or video screens}
- B60R 2011/0003: {inside the vehicle}
- B60R 2021/0027: {Post collision measures, e.g. notifying emergency services}
- B60R 2021/01315: {monitoring occupant displacement}
- E05F 2015/433: {using reflection from the obstruction}
- E05Y 2900/55: Windows
- G01S 13/04: Systems determining presence of a target (based on relative movement of target G01S13/56)
- G01S 15/04: Systems determining presence of a target
- G01S 15/06: Systems determining the position data of a target
- G01S 17/89: for mapping or imaging
- G10K 2210/1282: Automobiles
- G10K 2210/3219: Geometry of the configuration
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60J: WINDOWS, WINDSCREENS, NON-FIXED ROOFS, DOORS, OR SIMILAR DEVICES FOR VEHICLES; REMOVABLE EXTERNAL PROTECTIVE COVERINGS SPECIALLY ADAPTED FOR VEHICLES (fastening, suspending, closing, or opening of such devices E05)
- B60N: SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
- E05F: DEVICES FOR MOVING WINGS INTO OPEN OR CLOSED POSITION; CHECKS FOR WINGS; WING FITTINGS NOT OTHERWISE PROVIDED FOR, CONCERNED WITH THE FUNCTIONING OF THE WING
- G01F: MEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- E05Y: INDEXING SCHEME ASSOCIATED WITH SUBCLASSES E05D AND E05F, RELATING TO CONSTRUCTION ELEMENTS, ELECTRIC CONTROL, POWER SUPPLY, POWER SIGNAL OR TRANSMISSION, USER INTERFACES, MOUNTING OR COUPLING, DETAILS, ACCESSORIES, AUXILIARY OPERATIONS NOT OTHERWISE PROVIDED FOR, APPLICATION THEREOF
- G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR

## Abstract

Method and system for monitoring a vehicle driver includes an optical imaging system that obtains images of the driver, and a processor coupled to the optical imaging system and arranged to analyze the images obtained by the optical imaging system to locate a head of the driver in the images and monitor the driver's head or a part thereof over time. The processor also determines, based on the monitoring of the driver's head or part thereof, whether the driver has lost the ability to operate the vehicle. A reactive component is affected by the processor's determination that the driver has lost the ability to operate the vehicle, and requires action by the driver to indicate regaining of the ability to operate the vehicle or exerting control over the vehicle to slow the vehicle and bring it to a stop.

## Claims

### Claim 1

A vehicle including a system for monitoring a driver of the vehicle, comprising
a frame;
an optical imaging system arranged on said frame to obtain images of the driver;
a processor coupled to said optical imaging system and arranged to analyze the images obtained by said optical imaging system to locate a head of the driver in the images and monitor the driver's head or a part thereof over time, said processor being further arranged to determine, based on the monitoring of the driver's head or part thereof, whether the driver has lost the ability to operate the vehicle, the lost of ability to operate the vehicle arising from the driver falling asleep or otherwise being incapacitated; and
a reactive component arranged on the vehicle which is affected by the determination by said processor that the driver has lost the ability to operate the vehicle, said reactive component requiring action by the driver to indicate regaining of the ability to operate the vehicle or exerting control over the vehicle to slow the vehicle and bring it to a stop.

### Claim 2 (depends on 1)

The vehicle of claim 1, wherein said processor is arranged to use a trained pattern recognition technique to locate the head of the driver.

### Claim 3 (depends on 1)

The vehicle of claim 1, wherein said optical imaging system comprises a plurality of optical imagers spaced apart from one another and each positioned to obtain images of a face of the driver.

### Claim 4 (depends on 1)

The vehicle of claim 1, wherein said processor monitors the driver's head by determining a position of the head of the driver in images obtained at different times and analyzing the determined position of the head of the driver in the different images.

### Claim 5 (depends on 1)

The vehicle of claim 1, wherein said processor monitors the driver's eyes by determining a position of the eyes of the driver in images obtained at different times and analyzing the determined position of the eyes of the driver in the different images.

### Claim 6 (depends on 1)

The vehicle of claim 1, wherein said processor monitors the driver's eye lids by determining a position of the eye lids of the driver in images obtained at different times and analyzing the determined position of the eye lids of the driver in the different images.

### Claim 7 (depends on 1)

The vehicle of claim 1, wherein said reactive component comprises a warning light or a warning sound generating device.

### Claim 8 (depends on 1)

The vehicle of claim 1, wherein said reactive component comprises a button which, when pressed, indicates that the driver has regained the ability to operate the vehicle.

### Claim 9 (depends on 1)

The vehicle of claim 1, further comprising a horn and lights, said processor being arranged to operate said horn and lights to warn other vehicles of the inability of the driver to operate the vehicle.

### Claim 10

A method for monitoring operation of a vehicle by a driver, comprising:
obtaining images of the driver from at least one vehicle-mounted optical imager;
analyzing the obtained images to locate a head of the driver in the images;
monitoring the driver's head or a part thereof over time;
determining, based on the monitoring of the driver's head or part thereof, whether the driver has lost the ability to operate the vehicle, the lost of ability to operate the vehicle arising from the driver falling asleep or otherwise being incapacitated; and
upon a determination that the driver has lost the ability to operate the vehicle, affecting a reactive component which requires action by the driver to indicate regaining of the ability to operate the vehicle or exerts control over the vehicle to slow the vehicle and bring it to a stop.

### Claim 11 (depends on 10)

The method of claim 10, wherein the step of monitoring the driver's head or a part thereof over time comprises training a pattern recognition technique to locate the head of the driver.

### Claim 12 (depends on 10)

The method of claim 10, wherein the step of obtaining images of the driver comprises providing a plurality of optical imagers spaced apart from one another and positioning each of the optical imagers to obtain images of a face of the driver.

### Claim 13 (depends on 10)

The method of claim 10, wherein the step of monitoring the driver's head or a part thereof over time comprises determining a position of the head of the driver in images obtained at different times and analyzing the determined position of the head of the driver in the different images.

### Claim 14 (depends on 10)

The method of claim 10, wherein the step of monitoring the driver's head or a part thereof over time comprises monitoring the driver's eyes.

### Claim 15 (depends on 14)

The method of claim 14, wherein the step of monitoring the driver's eyes comprises determining a position of the eyes of the driver in images obtained at different times and analyzing the determined position of the eyes of the driver in the different images.

### Claim 16 (depends on 10)

The method of claim 10, wherein the step of monitoring the driver's head or a part thereof over time comprises monitoring the driver's eye lids.

### Claim 17 (depends on 16)

The method of claim 16, wherein the step of monitoring the driver's eyes comprises determining a position of the eye lids of the driver in images obtained at different times and analyzing the determined position of the eye lids of the driver in the different images.

### Claim 18 (depends on 10)

The method of claim 10, wherein the reactive component comprises a warning light or a warning sound generating device.

### Claim 19 (depends on 10)

The method of claim 10, wherein the reactive component comprises a button which, when pressed, indicates that the driver has regained the ability to operate the vehicle.

### Claim 20 (depends on 10)

The method of claim 10, further comprising operating a horn and lights to warn other vehicles of the inability of the driver to operate the vehicle.
