# US 2021/0276568 A1 · Attention-based notifications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0276568 A1
- Document ID: US20210276568A1
- Status: Published application
- Kind: A1
- Application number: 16/810,760
- Filed: 2020-03-05
- Published: 2021-09-09
- Earliest claimed: 2020-03-05
- Assignees: Harman International Industries, Incorporated
- Applicants: Harman International Industries, Incorporated
- Inventors: Joseph VERBEKE; Mingwen DONG; Sven KRATZ; Adam BOULANGER; Stefan MARTI; Evgeny BURMISTROV
- Web page: https://patentgrep.com/patent/US20210276568A1

## CPC

- B60W 40/08: related to drivers or passengers
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/65: Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- B60R 21/01552: {detecting position of specific human body parts, e.g. face, eyes or hands}
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06V 40/19: Sensors therefor
- B60K 2360/149: Instrument input by detecting viewing direction not otherwise provided for
- B60K 2360/21: using cameras
- B60K 2360/741: Instruments adapted for user detection
- B60W 2040/0818: {Inactivity or incapacity of driver}
- B60W 2040/0872: {Driver physiology}
- B60W 2050/146: {Display means}
- B60W 2540/221: Physiology, e.g. weight, heartbeat, health or special needs
- B60W 2540/225: Direction of gaze
- B60W 2540/229: Attention level, e.g. attentive to driving, reading or sleeping
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system may present notifications to an occupant of a vehicle and determine whether the occupant has given attention to the notification. The system oscillates a visual notification at an output frequency. The system captures images of an eye of the occupant to determine an eye pupil oscillation frequency associated with the occupant. The system compares the eye pupil oscillation frequency to the output frequency. The system determines whether attention was given to the notification based on whether the output frequency and the eye pupil oscillation frequency match. If the system determines that attention was not given, the system may cause performance of one or more follow-up operations.

## Claims

### Claim 1

A computer-implemented method, comprising:
causing a first visual stimulus to be oscillated at a first frequency;
determining a second frequency associated with a user characteristic; and
based on a determination that the second frequency does not match the first frequency, causing one or more operations to be performed.

### Claim 2 (depends on 1)

The method of claim 1, wherein causing the first visual stimulus to be oscillated at the first frequency comprises oscillating a brightness amplitude of the first visual stimulus at the first frequency.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first visual stimulus comprises an indicator light.

### Claim 4 (depends on 3)

The method of claim 3, wherein causing the first visual stimulus to be oscillated at the first frequency comprises blinking the indicator light at the first frequency.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first visual stimulus comprises a notification displayed via a display device.

### Claim 6 (depends on 1)

The method of claim 1, wherein the user characteristic comprises an eye pupil diameter of a user.

### Claim 7 (depends on 6)

The method of claim 6, wherein the second frequency corresponds to an oscillation of the eye pupil diameter.

### Claim 8 (depends on 6)

The method of claim 6, wherein determining the second frequency comprises capturing images of an eye of the user via an imaging device to determine the eye pupil diameter.

### Claim 9 (depends on 1)

The method of claim 1, wherein the user characteristic comprises an electroencephalograph (EEG) of a user or a magnetoencephalograph (MEG) of the user.

### Claim 10 (depends on 1)

The method of claim 1, further comprising determining, based on the determination that the second frequency does not match the first frequency, a lack of attention on the first visual stimulus by a user.

### Claim 11 (depends on 10)

The method of claim 10, wherein causing one or more operations to be performed comprises, in response to determining the lack of attention on the first visual stimulus by the user, causing a second visual stimulus associated with the first visual stimulus to be output.

### Claim 12 (depends on 11)

The method of claim 11, wherein the second visual stimulus is an enlarged version of the first visual stimulus.

### Claim 13 (depends on 10)

The method of claim 10, wherein causing one or more operations to be performed comprises, in response to determining the lack of attention on the first visual stimulus by the user, causing an audio component associated with the first visual stimulus to be output.

### Claim 14 (depends on 10)

The method of claim 10, wherein causing one or more operations to be performed comprises, in response to determining the lack of attention on the first visual stimulus by the user, intervening in operation of a first sub-system.

### Claim 15

One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
causing a first visual stimulus to be oscillated at a first frequency;
determining a second frequency associated with a user characteristic; and
based on a determination that the second frequency does not match the first frequency, determining a lack of attention on the first visual stimulus by a user.

### Claim 16 (depends on 15)

The one or more computer-readable storage media of claim 15, wherein causing the first visual stimulus to be oscillated at the first frequency comprises at least one of: oscillating a brightness amplitude of the first visual stimulus at the first frequency, or blinking the first visual stimulus at the first frequency.

### Claim 17 (depends on 15)

The one or more computer-readable storage media of claim 15, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of:
in response to determining the lack of attention on the first visual stimulus by the user, at least one of:
causing a second visual stimulus associated with the first visual stimulus to be output,
causing an audio component associated with the first visual stimulus to be output, or
intervening in operation of a first sub-system.

### Claim 18

A system, comprising:
a memory storing an application; and
a processor that, when executing the application, is configured to:
cause a first visual stimulus to be oscillated at a first frequency;
determine a second frequency associated with a pupil of an eye;
based on a determination that the second frequency does not match the first frequency, determine a lack of attention on the first visual stimulus; and
in response to determining the lack of attention on the first visual stimulus, cause one or more operations to be performed.

### Claim 19 (depends on 18)

The system of claim 18, wherein causing one or more operations to be performed comprises causing a second visual stimulus associated with the first visual stimulus to be output at a third frequency.

### Claim 20 (depends on 19)

The system of claim 19, wherein the processor, when executing the application, is further configured to:
determine a fourth frequency associated with the pupil of the eye; and
based on a determination that the fourth frequency matches the third frequency, determine a presence of attention on the second visual stimulus.
