# US 2025/0153718 A1 · Driving assistance apparatus, driving assistance system, and driving assistance method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0153718 A1
- Document ID: US20250153718A1
- Status: Published application
- Kind: A1
- Application number: 19/025,480
- Filed: 2025-01-16
- Published: 2025-05-15
- Earliest claimed: 2022-10-13
- Assignees: Panasonic Automotive Systems Co., Ltd.
- Applicants: Panasonic Automotive Systems Co., Ltd.
- Inventors: Takaaki SEKI; Koichi EMURA; Masataka KATO
- Web page: https://patentgrep.com/patent/US20250153718A1

## CPC

- B60W 40/08: related to drivers or passengers
- B60K 35/00: Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K 35/23: Head-up displays [HUD] (optical aspects of head-up displays G02B27/01)
- B60W 10/06: including control of combustion engines
- B60W 10/18: including control of braking systems
- B60W 50/0097: {Predicting future conditions}
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08)
- B60W 2040/0818: {Inactivity or incapacity of driver}
- B60W 2050/146: {Display means}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2540/225: Direction of gaze
- B60W 2540/229: Attention level, e.g. attentive to driving, reading or sleeping
- B60W 2710/06: Combustion engines, Gas turbines
- B60W 2710/18: Braking system
- B60W 2720/106: Longitudinal acceleration
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A driving assistance apparatus according to the present disclosure includes a memory and a processor. The processor is coupled to the memory, and configured to: calculate a prediction error that is a difference between a predicted image and an actual image, the predicted image being predicted from an image in a traveling direction of a vehicle captured by a vehicle exterior camera that captures a periphery of the vehicle, an actual situation being captured in the actual image; estimate an attention state of a driver, based on the prediction error; and output driving assistance information for prompting an attention state and/or a behavior change and/or a consciousness change in relation to a driving manipulation, which are more appropriate for driving at that time, based on the attention state.

## Claims

### Claim 1

A driving assistance apparatus comprising:
a memory; and
a processor coupled to the memory, and configured to:
calculate a prediction error that is a difference between a predicted image and an actual image, the predicted image being predicted from an image in a traveling direction of a vehicle captured by a vehicle exterior camera that captures a periphery of the vehicle, an actual situation being captured in the actual image;
estimate an attention state of a driver, based on the prediction error; and
output driving assistance information for prompting an attention state and/or a behavior change and/or a consciousness change in relation to a driving manipulation, which are more appropriate for driving at that time, based on the attention state.

### Claim 2 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to extract an area in which the prediction error is larger than a predetermined threshold among a plurality of areas in the image, and estimates one or more states in which the extracted area is biased from a center of the plurality of areas, excessively concentrated on a part, and/or excessively dispersed throughout.

### Claim 3 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to extract an area in which a temporal variance value of the prediction error is larger than a predetermined threshold among a plurality of areas of a temporally continuous image group, and estimates one or more states in which the extracted area is biased from a center of the plurality of areas, excessively concentrated on a part, and/or excessively dispersed throughout.

### Claim 4 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to output driving assistance information for unconsciously changing visual behavior to correct a bias when there is the bias in the estimated attention state, to disperse an excessive concentration when there is the excessive concentration on a part, and to concentrate an excessive dispersion when there is the excessive dispersion throughout.

### Claim 5 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to outputs driving assistance information for calling attention to correct a bias when there is the bias in the estimated attention state, to disperse an excessive concentration when there is the excessive concentration on a part, and to concentrate an excessive dispersion when there is the excessive dispersion throughout.

### Claim 6 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is further configured to estimate, from a face image of the driver captured by a driver monitoring camera that captures an image of an interior of the vehicle, an area to which a gaze of the driver is directed in the prediction error, and
the processor is configured to estimate the attention state of the driver based on a direction of the gaze in addition to the prediction error.

### Claim 7 (depends on 6)

The driving assistance apparatus according to claim 6, wherein
the processor is configured to estimate likelihood of attracting attention based on a prediction error of an area to which the gaze of the driver is directed and a prediction error of an arbitrary area to which the gaze is not directed.

### Claim 8 (depends on 7)

The driving assistance apparatus according to claim 7, wherein
the processor is configured to output driving assistance information for unconsciously changing visual behavior to induce an appropriate attention state when the likelihood of attracting the attention is high in the estimated attention state.

### Claim 9 (depends on 7)

The driving assistance apparatus according to claim 7, wherein
the processor is configured to output driving assistance information for calling an attention to make the driver conscious when the likelihood of attracting the attention is high in the estimated attention state.

### Claim 10 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is further configured to determine a skill level of driving by the driver based on manipulation information of the vehicle by the driver obtained from a detection apparatus that detects a state of each part of the vehicle, and
the processor is configured to control output based on the skill level in addition to the prediction error.

### Claim 11 (depends on 10)

The driving assistance apparatus according to claim 10, wherein
the processor is configured to output, in a more conspicuous mode in a case where the skill level is lower, driving assistance information for unconsciously changing visual behavior to correct a bias when there is the bias in the estimated attention state, to disperse an excessive concentration when there is the excessive concentration on a part, and to concentrate an excessive dispersion when there is the excessive dispersion throughout.

### Claim 12 (depends on 10)

The driving assistance apparatus according to claim 10, wherein
the processor is configured to output, in a more conspicuous mode in a case where the skill level is lower, driving assistance information for calling attention to correct a bias when there is the bias in the estimated attention state, to disperse an excessive concentration when there is the excessive concentration on a part, and to concentrate an excessive dispersion when there is the excessive dispersion throughout.

### Claim 13 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to output the driving assistance information to an information presentation control apparatus controlling an information presentation apparatus that presents information to the driver,
the information presentation apparatus includes a head-up display, and an in-vehicle monitor, a meter display, a pillar display, a mirror display, a light emitting device, and a speaker which are mounted in the vehicle, and
the information presentation control apparatus causes at least any of the head-up display, the in-vehicle monitor, the meter display, the pillar display, the mirror display, the light emitting device, and the speaker to present the information.

### Claim 14 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to output the driving assistance information to an information presentation control apparatus controlling an information presentation apparatus that presents information to the driver,
the information presentation apparatus includes a head-up display, and an in-vehicle monitor, a meter display, a pillar display, a mirror display, a light emitting device, and a speaker which are mounted in the vehicle, and
the information presentation control apparatus causes two or more information presentation apparatuses among the head-up display, the in-vehicle monitor, the meter display, the pillar display, the mirror display, the light emitting device, and the speaker to present the information divided into a plurality of portions, respectively.

### Claim 15 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to output, to an on-vehicle electronic controller that controls the vehicle, operation information that is associated with an area to which attention of the driver is to be called and gives an instruction on an operation of the vehicle; and
the on-vehicle electronic controller is configured to cause a vehicle control apparatus that decelerates the vehicle to perform the operation based on the operation information.

### Claim 16 (depends on 15)

The driving assistance apparatus according to claim 15, wherein
the vehicle control apparatus includes a braking device that brakes the vehicle and an engine control apparatus that performs control of an engine output of the vehicle, and
the on-vehicle electronic controller is configured to decelerate the vehicle by performing at least any of
braking the vehicle by the braking device, and
reducing acceleration of the vehicle by the engine control apparatus.

### Claim 17 (depends on 15)

The driving assistance apparatus according to claim 15, wherein
the processor is configured to output the operation information having a higher deceleration effect of the vehicle as the prediction error on which the area to which the attention of the driver is to be called is based is larger.

### Claim 18 (depends on 1)

The driving assistance apparatus according to claim 1, wherein
the processor is configured to output information of the attention state and vehicle manipulation information acquired from a vehicle network to an external server that accumulates the information of the attention state.

### Claim 19

A driving assistance system comprising:
a memory; and
a processor coupled to the memory, and configured to:
calculate a prediction error that is a difference between a predicted image and an actual image, the predicted image being predicted from an image in a traveling direction of a vehicle captured by a vehicle exterior camera that captures a periphery of the vehicle, an actual situation being captured in the actual image;
estimate an attention state of a driver, based on the prediction error; and
output driving assistance information for prompting an attention state and/or a behavior change and/or a consciousness change in relation to a driving manipulation, which are more appropriate for driving at that time, based on the attention state.

### Claim 20

A driving assistance method comprising:
calculating a prediction error that is a difference between a predicted image and an actual image, the predicted image being predicted from an image in a traveling direction of a vehicle captured by a vehicle exterior camera that captures a periphery of the vehicle, an actual situation being captured in the actual image;
estimating an attention state of a driver, based on the prediction error; and
outputting driving assistance information for prompting an attention state and/or a behavior change and/or a consciousness change in relation to a driving manipulation, which are more appropriate for driving at that time, based on the attention state.
