# US 2015/0145996 A1 · Vehicular display apparatus, vehicular display method, and vehicular display program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0145996 A1
- Document ID: US20150145996A1
- Status: Published application
- Kind: A1
- Application number: 14/401,305
- Filed: 2013-04-05
- Published: 2015-05-28
- Earliest claimed: 2012-05-18
- Assignees: Nissan Motor Co., Ltd.
- Applicants: Nissan Motor Co., Ltd.
- Inventors: Seigo Watanabe; Norimasa Kishi; Takura Yanagi; Yoshiro Takamatsu
- Web page: https://patentgrep.com/patent/US20150145996A1

## CPC

- B60R 1/24: in front of the vehicle
- B60R 1/30: providing vision in the non-visible spectrum, e.g. night or infrared vision
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 11/00: Two-dimensional [2D] image generation
- G08G 1/09626: {where the origin of the information is within the own vehicle, e.g. a local storage device, digital map}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- B60R 2300/305: merging camera image with lines or icons
- B60R 2300/308: by overlaying the real scene, e.g. through a head-up display on the windscreen
- B60R 2300/8053: for bad weather conditions or night vision
- G06T 2207/10024: Color image
- G06T 2207/10048: Infrared image
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Captured images obtained beforehand by imaging ranges corresponding to peripheries of a vehicle, or processed images generated to correspond to the captured images are stored to be associated with stored road information of a preset area. A superimposed area having a clarity lower than a preset clarity threshold is set in a captured image of a periphery of the vehicle. A superimposition image is extracted among the stored images, the superimposition image being an image of a range corresponding to the captured image. A superimposed image is generated by superimposing an image in an area in the extracted superimposition image corresponding to the superimposed area on the captured mage. A video generated from the superimposed image is displayed at a position visible from a driver.

## Claims

### Claim 1

A vehicular display apparatus comprising:
a map data storage unit in which road information of a preset area is stored;
a superimposition image information storage unit in which at least ones of captured images obtained beforehand by imaging ranges corresponding to peripheries of a vehicle or processed images generated to correspond to the captured images are stored to be associated with the road information stored in the map data storage unit;
a vehicle position detection unit configured to detect a position of the vehicle;
a vehicle travel direction detection unit configured to detect a travel direction of the vehicle;
a normal image capture unit configured to capture an image of a periphery of the vehicle;
a superimposed area setting unit configured to set a superimposed area having a clarity lower than a preset clarity threshold in the image captured by the normal image capture unit;
a superimposition image extraction unit configured to extract a superimposition image on a basis of the position of the vehicle detected by the vehicle position detection unit, the travel direction of the vehicle detected by the vehicle travel direction detection unit, the road information stored in the map data storage unit, among the images stored in the superimposition image information storage unit, the superimposition image being an image of a range corresponding to the image captured by the normal image capture unit;
an image superposition unit configured to generate a superimposed image, by superimposing the image in an area corresponding to the superimposed area set by the superimposed area setting unit in the superimposition image extracted by the superimposition image extraction unit, on the image captured by the normal image capture unit; and
an image display unit configured to display a video generated from the superimposed image generated by the image superposition unit at a position visible from a driver of the vehicle.

### Claim 2 (depends on 1)

The vehicular display apparatus according to claim 1, wherein the superimposed area setting unit is configured to detect a low gradation area where a gradation is lower than a preset gradation threshold, in the image captured by the normal image capture unit, and to set the detected low gradation area as the superimposed area.

### Claim 3 (depends on 2)

The vehicular display apparatus according to claim 2, wherein the image superposition unit is configured to, when there is a feature specific to a road in the superimposed area set by the superimposed area setting unit, generate the superimposed image in which there is only an image indicating the feature included in the superimposition image extracted by the superimposition image extraction unit, in an area where the feature lies in the image captured by the normal image capture unit.

### Claim 4 (depends on 2)

The vehicular display apparatus according to claim 2, wherein the image superposition unit is configured to, when there is a road structure in the superimposed area set by the superimposed area setting unit, generate the superimposed image in which, in the area where the road structure lies in the image captured by the normal image capture unit, a superimposition degree of the superimposition image extracted by the superimposition image extraction unit on the image captured by the normal image capture unit is set to a degree proportional to a contrast of the image captured by the normal image capture unit.

### Claim 5 (depends on 4)

The vehicular display apparatus according to claim 4, wherein the image superposition unit is configured to generate the superimposed image in which the superimposition degree of the superimposition image extracted by the superimposition image extraction unit on the image captured by the normal image capture unit increases from a lower part to an upper part in a screen on which the image display unit displays the video.

### Claim 6 (depends on 4)

The vehicular display apparatus according to claim 4, wherein the image superposition unit is configured to generate the superimposed image in which the superimposition degree of the superimposition image extracted by the superimposition image extraction unit on the image captured by the normal image capture unit increases from an area indicating a position closer to the normal image capture unit to another area indicating another position farther from the normal image capture unit in the superimposed image.

### Claim 7 (depends on 2)

The vehicular display apparatus according to claim 2, wherein the image superposition unit is configured to, when there is another vehicle which is expected to move into a course of the vehicle in the image captured by the normal image capture unit, generate the superimposed image in which a superimposition degree of the superimposition image extracted by the superimposition image extraction unit on the image captured by the normal image capture unit in an area where the another vehicle lies in the image captured by the normal image capture unit is less than another superimposition degree in another area other than the area where the another vehicle lies.

### Claim 8 (depends on 1)

The vehicular display apparatus according to claim 1, further comprising a covered area detection unit configured to detect a covered area in the image captured by the normal image capture unit, in the covered area, a road structure and a feature specific to a road included in the road information stored in the map data being covered with a cover,
wherein, the superimposition image extraction unit is configured to extract a covered object image from the images stored in the superimposition image information storage unit, the covered object image being an image of the road structure and the feature specific to the road being covered with the cover in the covered area detected by the covered area detection unit, and
the image superposition unit is configured to generate the superimposed image by superimposing the covered object image extracted by the superimposition image extraction unit and subjected to transparent processing, on the image captured by the normal image capture unit in the covered area detected by the covered area detection unit.

### Claim 9 (depends on 1)

The vehicular display apparatus according to claim 1, further comprising:
an infrared image capture unit configured to capture an infrared image of the periphery of the vehicle; and
a superimposition image coloring process unit configured to refer to an outline in the infrared image captured by the infrared image capture unit and a color displayed in the image captured by the normal image capture unit to color the superimposition image extracted by the superimposition image extraction unit.

### Claim 10 (depends on 1)

The vehicular display apparatus according to claim 1, further comprising a superimposition image coloring process unit configured to refer to a color displayed in the image captured by the normal image capture unit to color the superimposition image extracted by the superimposition image extraction unit.

### Claim 11 (depends on 1)

The vehicular display apparatus according to claim 1, further comprising a driver authentication unit configured to authenticate the driver of the vehicle,
wherein the road information is stored in the map data storage unit to be individually associated with the driver authenticated by the driver authentication unit.

### Claim 12

A vehicular display apparatus comprising:
a superimposition image information storage unit in which road positions preset on a road and images captured at the road positions, respectively, beforehand are stored to be associated with each other;
a vehicle position detection unit configured to detect a position of a vehicle;
a vehicle travel direction detection unit configured to detect a travel direction of the vehicle;
a normal image capture unit configured to capture a normal image, which is an image in the travel direction of the vehicle detected by the vehicle travel direction detection unit;
an image superposition unit configured to extract a superimposition area section from a superimposition image being an image of a range corresponding to the normal image captured by the normal image capture unit among the images stored in the superimposition image information storage unit, the superimposition area section corresponding to an unclear area in the normal image captured by the normal image capture unit where a clarity is lower than a preset clarity threshold, to generate a superimposed image by superimposing the extracted superimposition area section on the unclear area in the normal image; and
an image display unit configured to display the superimposed image generated by the image superposition unit at a position visible from a driver of the vehicle.

### Claim 13

A vehicular display method comprising:
storing at least ones of captured images obtained beforehand by imaging ranges corresponding to peripheries of a vehicle or processed images generated to correspond to the captured images, to be associated with road information;
extracting a superimposition image being an image of a range corresponding to a captured image of a periphery of the vehicle among the stored images;
setting a superimposed area having a clarity lower than a preset clarity threshold in the captured image of the periphery of the vehicle, and generating a superimposed image by superimposing an image in the extracted superimposition image corresponding the superimposed area on the captured image of the periphery of the vehicle; and
displaying a video generated from the superimposed image at a position visible from a driver of the vehicle.

### Claim 14

A non-transitory computer readable medium storing a vehicular display program for causing, a computer to execute a process, the process comprising:
a map data storing step for storing road information of a preset area;
a superimposition image information storing step for storing at least ones of captured images obtained beforehand by imaging ranges corresponding to peripheries of a vehicle or processed images generated to correspond to the captured images, to be associated with the road information stored in the map data storing step;
a superimposed area setting step for setting a superimposed area having a clarity lower than a preset clarity threshold in an image captured by a normal image capture unit capturing an image of a periphery of the vehicle;
a superimposition image extracting step for extracting a superimposition image on a basis of a position of the vehicle detected by a vehicle position detection unit detecting the position of the vehicle, a travel direction of the vehicle detected by a vehicle travel direction detection unit detecting the travel direction of the vehicle, the road information stored in the map data storing step, among the images stored in the superimposition image information storing step, the superimposition image being an image of a range corresponding to the image captured by the normal image capture unit;
an image superposing step for generate a superimposed image, by superimposing an image in an area corresponding to the superimposed area set in the superimposed area setting step in the superimposition image extracted in the superimposition image extracting step, on the image captured by the normal image capture unit; and
an image displaying step for displaying a video generated from the superimposed image generated in the image superposing step at a position visible from a driver of the vehicle.
