# US 10,311,618 B2 · Virtual viewpoint position control device and virtual viewpoint position control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,618 B2
- Document ID: US10311618B2
- Status: Granted
- Kind: B2
- Application number: 15/765,957
- Filed: 2016-09-09
- Published: 2018-10-04
- Granted: 2019-06-04
- Earliest claimed: 2015-10-08
- Assignees: Nissan Motor Co., Ltd.
- Applicants: Nissan Motor Co., Ltd.
- Inventors: Yoshinori Kusayanagi; Norimasa Kishi
- Web page: https://patentgrep.com/patent/US10311618B2

## CPC

- H04N 7/181: {for receiving images from a plurality of remote sources}
- B60R 1/27: providing all-round vision, e.g. using omnidirectional cameras
- B60R 1/28: with an adjustable field of view
- G06T 11/65: on geographic maps
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- B60R 2300/105: using multiple cameras
- B60R 2300/302: combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
- B60R 2300/602: with an adjustable viewpoint
- B60R 2300/605: the adjustment being automatic
- B60R 2300/607: from a bird's eye viewpoint
- B60R 2300/8066: for monitoring rearward traffic
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A display assistance device detects the steering angle of a vehicle by using a steering angle sensor, and changes the position of a virtual viewpoint behind the host vehicle based on the detected steering angle. Then, the display assistance device converts images captured using a plurality of cameras such as a front camera and a rear camera into an overhead image of a downward view from the virtual viewpoint, and displays the converted overhead image on a display with a host vehicle icon superimposed on the overhead image, the host vehicle icon indicating the position of the vehicle.

## Claims

### Claim 1

A display assistance device that converts images captured by a plurality of cameras installed on a vehicle into an overhead image of a downward view from a virtual viewpoint in air, and displays the overhead image on a display with a host vehicle icon superimposed on the overhead image, the host vehicle icon indicating a position of the vehicle, the display assistance device comprising:
a computer that determines whether or not the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane, and changes a position of the virtual viewpoint if determining that the vehicle is making the lane change,
wherein the computer changes the position of the virtual viewpoint when the position of the virtual viewpoint moves out of a host vehicle lane,
the computer does not change the position of the virtual viewpoint when the position of the virtual viewpoint does not move out of the host vehicle lane, and
in a case where the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane, the computer changes the position of the virtual viewpoint toward the next lane.

### Claim 2 (depends on 1)

The display assistance device according to claim 1, wherein in a case where the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane, the computer changes the position of the virtual viewpoint to above the next lane or above a boundary line between the travel lane and the next lane.

### Claim 3 (depends on 1)

The display assistance device according to claim 1, wherein when a predetermined time elapses after the vehicle makes a lane change, the computer changes the position of the virtual viewpoint to a lane to which the lane change was made.

### Claim 4 (depends on 1)

The display assistance device according to claim 1, wherein the computer sets the position of the virtual viewpoint as a position relative to the host vehicle in accordance with a speed of the host vehicle.

### Claim 5 (depends on 1)

The display assistance device according to claim 1,
wherein in a case where the vehicle is traveling on a curve and making a lane change from an outside lane of the curve to an inside lane of the curve, the computer moves the position of the virtual viewpoint toward an inner side of the curve or turns a field of view from the virtual viewpoint toward the inner side of the curve, and
in a case where the vehicle is traveling on the curve and making a lane change from the inside lane of the curve to the outside lane of the curve, the computer moves the position of the virtual viewpoint toward the inner side of the curve or turns the field of view from the virtual viewpoint toward an outer side of the curve.

### Claim 6 (depends on 1)

The display assistance device according to claim 1, further comprising:
a laser that detects an environment around the vehicle, wherein in a case where the laser detects a narrower road ahead of the vehicle, the computer changes the position of the virtual viewpoint such that the virtual viewpoint faces a center of the narrower road straightforwardly.

### Claim 7

A display assistance method involving converting images captured by a plurality of cameras installed on a vehicle into an overhead image of a downward view from a virtual viewpoint in air, and displaying the overhead image on a display with a host vehicle icon superimposed on the overhead image, the host vehicle icon indicating a position of the vehicle, the display assistance method comprising:
a first step of determining whether or not the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane; and
a second step of changing a position of the virtual viewpoint if determining that the vehicle is making the lane change,
wherein in the second step, the position of the virtual viewpoint is changed in a case where the position of the virtual viewpoint moves out of a host vehicle lane,
wherein in the second step, the position of the virtual viewpoint is not changed in a case where the position of the virtual viewpoint does not move out of the host vehicle lane, and
wherein in a case where the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane, the changes the position of the virtual viewpoint toward the next lane.

### Claim 8

A display assistance device that converts images captured by a plurality of cameras installed on a vehicle into an overhead image of a downward view from a virtual viewpoint in air, and displays the overhead image on a display with a host vehicle icon superimposed on the overhead image, the host vehicle icon indicating a position of the vehicle, the display assistance device comprising:
a computer that determines whether or not the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane, and changes a position of the virtual viewpoint if determining that the vehicle is making the lane change,
wherein the computer changes the position of the virtual viewpoint when the position of the virtual viewpoint moves out of a host vehicle lane,
the computer does not change the position of the virtual viewpoint when the position of the virtual viewpoint does not move out of the host vehicle lane, and
the computer sets the position of the virtual viewpoint as a position relative to the host vehicle in accordance with a speed of the host vehicle.

### Claim 9 (depends on 8)

The display assistance device according to claim 8, wherein in a case where the vehicle is making a lane change from a travel lane in which the vehicle is traveling to a next lane next to the travel lane, the computer changes the position of the virtual viewpoint toward the next lane.

### Claim 10 (depends on 9)

The display assistance device according to claim 9,
wherein the computer superimposes an upper end line indicating an upper end of the host vehicle icon and a lower end line indicating a lower end of the host vehicle icon onto the next lane.

### Claim 11 (depends on 10)

The display assistance device according to claim 10, wherein the computer superimposes the host vehicle icon as a transparent icon capable of being seen through.

### Claim 12 (depends on 10)

The display assistance device according to claim 10, wherein the computer superimposes the host vehicle icon onto the overhead image above a center thereof.