# US 2023/0054971 A1 · Vehicle display device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0054971 A1
- Document ID: US20230054971A1
- Status: Published application
- Kind: A1
- Application number: 17/796,967
- Filed: 2021-02-24
- Published: 2023-02-23
- Earliest claimed: 2020-02-28
- Assignees: Nippon Seiki Co., Ltd.
- Applicants: Nippon Seiki Co., Ltd.
- Inventors: Makoto Hada
- Web page: https://patentgrep.com/patent/US20230054971A1

## CPC

- G06T 7/73: using feature-based methods
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60K 35/234: controlling the brightness, colour or contrast of virtual images depending on the driving conditions or on the condition of the vehicle or the driver
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/60: Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10)
- B60K 35/81: for controlling displays
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/0179: {Display position adjusting means not related to the information to be displayed}
- G09G 3/002: {to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT}
- B60K 35/65: Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
- B60K 2360/21: using cameras
- B60K 2360/23: using reflectors
- B60K 2360/741: Instruments adapted for user detection
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/0181: {Adaptation to the pilot/driver}
- G06T 2207/30201: Face
- G09G 2380/10: Automotive applications
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A head-up display device as a vehicle display device projects display light onto a front windshield of a vehicle, displays a virtual image by reflected display light, on the front windshield, so as to be superimposed on a real scene transparent through the front windshield, captures an image of a driver of the vehicle by projecting infrared light onto the driver, and detects the positions of the eyes of the driver based on the captured image. A control unit detects the position of a feature point of the face of the driver in the captured image and stores a positional relationship with the detected positions of the eyes, and, when it is difficult to detect the positions of the eyes in die captured image, estimates a viewpoint position of the driver based on the position of the feature point detected at this time and the positional relationship stored beforehand.

## Claims

### Claim 1

A vehicle display device, which projects display light onto a reflective light-transmissive member provided in a vehicle, generates a virtual image by the display light reflected by the reflective light-transmissive member and displays the virtual image to be superimposed on an actual view, which is seen transparently through the reflective light-transmissive member, and also projects infrared light onto a user of the vehicle to capture an image of the user, thereby enabling detection of a viewpoint position of the user based on the captured image, the vehicle display device comprising:
an eye detection means that detects a position of an eye of the user in the captured image;
a viewpoint position detection means that detects the viewpoint position based on the position of the eye detected by the eye detection means;
a feature point detection means that detects a position of a feature point of the user in the captured image; and
a storage means that stores a positional relationship between the position of the eye detected by the eye detection means and the position of the feature point detected by the feature point detection means, wherein
the viewpoint position detection means estimates, when it is difficult to detect the position of the eye by the eye detection means, the viewpoint position based on the position of the feature point detected by the feature point detection means, and the positional relationship stored in the storage means.

### Claim 2 (depends on 1)

The vehicle display device according to claim 1, further comprising a face direction detection means that detects a direction of a face of the user, wherein
the viewpoint position detection means estimates the viewpoint position based on the direction of the face detected by the face direction detection means.

### Claim 3 (depends on 1)

The vehicle display device according to claim 1, wherein:
the eye detection means detects positions of left and right eyes of the user in the captured image obtained by regular or irregular image capturing during the display of the virtual image; and
the storage means stores the positional relationship obtained from the captured image of a time when a separated distance between the left and right eyes is maximized on or after a predetermined reference time.

### Claim 4 (depends on 1, 2)

The vehicle display device according to claim 1 or 2, further comprising a line-of-sight information acquisition means that acquires line-of-sight information of the user, wherein:
the eye detection means detects the position of the eye in the captured image obtained by regular or irregular image capturing during the display of the virtual image; and
the storage means stores the positional relationship obtained from the captured image of a time when the user is facing a front, based on the line-of-sight information acquired by the line-of-sight information acquisition means.