# US 11,897,394 B2 · Head up display and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,897,394 B2 - Document ID: US11897394B2 - Status: Granted - Kind: B2 - Application number: 17/380,543 - Filed: 2021-07-20 - Published: 2022-03-03 - Granted: 2024-02-13 - Earliest claimed: 2020-08-27 - Assignees: Naver Labs Corporation - Applicants: Naver Labs Corporation - Inventors: Jae Won Cha; Jeseon Lee; Kisung Kim; Jongjin Park; Eunyoung Jeong; Yongho Shin - Web page: https://patentgrep.com/patent/US11897394B2 ## CPC - B60R 11/0229: {for displays, e.g. cathodic tubes} - B60K 35/211: {producing three-dimensional [3D] effects, e.g. stereoscopic images} - B60K 35/23: Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) - B60K 35/231: characterised by their arrangement or structure for integration into vehicles - B60K 35/235: with means for detecting the driver's gaze direction or eye points - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60Q 2400/50: Projected symbol or information, e.g. onto the road or car body - B60R 2300/205: using a head-up display - B60R 2300/30: characterised by the type of image processing - B60R 2300/308: by overlaying the real scene, e.g. through a head-up display on the windscreen - G01C 21/365: {Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself} - G02B 27/01: Head-up displays - G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0141: {characterised by the informative content of the display} - G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence) - G09G 2380/10: Automotive applications - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract A head up display for a vehicle including a display device configured to output light forming an image, an optical system configured to control a path of the light such that the image is output towards a light transmission region, and a controller configured to generate the image based on a first view and a second view such that a virtual image is produced on a ground surface in the light transmission region, the first view being towards the ground surface, the second view being towards a 3D space above the ground surface, the first view and the second view being based on an eye-box, the ground surface being in front of the vehicle, and the virtual image including a graphic object having a stereoscopic effect, and control the display device to output the image. ## Claims ### Claim 1 A head up display for a vehicle, comprising: a display device configured to output light forming an image; an optical system configured to control a path of the light such that the image is output towards a light transmission region; and a controller configured to, generate the image based on a first view and a second view such that a virtual image is produced on a ground surface in the light transmission region, the first view being towards the ground surface, the second view being towards a 3D space above the ground surface, the first view and the second view being based on an eye-box, the ground surface being in front of the vehicle, the virtual image including a graphic object having a stereoscopic effect, and control the display device to output the image, wherein the optical system is configured to output the virtual image corresponding to the image on the ground surface, wherein the controller is configured to generate the graphic object with respect to the ground surface by using the optical system, and wherein the graphic object is generated by, making the first view and the second view on the basis of the eye-box consistent with each other, and projecting a first object onto the ground surface using a first conversion matrix, the first object corresponding to the graphic object, and the first conversion matrix corresponding to the first view and the second view. ### Claim 2 (depends on 1) The head up display for a vehicle of claim 1, wherein the first view is a driver view of the ground surface; the second view is a camera view of a virtual camera; and the controller is configured to generate the image including making the driver view and the camera view consistent with each other. ### Claim 3 (depends on 2) The head up display for a vehicle of claim 2, wherein the first object is formed in the 3D space based on the camera view, and the projecting including projecting the first object onto a 2D plane of the ground surface based on the driver view. ### Claim 4 (depends on 1) The head up display for a vehicle of claim 1, wherein the controller is configured to generate the graphic object using characteristics of the optical system, and the first object is a 3D object. ### Claim 5 (depends on 4) The head up display for a vehicle of claim 4, wherein the controller is configured to: generate a projected image by projecting the 3D object onto the ground surface; and convert the projected image into the image using the characteristics of the optical system. ### Claim 6 (depends on 4) The head up display for a vehicle of claim 4, wherein the controller is configured to convert the 3D object into the graphic object using a scalar product of the first conversion matrix and a second conversion matrix, the first conversion matrix being configured to project the 3D object onto the ground surface to obtain a projected image, and the second conversion matrix being configured to convert the projected image into the image. ### Claim 7 (depends on 1) The head up display for a vehicle of claim 1, wherein the virtual image includes a 2D graphic object formed on a virtual image plane corresponding to the ground surface; and the graphic object having the stereoscopic effect is a 3D graphic object erected on the virtual image plane. ### Claim 8 (depends on 7) The head up display for a vehicle of claim 7, wherein the image is a single image including both the 2D graphic object and the 3D graphic object. ### Claim 9 (depends on 7) The head up display for a vehicle of claim 7, wherein the 2D graphic object includes information related to at least one of a lane, a driving speed or a driving direction, and the 3D graphic object includes information related to at least one of a point of interest (POI), a guiding message, a destination or a peripheral vehicle. ### Claim 10 (depends on 1) The head up display for a vehicle of claim 1, wherein the light transmission region is at a combiner or a windshield. ### Claim 11 (depends on 1) The head up display for a vehicle of claim 1, wherein the making the first view and the second view consistent with each other includes configuring a third conversion matrix to be equal to a fourth conversion matrix, the third conversion matrix representing the second view, and the fourth conversion matrix representing the first view. ### Claim 12 A head up display for a vehicle, comprising: a display device configured to output an image; an image forming optical system configured to convey light corresponding to the image through a light transmission region such that a virtual image corresponding to the image appears on a ground surface in the light transmission region, the ground surface being in front of a vehicle; and a controller configured to, generate the image based on an eye-box such that a 2D graphic object and a 3D graphic object appear at the ground surface in the light transmission region, the 2D graphic object corresponding to the ground surface and the 3D graphic object having a stereoscopic effect, the eye-box being defined based on a viewing angle of a driver of the vehicle, the 3D graphic object corresponding to a 3D object projected onto the ground surface using a first conversion matrix, the first conversion matrix corresponding to a first view and a second view, the first view being towards the ground surface, the second view being towards a 3D space above the ground surface, and the first view and the second view being based on the eye-box, and control the display device to output the image. ### Claim 13 (depends on 12) The head up display of claim 12, wherein the controller is configured to generate the 3D graphic object using characteristics of the image forming optical system. ### Claim 14 A control method of a head up display for a vehicle, comprising: generating an image based on a first view and a second view such that a virtual image is produced on a ground surface in a light transmission region, the first view being towards the ground surface, the second view being towards a 3D space above the ground surface, the first view and the second view being based on an eye-box, the ground surface being in front of a vehicle, the virtual image including a graphic object having a stereoscopic effect; and controlling a display device to output the image to an optical system configured to convey the image to the light transmission region, wherein the virtual image is produced by an optical system configured to control a path of light such that the image formed by the light is output towards the light transmission region, wherein the graphic object is generated by using the optical system, and wherein the graphic object is generated by, making the first view and the second view on the basis of the eye-box consistent with each other, and projecting a first object onto the ground surface using a first conversion matrix, the first object corresponding to the graphic object to obtain a projected image, and the first conversion matrix corresponding to the first view and the second view. ### Claim 15 (depends on 14) The method of claim 14, wherein the generating an image comprises: converting the projected image into the image based on characteristics of the optical system. ### Claim 16 (depends on 14) The method of claim 14, wherein the first object is a 3D object; and the generating an image comprises generating the graphic object using a scalar product of the first conversion matrix and a second conversion matrix, the first conversion matrix being configured to project the 3D object onto the ground surface to obtain the projected image, and the second conversion matrix being configured to convert the projected image into the image. ### Claim 17 (depends on 14) The method of claim 14, wherein the virtual image includes a 2D graphic object formed on a virtual image plane corresponding to the ground surface; and the graphic object having the stereoscopic effect is a 3D graphic object erected on the virtual image plane. ### Claim 18 (depends on 17) The method of claim 17, wherein the image is a single image including both the 2D graphic object and the 3D graphic object. ### Claim 19 (depends on 17) The method of claim 17, wherein the 2D graphic object includes information related to at least one of a lane, a driving speed or a driving direction, and the 3D graphic object includes information related to at least one of a point of interest (POI), a guiding message, a destination or a peripheral vehicle. ### Claim 20 (depends on 14) The method of claim 14, wherein the light transmission region is at a combiner or a windshield. ### Claim 21 (depends on 14) The method of claim 14, further comprising: selecting the graphic object in response to determining information related to at least one of a point of interest (POI), a guiding message, a destination or a peripheral vehicle is to be included in the image.