# US 2018/0322673 A1 · User interface apparatus for vehicle and vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322673 A1 - Document ID: US20180322673A1 - Status: Published application - Kind: A1 - Application number: 15/854,932 - Filed: 2017-12-27 - Published: 2018-11-08 - Earliest claimed: 2017-05-08 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Sunghwan Choi; Ilwan Kim; Jaeho Lee; Dukyung Jung - Web page: https://patentgrep.com/patent/US20180322673A1 ## CPC - G06T 11/60: Creating or editing images; Combining images with text - G01P 15/08: with conversion into electric or magnetic values - G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)} - G06T 3/18: Image warping, e.g. rearranging pixels individually - G06T 3/20: Linear translation of whole images or parts thereof, e.g. panning - G06T 5/70: Denoising; Smoothing - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60R 2300/205: using a head-up display - B60R 2300/302: combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data - B60R 2300/306: using a re-scaling of images - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0141: {characterised by the informative content of the display} - G02B 2027/0183: {Adaptation to parameters characterising the motion of the vehicle} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G}) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR ## Abstract A user interface apparatus for a vehicle includes: an interface unit; a display unit configured to project an augmented reality (AR) graphic object onto a screen; at least one processor; and a computer-readable medium coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, causes the at least one processor to perform operations including: acquiring, through the interface unit, front view image information and vehicle motion information; based on the front view image information, generating the AR graphic object; and based on the vehicle motion information, warping the AR graphic object. ## Claims ### Claim 1 A user interface apparatus for a vehicle comprising: an interface unit; a display unit configured to project an augmented reality (AR) graphic object onto a screen; at least one processor; and a computer-readable medium coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, causes the at least one processor to perform operations comprising: acquiring, through the interface unit, front view image information and vehicle motion information; based on the front view image information, generating the AR graphic object; and based on the vehicle motion information, warping the AR graphic object. ### Claim 2 (depends on 1) The user interface apparatus according to claim 1, wherein acquiring the front view image information and the vehicle motion information comprises: receiving, from a first camera of the vehicle, the front view image information, and wherein the operations further comprise: based on the front view image information, generating first motion information associated with a first motion of the vehicle, the first motion information being defined with respect to a first coordinate system. ### Claim 3 (depends on 2) The user interface apparatus according to claim 2, wherein the operations comprise: receiving, through the interface unit, vehicle motion sensing information from an Inertial Measurement Unit (IMU) of the vehicle; and based on the vehicle motion sensing information, generating second motion information associated with a second motion of the vehicle, the second motion information being defined with respect to a second coordinate system different from the first coordinate system. ### Claim 4 (depends on 3) The user interface apparatus according to claim 3, wherein warping the AR graphic object comprises: transforming the second motion information to be defined with respect to the first coordinate system; and compensating the first motion information based on the transformed second motion information. ### Claim 5 (depends on 4) The user interface apparatus according to claim 4, wherein compensating the first motion information based on the transformed second motion information comprises: counteracting the second motion of the vehicle in the first motion information. ### Claim 6 (depends on 4) The user interface apparatus according to claim 4, wherein warping the AR graphic object comprises: changing, through the display unit, a location of the AR graphic object projected onto the screen based on the compensated first motion information. ### Claim 7 (depends on 6) The user interface apparatus according to claim 6, wherein changing the location of the AR graphic object projected onto the screen based on the compensated first motion information comprises: based on the compensated first motion information, determining a first motion direction of the vehicle; and changing the location of the AR graphic object in a direction opposite to the first motion direction of the vehicle. ### Claim 8 (depends on 6) The user interface apparatus according to claim 6, wherein changing the location of the AR graphic object projected onto the screen based on the compensated first motion information comprises: based on the compensated first motion information, determining a first displacement magnitude of the vehicle; and changing the location of the AR graphic object in correspondence to the first displacement magnitude of the vehicle. ### Claim 9 (depends on 1) The user interface apparatus according to claim 1, wherein the operations comprise: based on the vehicle motion information, determining a change in a motion of the vehicle; and based on the determination of the change in the motion of the vehicle, adding a graphical effect to the AR graphic objects. ### Claim 10 (depends on 9) The user interface apparatus according to claim 9, wherein the graphical effect comprises one or more of fade-out effect or blur effect. ### Claim 11 (depends on 1) The user interface apparatus according to claim 1, wherein the operations comprise: acquiring, through the interface unit, vehicle pitch information associated with a change in a pitch of the vehicle; and warping the AR graphic object based on the vehicle pitch information. ### Claim 12 (depends on 1) The user interface apparatus according to claim 1, wherein the operations comprise: acquiring, through the interface unit, vehicle yaw information associated with a change in yaw of the vehicle which is generated when a steering wheel is rotated; and warping the AR graphic object based on the vehicle yaw information. ### Claim 13 (depends on 1) The user interface apparatus according to claim 1, further comprising a second camera configured to capture images of eyes of a user of the vehicle, wherein the operations comprise: based on the captured images, detecting the eyes of the user; tracking a change in a location of the eyes of the user in response to a motion of the vehicle; and warping the AR graphic object based on the tracked change in the location of the eyes of the user. ### Claim 14 (depends on 1) The user interface apparatus according to claim 1, wherein the operations comprise: based on the front view image information, generating frontal object information associated with an object located forward of the vehicle; determining a first time corresponding to an estimated time of arrival of the vehicle at the object; and changing, through the display unit, a location of the AR graphic object projected onto the screen based on the first time. ### Claim 15 (depends on 14) The user interface apparatus according to claim 14, wherein the operations comprise: determining that the object is a bump; based on the determination that the object is a bump, acquiring height information of the bump; and based on the height information, generating, within the first time, a first anticipated motion information associated with a first anticipated motion of the vehicle at the bump. ### Claim 16 (depends on 15) The user interface apparatus according to claim 15, wherein the operations comprise: receiving, through the interface unit, vehicle motion sensing information associated with the bump from an Inertial Measurement Unit (IMU); based on the vehicle motion sensing information, generating second motion information associated with the bump; and compensating the first anticipated motion information based on the second motion information. ### Claim 17 (depends on 16) The user interface apparatus according to claim 16, wherein the operations comprise: determining that the object is a curved section; based on the determination that the object is a curved section, acquiring curvature information of the curved section; and based on the curvature information, generating, within the first time, a second anticipated motion information associated with a second anticipated motion of the vehicle at the curved section. ### Claim 18 (depends on 1) The user interface apparatus according to claim 1, wherein the display unit comprises: an imaging unit configured to output the AR graphic object; a flat mirror configured to reflect the AR graphic object; and a concave mirror configured to reflect the AR graphic object reflected by the flat mirror toward the screen. ### Claim 19 (depends on 18) The user interface apparatus according to claim 18, wherein the display unit further comprises a flat mirror drive unit configured to adjust a position of the flat mirror, and wherein the operations comprise: warping the AR graphic object by adjusting, through the flat mirror drive unit, the position of the flat mirror. ### Claim 20 (depends on 1) A vehicle comprising: a plurality of wheels; a power source configured to drive at least one of the plurality of wheels; and the user interface apparatus according to claim 1.