# US 12,179,590 B2 · Image output device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,179,590 B2 - Document ID: US12179590B2 - Status: Granted - Kind: B2 - Application number: 17/427,402 - Filed: 2020-01-30 - Published: 2022-03-31 - Granted: 2024-12-31 - Earliest claimed: 2019-01-31 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Dukyung Jung; Kihyung Lee - Web page: https://patentgrep.com/patent/US12179590B2 ## CPC - B60R 1/001: {integrated in the windows, e.g. Fresnel lenses} - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/22: Display screens - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60K 35/29: Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions - B60K 35/80: Arrangements for controlling instruments - B60K 35/85: Arrangements for transferring vehicle- or driver-related data - B60R 1/22: for viewing an area outside the vehicle, e.g. the exterior of the vehicle - B60W 40/02: related to ambient conditions - G08G 1/0967: Systems involving transmission of highway information, e.g. weather, speed limits (transmission of navigation instructions to the vehicle G08G1/0968) - H04N 5/262: Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects {; Cameras specially adapted for the electronic generation of special effects} - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - B60K 35/23: Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) - B60K 35/90: Calibration of instruments, e.g. setting initial or reference parameters; Testing of instruments, e.g. detecting malfunction - B60K 2360/166: Navigation - B60K 2360/176: Camera images - B60K 2360/177: Augmented reality - B60K 2360/191: Highlight information - B60K 2360/21: using cameras - B60K 2360/5915: Inter vehicle communication - B60K 2360/66: Projection screens or combiners - B60R 2300/205: using a head-up display - B60R 2300/8006: for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo - B60R 2300/804: for lane monitoring - B60R 2300/8093: for obstacle warning - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present invention relates to an image output device mounted on a vehicle to implement augmented reality. One or more of an autonomous driving vehicle, a user terminal, and a server of the present invention can be linked to an artificial intelligence module, a drone (unmanned aerial vehicle, UAV), a robot, an augmented reality (AR) device, a virtual reality (VR) device, a device related to 5G services, etc. ## Claims ### Claim 1 An image output device provided in a vehicle to enable augmented reality, the device comprising: a communication unit comprising (i) a receiver configured to receive images from a camera installed at the vehicle and images captured from another vehicle and (ii) a transmitter configured to transmit image information to an image output unit that is provided at the vehicle and configured to output the image information, a controller configured to: control the receiver to receive, in real time, a forward image capturing an image in front of the vehicle, search for one or more lanes on which the vehicle is expected to travel in the forward image, generate image information including a carpet image or images indicating the searched one or more lanes in lane units, and control the transmitter to transmit the image information to the image output unit such that the image output unit outputs the image information, wherein the controller is further configured to: control the receiver to receive, in real time, an image captured from another vehicle present on a route on which the vehicle is expected to travel, control the transmitter to transmit the forward image and the image captured from the another vehicle to the image output unit such that the image output unit displays the forward image and the image captured from the another vehicle together, generate image information in which the carpet image indicating the route on which the vehicle is expected to travel is superimposed on the image captured from the another vehicle, generate image information in which the carpet image is superimposed on the forward image and the image captured from the another vehicle, control the transmitter to transmit the forward image, on which the carpet image is superimposed, together with the image captured from the another vehicle, on which the carpet image is superimposed, to the image output unit, and based on an image sharing request from a driver of the vehicle to share the image captured from the another vehicle, share a camera of the another vehicle by receiving the image captured from the camera of the another vehicle corresponding to the image sharing request. ### Claim 2 (depends on 1) The device of claim 1, wherein the controller, when the forward image and the image captured from the another vehicle are at least partially the same, combines the forward image and the image captured from the another vehicle and generates image information in which the carpet image is superimposed on the combined image. ### Claim 3 (depends on 2) The device of claim 2, wherein the controller receives the route on which the another vehicle is expected to travel from the another vehicle and generates image information in which a first carpet image indicating the route on which the vehicle is expected to travel and a second carpet image indicating the route on which the another vehicle is expected to travel are superimposed on the combined image. ### Claim 4 (depends on 3) The device of claim 3, wherein the controller generates image information in which a third carpet image having a different shape from the first and second carpet images is superimposed on the combined image when the vehicle and the another vehicle are expected to travel on the same route. ### Claim 5 (depends on 4) The device of claim 4, wherein the controller, when the route of at least one of the vehicle and the another vehicle is changed while the third carpet image is displayed on the image output unit, stops the generation of the image information including the third carpet image, and creates an image in which the first and second carpet images are superimposed on the combined image. ### Claim 6 (depends on 1) The device of claim 1, wherein the controller receives the route on which the another vehicle is expected to travel from the another vehicle and generates image information in which a first carpet image indicating the route on which the vehicle is expected to travel is superimposed on the forward image, and a second carpet image indicating the route on which the another vehicle is expected to travel is superimposed on the image captured from the another vehicle. ### Claim 7 (depends on 6) The device of claim 6, wherein the controller generates image information in which the first carpet image and the second carpet image are superimposed on the image captured from the another vehicle. ### Claim 8 (depends on 7) The device of claim 7, wherein the controller generates image information in which the first carpet image and the second carpet image are superimposed on the forward image. ### Claim 9 (depends on 1) The device of claim 1, wherein the controller is further configured to: control the receiver to receive information regarding the another vehicle from the another vehicle; generate image information including the information regarding the another vehicle and the image captured from the another vehicle; and control the transmitter to transmit the generated image information to the image output unit. ### Claim 10 (depends on 1) The device of claim 1, wherein the controller is further configured to: based on the image sharing request from the driver of the vehicle, generate a message for indicating a payment of points to the another vehicle upon receiving the image sharing request; and control the transmitter to transmit the generated message to the image output unit. ### Claim 11 (depends on 10) The device of claim 10, wherein the controller is further configured to control the transmitter to transmit information about the points paid to the another vehicle, so as to receive the image captured from the another vehicle. ### Claim 12 (depends on 1) The device of claim 1, wherein the controller is further configured to: control the receiver to receive, in real time, images captured from a plurality of vehicles present on the route on which the vehicle is expected to travel; control the transmitter to transmit the forward image and the images captured from the plurality of vehicles to the image output unit; and generate image information in which the carpet image is superimposed on at least one of the forward image and each of the images captured from the plurality of vehicles. ### Claim 13 (depends on 12) The device of claim 12, wherein the controller generates image information such that the images captured from the plurality of vehicles are displayed in a smaller size than the forward image. ### Claim 14 An image output device provided in a vehicle to enable augmented reality, the device comprising: an image output unit configured to output visual information for enabling the augmented reality; a communication unit configured to communicate with another vehicle and a server, and receive, in real time, a forward image capturing an image in front of the vehicle; and a controller configured to control the image output unit to search for one or more lanes on which the vehicle is expected to travel in the forward image, and to output a carpet image or images indicating the searched one or more lanes in lane units, wherein the controller controls the communication unit such that an image captured from another vehicle present on a route on which the vehicle is expected to travel is received in real time, wherein the controller is configured to, based on an image sharing request from a driver of the vehicle to share the image captured from the another vehicle, share a camera of the another vehicle by receiving the image captured from the camera of the another vehicle corresponding to the image sharing request, wherein the controller is configured to, based on receiving the image captured from the camera of the another vehicle, control the image output unit to display the forward image and the image captured from the another vehicle together, wherein the controller controls the image output unit such that the carpet image indicating the route on which the vehicle is expected to travel is superimposed on the image captured from the another vehicle is generated, p1 wherein the controller controls the image output unit to generate image information in which the carpet image is superimposed on the forward image and the image captured from the another vehicle, and wherein the controller outputs, to the image output unit, the forward image on which the carpet image is superimposed together with the image captured from the another vehicle on which the carpet image is superimposed. ### Claim 15 A method for controlling an image output device provided in a vehicle to enable augmented reality, the method comprising: receiving, in real time through a receiver of the image output device, a forward image capturing an image in front of the vehicle, the forward image being captured by a camera of the vehicle; searching for one or more lanes on which the vehicle is expected to travel in the forward image; receiving, in real time through the receiver, an image captured from another vehicle present on a route on which the vehicle is expected to travel; based on receiving the image captured from the another vehicle, outputting the forward image capturing the image in front of the vehicle and the image captured from the another vehicle together; generating image information including a carpet image or images indicating the searched one or more lanes in lane units using at least one of the forward image and the image captured from the another vehicle; and outputting the image information, wherein the image information includes (i) an image in which the carpet image indicating the route on which the vehicle is expected to travel is superimposed on the image captured from the another vehicle and (ii) an image in which the carpet image is superimposed on the forward image and the image captured from the another vehicle, wherein receiving the image captured from the another vehicle comprises: based on an image sharing request from a driver of the vehicle to share the image captured from the another vehicle, sharing a camera of the another vehicle by receiving the image captured from the camera of the another vehicle corresponding to the image sharing request, and receiving, through the receiver in real time, an image captured from the shared camera of the another vehicle. ### Claim 16 (depends on 15) The method of claim 15, wherein the image information includes an image in which the carpet image is superimposed on the forward image and the image captured from the another vehicle, and wherein the method further comprises outputting the forward image on which the carpet image is superimposed together with the image captured from the another vehicle on which the carpet image is superimposed. ### Claim 17 (depends on 16) The method of claim 16, further comprising: combining the forward image and the image captured from the another vehicle when the forward image and the image captured from the another vehicle are at least partially the same; and outputting the combined image to the image output unit. ### Claim 18 (depends on 17) The method of claim 17, further comprising: receiving, through the receiver, the route on which the another vehicle is expected to travel from the another vehicle; generating image information in which a first carpet image indicating the route on which the vehicle is expected to travel and a second carpet image indicating the route on which the another vehicle is expected to travel are superimposed on the combined image; and outputting the generated image information. ### Claim 19 (depends on 18) The method of claim 18, further comprising, when the vehicle and the another vehicle are expected to travel on the same route: generating image information in which a third carpet image having a different shape from the first and second carpet images is superimposed on the combined image; and outputting the generated image information. ### Claim 20 (depends on 1) The device of claim 1, wherein the image output unit is configured to display the image captured from the another vehicle at a side of the forward image.