# US 12,214,662 B2 · Electronic device mounted on vehicle and operating method of the electronic device to receive information about proximate objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,214,662 B2
- Document ID: US12214662B2
- Status: Granted
- Kind: B2
- Application number: 17/953,431
- Filed: 2022-09-27
- Published: 2023-04-20
- Granted: 2025-02-04
- Earliest claimed: 2021-10-20
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Youngil Kim; Taejun Kim; Bosung Seo; Sanghoon Lee; Youngseok Lee; Woong Lee
- Web page: https://patentgrep.com/patent/US12214662B2

## CPC

- B60K 35/81: for controlling displays
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60K 35/23: Head-up displays [HUD] (optical aspects of head-up displays G02B27/01)
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/50: Instruments characterised by their means of attachment to or integration in the vehicle (B60K35/231 takes precedence)
- B60K 35/60: Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10)
- B60K 35/85: Arrangements for transferring vehicle- or driver-related data
- B60R 11/02: for radio sets, television sets, telephones, or the like; Arrangement of controls thereof
- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- B60W 40/02: related to ambient conditions
- B60W 50/00: Details of control systems for road vehicle drive control not related to the control of a particular sub-unit {, e.g. process diagnostic or vehicle driver interfaces}
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G06N 3/045: Combinations of networks
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/20: in augmented reality scenes
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/096725: {where the received information generates an automatic action on the vehicle control}
- G08G 1/09675: {where a selection from the received information takes place in the vehicle}
- G08G 1/096783: {where the origin of the information is a roadside individual element}
- G08G 1/096791: {where the origin of the information is another vehicle}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- B60K 35/213: {Virtual instruments}
- B60K 35/29: Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
- B60K 2360/11: Instrument graphical user interfaces or menu aspects
- B60K 2360/164: Infotainment
- B60K 2360/166: Navigation
- B60K 2360/177: Augmented reality
- B60K 2360/178: Warnings
- B60K 2360/179: Distances to obstacles or vehicles
- B60K 2360/191: Highlight information
- B60K 2360/589: Wireless data transfers
- B60K 2360/5915: Inter vehicle communication
- G06V 2201/08: Detecting or categorising vehicles
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

Provided are an electronic device mounted on a vehicle and an operating method of the electronic device. An electronic device, according to an embodiment of the disclosure, obtains a surrounding environment image by using a camera, detects at least one object from the surrounding environment image by using an artificial intelligence (AI) model, receives a vehicle to everything (V2X) data set including information about the at least one object, selects a first object in the detected at least one object by a user input, obtains information about a type of the first object by using a detection result determined based on the surrounding environment image and the received V2X data set, and displays a user interface (UI) for wireless communication connection with the first object, based on the information about the type of the first object.

## Claims

### Claim 1

An electronic device mounted on a vehicle, the electronic device comprising:
a communication interface;
a user input interface;
a display;
a camera mounted on the vehicle and configured to obtain a surrounding environment image by photographing a surrounding environment of the vehicle;
a memory storing at least one instruction; and
at least one processor configured to execute the at least one instruction stored in the memory,
wherein the at least one processor is further configured to, by executing the at least one instruction:
identify a plurality of objects from the surrounding environment image by using an object detection model;
receive a message, the message including vehicle to everything (V2X) data set including information about the plurality of objects;
match each one of the identified plurality of objects from the surrounding environment image with a respective plurality of portions of the information about the plurality of objects included in the V2X data set;
select a first object in the identified plurality of objects based on a user input received through the user input interface;
obtain information about a type of the first object by using an identification result determined based on the surrounding environment image and the received V2X data set; and
display a user interface (UI) for wireless communication connection with the first object on the display, based on the information about the type of the first object,
wherein the UI includes a menu UI selectable to perform a function or an operation determined based on the type of the first object and an object information UI representing information about at least one of profile information, wireless communication connection signal strength, or battery level of the first object.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to, by executing the at least one instruction, control the display to display images corresponding to the plurality of objects included in the surrounding environment image in different colors, according to types of the plurality of objects.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to, by executing the at least one instruction, control the display to display a focusing UI including an icon or a symbol encompassing an image corresponding to the first object on the surrounding environment image.

### Claim 4 (depends on 3)

The electronic device of claim 3, wherein the at least one processor is further configured to, by executing the at least one instruction, control the display to display the focusing UI encompassing the first object in different colors according to moving speed of the first object.

### Claim 5 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to, by executing the at least one instruction:
receive a content image including an advertisement image or an introduction image from the first object through the communication interface; and
control the display to display the received content image, wherein the UI is displayed after the content image is displayed.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein matching further comprises:
obtain, from the surrounding environment image, license plate information, position information, type information, and/or predicted speed information of a first object;
match the obtained information with a first portion of the respective plurality of portions of the information included in the received V2X data set; and
extract identification information and type information of the first object from the received V2X data set, based on a result of the matching.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to identify corresponding information about the plurality of objects, and
wherein the matching is based on the corresponding information.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the plurality of objects from the surrounding environment image are objects selected from a handheld portable electronic device and a stationary road side unit, and wherein the matching comprises matching at least one of the handheld portable electronic devices and the stationary road side unit with a respective plurality of portions of the information about the plurality of objects.

### Claim 9

An operating method of an electronic device mounted on a vehicle, the operating method comprising:
obtaining a surrounding environment image by photographing a surrounding environment of the vehicle by using a camera mounted on a front of the vehicle;
identifying plurality of objects from the surrounding environment image by using an artificial intelligence (AI) model;
receiving a message, the message including vehicle to everything (V2X) data set including information about the plurality of objects;
matching each of the identified plurality of objects from the surrounding environment image with a respective plurality of portions of the information about the plurality of objects included in the V2X data set;
selecting a first object in the identified plurality of objects, based on a user input;
obtaining information about a type of the first object by using an identification result determined based on the surrounding environment image and the received V2X data set; and
displaying a user interface (UI) for wireless communication connection with the first object on a display, based on the information about the type of the first object,
wherein the UI includes a menu UI selectable to perform a function or an operation determined based on the type of the first object and an object information UI representing information about at least one of profile information, wireless communication connection signal strength, or battery level of the first object.

### Claim 10 (depends on 9)

The operating method of claim 9, wherein the displaying of the UI further comprises displaying images corresponding to the plurality of objects included in the surrounding environment image in different colors, according to types of the plurality of objects.

### Claim 11 (depends on 9)

The operating method of claim 9, wherein the displaying of the UI further comprises displaying a focusing UI including an icon or a symbol encompassing an image corresponding to the first object on the surrounding environment image.

### Claim 12 (depends on 11)

The operating method of claim 11, wherein the displaying of the focusing UI further comprises displaying the focusing UI encompassing the first object in different colors according to moving speed of the first object.

### Claim 13 (depends on 9)

The operating method of claim 9, further comprising displaying advertisement content or an introduction image received from the first object,
wherein the displaying of the UI is performed after the advertisement content or the introduction image is displayed.

### Claim 14 (depends on 9)

The operating method of claim 9, wherein matching further comprises:
obtaining, from the surrounding environment image, license plate information, position information, type information, and/or predicted speed information of the first object;
matching the obtained information with information included in the received V2X data set; and
extracting identification information and type information of the first object from the V2X data set, based on a result of the matching.

### Claim 15

A computer program product comprising a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises instructions, which are readable by an electronic device, to perform operations of:
obtaining a surrounding environment image by photographing a surrounding environment of a vehicle by using a camera;
identifying plurality of objects from the surrounding environment image by using an artificial intelligence (AI) model;
receiving a message, the message including vehicle to everything (V2X) data set including information about the plurality of objects;
matching each one of the identified plurality of objects from the surrounding environment image with a respective plurality of portions of the information about the plurality of objects included in the V2X data set;
selecting a first object in the identified plurality of objects, based on a user input;
obtaining information about a type of the first object by using an identification result determined based on the surrounding environment image and the received V2X data set; and
displaying a user interface (UI) for wireless communication connection with the first object, based on the information about the type of the first object and an object information UI representing information about at least one of profile information, wireless communication connection signal strength, or battery level of the first object.