# US 2024/0404287 A1 · Electronic device and method for providing service corresponding to selection of object in image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0404287 A1
- Document ID: US20240404287A1
- Status: Published application
- Kind: A1
- Application number: 18/799,647
- Filed: 2024-08-09
- Published: 2024-12-05
- Earliest claimed: 2019-02-19
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Sihyoung LEE; Kyungjae LEE; Daehee KIM; Taehwa HONG
- Web page: https://patentgrep.com/patent/US20240404287A1

## CPC

- G06V 20/50: Context or environment of the image
- G06F 16/583: using metadata automatically derived from the content
- G06F 16/953: Querying, e.g. by the use of web search engines
- G06Q 30/06: Buying, selling or leasing transactions
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06V 20/60: Type of objects
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

An electronic device and a method for providing a service corresponding to the selection of an object in an image are provided. The electronic device includes a camera, a display, a memory, and a processor operationally connected to the camera, the display, and the memory, wherein the processor can be configured to detect a plurality of object regions included in an image acquired using the camera, remove object regions having a size exceeding a threshold range from among the detected plurality of object regions, calculate the importance of objects within the threshold range, select a number of main objects that is no greater than the number of objects allowed by the electronic device, select auxiliary objects spatially related to the selected main objects, recognize the main objects and the auxiliary objects, recognize service attributes provided by the recognized main objects and auxiliary objects, determine services provided by the main objects and the auxiliary objects, and display the determined services on the display.

## Claims

### Claim 1

An electronic device comprising:
a camera;
a display;
a processor; and
memory storing instructions that, when executed by the processor, cause the electronic device to:
obtain an image using the camera,
display the obtained image on the display,
detect a plurality object regions included in the displayed image,
identify, based on importance of objects within the detected plurality of object regions, a main object,
recognize a plurality of services related to the identified main object,
determine, based on a user's selection, a service among the recognized plurality of services related to the main object, and
display detailed information corresponding to the determined service on the display.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to remove an object region having a size out of a predetermined threshold range from among the detected plurality object regions.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to calculate the importance of objects by using at least one of position information or size information of the object.

### Claim 4 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to:
identify an auxiliary object displayed within the identified main object;
recognize a plurality of services related to the identified auxiliary object;
determine, based on a user's selection, a service among the recognized plurality of services related to the auxiliary object; and
display detailed information corresponding to the determined service on the display.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the auxiliary object is an object having spatial relevance to the main object.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to determine the main object to be less than or equal to a number of objects allowed in the electronic device based on the importance of objects.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to:
classify objects in order of importance based on the importance of objects; and
determine the object with a highest importance as the main object.

### Claim 8

A method for a processor of an electronic device to provide a service corresponding to selection of an object in an image, the method comprising:
obtaining an image using the camera;
displaying the obtained image on the display;
detecting, by the processor, a plurality object regions included in the displayed image;
identifying, by the processor, based on importance of objects within the detected plurality of object regions, a main object;
recognizing, by the processor, a plurality of services related to the identified main object;
determining, by the processor, based on a user's selection, a service among the recognized plurality of services related to the main object; and
displaying, by the processor, detailed information corresponding to the determined service on the display.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
removing an object region having a size out of a predetermined threshold range from among the detected plurality object regions.

### Claim 10 (depends on 8)

The method of claim 8, further comprising:
calculating the importance of objects by using at least one of position information or size information of the object.

### Claim 11 (depends on 8)

The method of claim 8, further comprising:
identifying an auxiliary object displayed within the identified main object;
recognizing a plurality of services related to the identified auxiliary object;
determining, based on a user's selection, a service among the recognized plurality of services related to the auxiliary object; and
displaying detailed information corresponding to the determined service on the display.

### Claim 12 (depends on 11)

The method of claim 11, wherein the auxiliary object is an object having spatial relevance to the main object.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
determining the main object to be less than or equal to a number of objects allowed in the electronic device based on the importance of objects.

### Claim 14 (depends on 8)

The method of claim 8, further comprising:
classifying objects in order of importance based on the importance of objects; and
determining the object with a highest importance as the main object.

### Claim 15

A non-transitory computer-readable storage medium configured to store one or more computer programs including instructions that, when executed by a processor, cause an electronic device to:
obtain an image using the camera;
display the obtained image on the display;
detect a plurality object regions included in the displayed image;
identify, based on importance of objects within the detected plurality of object regions, a main object;
recognize a plurality of services related to the identified main object;
determine, based on a user's selection, a service among the recognized plurality of services related to the main object; and
display detailed information corresponding to the determined service on the display.
