# US 11,443,507 B2 · Electronic device for identifying external object and operating method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,507 B2
- Document ID: US11443507B2
- Status: Granted
- Kind: B2
- Application number: 17/011,246
- Filed: 2020-09-03
- Published: 2021-03-11
- Granted: 2022-09-13
- Earliest claimed: 2019-09-05
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Sangmin Park; Jaeyung Yeo
- Web page: https://patentgrep.com/patent/US11443507B2

## CPC

- G06Q 20/208: {Input by product or record sensing, e.g. weighing or scanner processing}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/24: Classification techniques
- G06V 10/764: using classification, e.g. of video objects
- G06V 2201/10: Recognition assisted with metadata
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An electronic device for identifying an external object, based on related information and a method of operating the same are provided. The electronic device includes a memory configured to store an object list and object image learning data, and at least one processor. The at least one processor may be configured to acquire an image including a specific object, update the object list, based on user purchase history information, collect image data relating to the object, based on the updated object list to update the object image learning data, learn an object classification model, based on the updated object list and the updated object image learning data, and identify the specific object by the learned object classification model.

## Claims

### Claim 1

An electronic device comprising:
a memory configured to store an object list, object image learning data used to identify an object in an image, and user purchase history information comprising product-related information on products according to at least one purchase history of at least one user; and
at least one processor operatively connected to the memory,
wherein the at least one processor is configured to:
acquire an image including a specific object,
update the object list by adding at least one object to the object list, wherein the at least one object is identified based on the user purchase history information,
collect image information on the specific object, based on the updated object list,
update the object image learning data,
learn an object classification model, based on the updated object list and the updated object image learning data, and
identify the specific object by the learned object classification model.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to acquire the user purchase history information from product-related information generated by purchasing or searching for a specific product.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to acquire the user purchase history information from an application for purchasing or searching for a specific product.

### Claim 4 (depends on 1)

The electronic device of claim 1, further comprising:
a communication circuit operatively connected to the at least one processor,
wherein the at least one processor is further configured to acquire image data through the communication circuit from a site related to purchasing or searching for a specific product based on the user purchase history information.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the at least one processor is further configured to acquire similar image data of a product similar to the specific product as the image data through the communication circuit, based on information generated by searching for or purchasing of the specific product.

### Claim 6 (depends on 4)

The electronic device of claim 4, wherein the at least one processor is further configured to:
acquire object-related management information, based on information generated by searching for or purchasing of the specific product, and
provide object-related management service, based on the object-related management information.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to:
attempt to identify the specific object, based on the object list,
update the object list based on the attempt failing,
primarily learn the object classification model, based on the updated object list, and
identify the specific object by the primarily learned object classification model.

### Claim 8 (depends on 7)

The electronic device of claim 7, further comprising:
a communication circuit operatively connected to the at least one processor,
wherein based on the identification of the specific object by the primarily learned object classification model failing, the at least one processor is further configured to:
acquire image data relating to the specific object from an external device through the communication circuit, based on the user purchase history information, and
perform secondarily learning on the primary learned object classification model using the updated object image learning data by adding the acquired image data.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the at least one processor is further configured to:
identify the specific object by a secondarily learned object classification model, and
based on the identification of the specific object by the secondarily learned object classification model failing, obtain information related to the specific object from a user.

### Claim 10 (depends on 1)

The electronic device of claim 1, wherein the at least one processor is further configured to:
acquire place information where the specific object is located, and
identify the specific object in the object list by limiting to the object list based on the place information.

### Claim 11

A method of operating an electronic device, the method comprising:
acquiring an image including a specific object;
updating an object list by adding at least one object to the object list, the at least one object being identified based on user purchase history information, wherein the user purchase history information comprises product-related information on products according to at least one purchase history of at least one user;
collecting image information on the specific object, based on the updated object list;
updating object image learning data, wherein the object image learning data is used to identify an object in an image;
learning an object classification model, based on the updated object list and the updated object image learning data; and
identifying the specific object by the learned object classification model.

### Claim 12 (depends on 11)

The method of claim 11, wherein the user purchase history information is acquired from information generated by purchasing or searching for a specific product.

### Claim 13 (depends on 11)

The method of claim 11, wherein the user purchase history information is acquired from an application for purchasing or searching for a specific product.

### Claim 14 (depends on 11)

The method of claim 11, wherein image data is acquired from a site related to purchasing or searching for a specific product, based on the user purchase history information.

### Claim 15 (depends on 14)

The method of claim 14, wherein the image data includes similar image data of a product similar to the specific product acquired based on information generated by searching for or purchasing of the specific product.

### Claim 16 (depends on 14)

The method of claim 14, further comprising:
acquiring object-related management information based on information generated by searching for or purchasing of the specific product; and
providing object-related management service, based on the object-related management information.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
attempting to identify the specific object, based on the object list,
wherein the updating the object list is performed based on the attempt failing,
wherein the learning of the object classification model comprises primarily learning the object classification model, based on the updated object list, and
wherein the identifying of the specific object further comprises identifying the specific object by a primarily learned object classification model.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
acquiring image data relating to the specific object from an external device through a communication circuit, based on the user purchase history information, and based on the identification of the specific object by the primary learned object classification model failing,
wherein the learning of the object classification model further comprises performing secondarily learning on the primary learned object classification model using the updated object image learning data by adding the acquired image data.

### Claim 19 (depends on 18)

The method of claim 18,
wherein the identifying of the specific object comprises identifying the specific object by the a secondarily learned object classification model, and
wherein based on the identification of the specific object by the secondarily learned object classification model failing, the method further comprising:
obtaining information related to the specific object from a user.

### Claim 20 (depends on 11)

The method of claim 11, further comprising:
acquiring place information where the specific object is located,
wherein the specific object in the object list is identified by limiting the object list based on the place information.