# US 2019/0244270 A1 · Method, device and computer program for providing image search information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244270 A1
- Document ID: US20190244270A1
- Status: Published application
- Kind: A1
- Application number: 16/329,762
- Filed: 2017-08-24
- Published: 2019-08-08
- Earliest claimed: 2016-08-29
- Assignees: Odd Concepts Inc.
- Applicants: Odd Concepts Inc.
- Inventors: Jung Tae KIM; Kyung Mo KOO
- Web page: https://patentgrep.com/patent/US20190244270A1

## CPC

- G06Q 30/0625: {by formulating product or service queries, e.g. using keywords or predefined options}
- G06F 16/00: Information retrieval; Database structures therefor; File system structures therefor
- G06F 16/3331: Query processing
- G06F 16/338: Presentation of query results
- G06F 16/732: Query formulation
- G06F 16/735: Filtering based on additional data, e.g. user or group profiles
- G06F 16/738: Presentation of query results
- G06F 16/7837: {using objects detected or recognised in the video content}
- G06T 7/74: {involving reference images or patches}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present invention relates to a method, device and computer program for providing image search information. The method for an electronic device to provide image search information according to the present invention comprises: a step of generating a query by using an image if a predetermined query generation event occurs while playing the image; a step of transmitting the query to a server; and a step of receiving a search result corresponding to the query from the server and displaying same on a screen, wherein the search result is a search result of at least one object included in the image. A purpose of the invention is to provide a method, device and program for providing image search information, allowing the search of a product similar to a product exposed in an image, without an additional input of a search word.

## Claims

### Claim 1

A method of providing video search information by an electronic device, the method comprising:
when a preset query generation event is made during reproduction of a video, generating a query using the video;
transmitting the query to a server; and
receiving a search result corresponding to the query from the server and displaying the search result on a screen,
wherein the search result is a search result including product information or advertisement information of one or more objects included in the video.

### Claim 2 (depends on 1)

The method of claim 1, wherein the query includes at least one piece of object information extracted from a video frame displayed on a screen, an image in one area included in the video frame, a plurality of video frames which satisfy a preset condition based on the screen, or one frame of the video.

### Claim 3 (depends on 2)

The method of claim 2, wherein the object information includes at least one piece of image feature information extracted from the video or label information learned to correspond to a specific object.

### Claim 4 (depends on 1)

The method of claim 1, wherein the generating of the query comprises determining that the query generation event is made when a preset time passes or a search event is generated after a pause event is generated during the reproduction of the video.

### Claim 5 (depends on 1)

The method of claim 1, wherein the generating of the query comprises determining that the query generation event is made when a search event is generated during the reproduction of the video.

### Claim 6 (depends on 1)

The method of claim 1, wherein the generating of the query comprises:
capturing the video when a voice control mode activation command is received during the reproduction of the video; and
generating the query using the captured image when a search event is generated in a voice control mode.

### Claim 7 (depends on 1)

The method of claim 1, wherein the generating of the query comprises:
displaying a time remaining to generate the query on the screen when a pause event is generated during the reproduction of the video; and
determining that the query generation event is made when there is no user input before the remaining time reaches 0.

### Claim 8 (depends on 1)

The method of claim 1, wherein the generating of the query comprises:
extracting one or more objects from a video frame displayed on a paused screen; and
generating a query including the one or more objects.

### Claim 9 (depends on 8)

The method of claim 8, wherein the extracting of the objects comprises extracting one area including image feature information learned to correspond to a specific object through a pre-stored learning model as an object to be searched for and attaching a label corresponding to the image feature information to the object to be searched for.

### Claim 10 (depends on 8)

The method of claim 8, wherein the extracting of the objects comprises:
extracting image feature information from the video frame;
partitioning one area within the video frame according to a preset reference using the extracted image feature information; and
configuring the partitioned area as an object to be searched for.

### Claim 11 (depends on 1)

The method of claim 1, wherein the displaying of the search result on the screen comprises preferentially displaying, on the screen, a search result of an object including a label which is equal or similar to that of an object for which a user has searched using a search log of the user.

### Claim 12

(canceled)

### Claim 13

(canceled)

### Claim 14

A method of providing video search information to a client by a server, the method comprising:
generating a product database including image feature information of a predetermined product and a label corresponding to the image feature information of the product;
receiving a query generated using a video from the client;
analyzing the query;
when the query includes an object to be searched for, to which a label is attached, based on a result of the analysis of the query,
searching for product information corresponding to the label attached to the object to be searched for or image feature information of the object to be searched for in the product database; and
transmitting the search result to the client.

### Claim 15 (depends on 14)

The method of claim 14, further comprising, when the query includes at least one of a video frame, an image in one area included in the video frame, or a first object to which no label is attached, based on a result of the analysis of the query, extracting, as an object to be searched for, one area including image feature information pre-learned as a specific object from one of the image frame, the image, and the first object and attaching a label corresponding to the image feature information to the object to be searched for.

### Claim 16 (depends on 14)

The method of claim 14, further comprising receiving identification information of a user and a search log of the user from the client, wherein the transmitting of the search result comprises providing the user with product information corresponding to a label included in the search log through the identification information and the search log.

### Claim 17 (depends on 14)

The method of claim 14, further comprising, when the query includes image feature information learned as noise, based on a result of the analysis of the query, excluding the query from a target to be searched for.

### Claim 18

(canceled)

### Claim 19 (depends on 1)

A program for providing video search information stored in a computer-readable medium to perform the method of claim 1.
