# US 2020/0005042 A1 · Method and apparatus for providing information based on image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0005042 A1
- Document ID: US20200005042A1
- Status: Published application
- Kind: A1
- Application number: 16/567,331
- Filed: 2019-09-11
- Published: 2020-01-02
- Earliest claimed: 2019-08-06
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Eun Sang LEE; Hye Young KOO
- Web page: https://patentgrep.com/patent/US20200005042A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06V 20/20: in augmented reality scenes
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image-based information providing apparatus capable of executing a mounted artificial intelligence (AI) algorithm and/or machine learning algorithm and communicating with other electronic devices and an external server in a 5G communication environment is provided. The image-based information providing apparatus includes a camera, a display, and a processor. Therefore, it is possible to correctly recognize an object to be photographed, and provide various services for the convenience of a user.

## Claims

### Claim 1

An image-based information providing method comprising:
recognizing an object arranged in a photographing range of a camera;
obtaining specification information of the object;
displaying the specification information of the object together with the object based on augmented reality (AR);
obtaining search information based on the specification information of the object when a predetermined search command is inputted; and
providing the search information.

### Claim 2 (depends on 1)

The image-based information providing method of claim 1, further comprising obtaining item information of the object based on the specification information of the object.

### Claim 3 (depends on 2)

The image-based information providing method of claim 2, wherein the obtaining of the search information comprises obtaining information on one or more search items having similarity in a predetermined range to the item information of the object based on a previously stored learning model for inferring result data based on input data.

### Claim 4 (depends on 2)

The image-based information providing method of claim 2, wherein the obtaining of the search information comprises:
transmitting the item information of the object to an information providing system; and
obtaining information on one or more search items having similarity in a predetermined range to the item information of the object from the information providing system.

### Claim 5 (depends on 3)

The image-based information providing method of claim 3, wherein the providing of the search information comprises providing the information on the search item with similarity to the specification information of the object prioritized.

### Claim 6 (depends on 5)

The image-based information providing method of claim 5, wherein the providing of the information on the search item comprises:
setting similarity to the specification information of the object as a filter, and filtering information on a search item of which the similarity is outside a predetermined range; and
providing information on a search item in the predetermined range.

### Claim 7 (depends on 3)

The image-based information providing method of claim 3, wherein the providing of the search information comprises providing at least one of external appearance information of the item, function information of the item, shape information of the item, or a method of use of a configuration of the item, based on the item information of the object.

### Claim 8 (depends on 3)

The image-based information providing method of claim 3, wherein, when one or more search items of the same kind and having different specification information are searched, the providing of the search information comprises comparing data information on the one or more search items with each other, and providing the comparison result.

### Claim 9 (depends on 2)

The image-based information providing method of claim 2,
wherein, when only a partial region of the object is recognized, the obtaining of the specification information of the object comprises obtaining specification information of the partial region, and
wherein the obtaining of the search information comprises:
transmitting information on the partial region and the specification information of the partial region to an information providing system; and
obtaining information on a search item with similarity in a predetermined range to the information on the partial region and the specification information of the partial region.

### Claim 10 (depends on 9)

The image-based information providing method of claim 9, wherein the providing of the search information comprises displaying a partial region of the object, extracting a remaining region other than the partial region of the object from the information on the search item, and complementing the extracted region based on AR.

### Claim 11 (depends on 1)

The image-based information providing method of claim 1, wherein, when one or more objects are recognized in a photographing range of the camera, the recognizing of the object comprises selecting an object by a manipulation of a user or selecting an object that occupies the greatest part of a photographing region.

### Claim 12

An image-based information providing apparatus comprising:
an inputter including a camera;
a display; and
a processor configured to recognize an object arranged in a photographing range of the camera, and obtain specification information of the recognized object,
wherein the processor displays the specification information of the object on the display together with the object based on augmented reality (AR), obtains search information based on the object and the specification information when a predetermined search command is inputted through the inputter, and provides the obtained search information.

### Claim 13 (depends on 12)

The image-based information providing apparatus of claim 12, wherein the processor obtains item information of the object based on the specification information of the object.

### Claim 14 (depends on 13)

The image-based information providing apparatus of claim 13, further comprising a storage configured to store a learning model for inferring result data based on input data,
wherein the processor obtains information on one or more search items with similarity in a predetermined range to item information of the object, based on the learning model.

### Claim 15 (depends on 13)

The image-based information providing apparatus of claim 13, further comprising a communicator configured to communicate with an information providing system,
wherein the processor transmits the item information of the object to the information providing system through the communicator, and obtains information on one or more search items with similarity in a predetermined range to the item information of the object from the information providing system through the communicator.

### Claim 16 (depends on 14)

The image-based information providing apparatus of claim 14, wherein the processor provides, through the display, the information on the search item through the display with similarity to the specification information of the object prioritized.

### Claim 17 (depends on 16)

The image-based information providing apparatus of claim 16, wherein the processor sets similarity to the specification information of the object as a filter and filters information on a search item of which the similarity is outside a predetermined range, and provides information on a search item in the predetermined range.

### Claim 18 (depends on 14)

The image-based information providing apparatus of claim 14, wherein the processor provides, through the display, at least one of external appearance information of the item, function information of the item, shape information of the item, or a method of use of a configuration of the item, based on the item information of the object.

### Claim 19 (depends on 14)

The image-based information providing apparatus of claim 14, wherein, when one or more search items of the same kind and having different specification information are searched, the processor compares data information on the one or more search items with each other, and provides the comparison result through the display.

### Claim 20 (depends on 13)

The image-based information providing apparatus of claim 13, wherein, when only a partial region of the object is recognized, the processor obtains specification information of the partial region, transmits information on the partial region and the specification information of the partial region to an information providing system through the communicator, and obtains information on a search item with similarity in a predetermined range to the information on the partial region and the specification information of the partial region from the information providing system through the communicator.
