# US 2018/0018825 A1 · Augmented Reality Device and Operation Thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0018825 A1
- Document ID: US20180018825A1
- Status: Published application
- Kind: A1
- Application number: 15/590,920
- Filed: 2017-05-09
- Published: 2018-01-18
- Earliest claimed: 2016-07-15
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics, Co. Ltd.
- Inventors: Jihee KIM; Sung Ho PARK; Simon GIBBS; Anthony LIOT; Arnaud RENEVIER; Lu LUO
- Web page: https://patentgrep.com/patent/US20180018825A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/017: {Head mounted}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A device may include a camera configured to capture a live image data comprising an image of an object, and a processor coupled to the camera. The processor may be configured to recognize content of a selected portion of the live image data based on a contextual information relevant to the object using computer vision technology, and generate a visual information based on the recognized content. The device may also include an interface circuitry coupled to the processor. The interface circuitry may be configured to present the live image data, and overlay the live image data with the visual information.

## Claims

### Claim 1

A device, comprising:
a camera configured to capture a live image data comprising an image of an object;
a processor coupled to the camera and configured to:
recognize content of a selected portion of the live image data based on a contextual information relevant to the object, using computer vision technology, and
generate a visual information based on the recognized content; and
an interface circuitry coupled to the processor and configured to:
present the live image data, and
overlay the live image data with the visual information.

### Claim 2 (depends on 1)

The device of claim 1, wherein the processor is further configured to:
retrieve a digital representation associated with the object based on the contextual information; and
recognize the content of the selected portion by correlating the selected portion with the digital representation.

### Claim 3 (depends on 1)

The device of claim 1, wherein the processor is further configured to:
determine a first service associated with the recognized content;
retrieve additional information related to the recognized content from the first service; and
generate the visual information based on the additional information.

### Claim 4 (depends on 1)

The device of claim 1, wherein the processor is further configured to:
retrieve personal information of a user of the device; and
generate the visual information based on the personal information.

### Claim 5 (depends on 1)

The device of claim 1, wherein the processor is further configured to:
detect an interaction with the visual information;
activate a second service in response to the interaction; and
presenting a user interface of the second service.

### Claim 6 (depends on 1)

The device of claim 1, wherein the contextual information includes at least one of geo location of the object, time, crowd-sourced data, or a combination thereof.

### Claim 7

A method of image processing, comprising:
presenting, by a device, a live image data captured by a camera coupled to the device, wherein the live image data comprises an image of an object;
recognizing, by the device and using computer vision technology, content of a selected portion of the live image data based on a contextual information relevant to the object;
generating, by the device, a visual information based on the recognized content; and
overlaying, by the device, the live image data with the visual information.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
retrieving a digital representation associated with the object based on the contextual information; and
recognizing the content of the selected portion by correlating the selected portion with the digital representation.

### Claim 9 (depends on 7)

The method of claim 7, further comprising:
determining a first service associated with the recognized content;
retrieving additional information related to the recognized content from the first service ; and
generating the visual information based on the additional information.

### Claim 10 (depends on 7)

The method of claim 7, further comprising:
retrieving personal information of a user of the device; and
generating the visual information based on the personal information.

### Claim 11 (depends on 7)

The method of claim 7, further comprising:
detecting an interaction with the visual information;
activating a second service in response to the interaction; and
presenting a user interface of the second service.

### Claim 12 (depends on 7)

The method of claim 7, wherein the contextual information includes at least one of geo location of the object, time, crowd-sourced data, or a combination thereof.

### Claim 13 (depends on 7)

The method of claim 7, wherein the object comprises an eatery menu, a page of a book, a ticket, and a poster.

### Claim 14 (depends on 7)

The method of claim 7, wherein the visual information is formulated based on at least one of a rule, a presentation style template, guidance provided by a 3rd party, learnt preference of a user of the device, or a combination thereof.

### Claim 15

A non-transitory computer readable storage medium including computer readable program instructions that, when executed by a processor, cause the processor to perform a method comprising:
presenting, by a device, a live image data captured by a camera coupled to the device, wherein the live image data comprises an image of an object;
recognizing, by the device and using computer vision technology, content of a selected portion of the live image data based on a contextual information relevant to the object;
generating, by the device, a visual information based on the recognized content; and
overlaying, by the device, the live image data with the visual information.

### Claim 16 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the method further comprises:
retrieving a digital representation associated with the object based on the contextual information; and
recognizing the content of the selected portion by correlating the selected portion with the digital representation.

### Claim 17 (depends on 15)

The non-transitory computer readable storage medium of claim 15, the method further comprises:
determining a first service associated with the recognized content;
retrieving additional information related to the recognized content from the first service; and
generating the visual information based on the additional information.

### Claim 18 (depends on 15)

The non-transitory computer readable storage medium of claim 15, the method further comprises:
retrieving personal information of a user of the device; and
generating the visual information based on the personal information.

### Claim 19 (depends on 15)

The non-transitory computer readable storage medium of claim 15, the method further comprises:
detecting an interaction with the visual information;
activating a second service in response to the interaction; and
presenting a user interface of the second service.

### Claim 20 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the contextual information includes at least one of geo location of the object, time, crowd-sourced data, or a combination thereof.
