# US 10,484,673 B2 · Wearable device and method for providing augmented reality information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,673 B2
- Document ID: US10484673B2
- Status: Granted
- Kind: B2
- Application number: 14/725,396
- Filed: 2015-05-29
- Published: 2015-12-10
- Granted: 2019-11-19
- Earliest claimed: 2014-06-05
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd
- Inventors: Youngtae Jin
- Web page: https://patentgrep.com/patent/US10484673B2

## CPC

- H04N 13/339: using spatial multiplexing (H04N13/337 takes precedence)
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/03545: {Pens or stylus}
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 13/211: using temporal multiplexing
- H04N 13/293: Generating mixed stereoscopic images; Generating mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
- H04N 13/296: Synchronisation thereof; Control thereof
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An electronic device includes a camera module configured to capturing an image through a camera module of a wearable device, a transceiver configured to perform a functionally coupling between an electronic device and a wearable device, a display module, and a processor functionally coupled to the camera module and the display module, wherein the processor is configured to transmit the image to the electronic device to generate augmented reality using the captured image through the transceiver, receive the augmented reality on a display module of the wearable device through the transceiver, detect a user's hand gesture through the camera module, and control the display module to display a change of augmented reality information, based on the detected user's hand gesture.

## Claims

### Claim 1

A method for providing augmented reality, the method comprising:
generating augmented reality using an image captured through a camera module of a wearable device, the image comprising a plurality of real objects;
displaying the augmented reality on a display of the wearable device, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;
detecting at least one finger of a user within the augmented reality by the camera module;
displaying a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the finger is associated with the first window; and
in response to detecting that a finger of the user is bent to at least a predetermined angle, transitioning the first window into a second window different from the first window, displayed on the display of the wearable device, that provides detailed information regarding the real object corresponding to the virtual object.

### Claim 2 (depends on 1)

The method of claim 1, further comprising, when the bent finger is moved, displaying a further change of the augmented reality on the display.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
detecting at least one finger further comprises detecting at least two fingers of the user within the augmented reality, and
in response to detecting a gestures of the at least two fingers, performing at least one of:
displaying a reduced form of the virtual object,
displaying an enlarged form of the virtual object, or
displaying a moved form of the virtual object.

### Claim 4

A wearable electronic device comprising:
a camera module;
a display module; and
a processor configured to:
generate augmented reality using an image captured through the camera module;
display the augmented reality on the display module, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;
detect at least one finger of a user detected within the augmented reality by the camera module;
display a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the at least one finger is associated with the first window;
detect that a finger, of the at least one detected finger, of the user is bent to at least a predetermined angle; and
in response to detecting that the finger is bent to at least the predetermined angle, transitioning the first window into a second window different from the first window, displayed on the display, that provides detailed information regarding the real object corresponding to the virtual object.

### Claim 5 (depends on 4)

The wearable electronic device of claim 4, wherein the processor is further configured to, when the bent finger is moved, control the display module to display a further change of the augmented reality on the display.

### Claim 6 (depends on 4)

The wearable electronic device of claim 4, wherein to detect at least one finger of the user, the processor is further configured to:
detect at least two fingers of the user within the augmented reality, and
in response to detecting at least two fingers, control the display to display at least one of:
a reduced form of the virtual object,
an enlarged form of the virtual object, or
a moved form of the virtual object.

### Claim 7

A method for providing augmented reality information, the method comprising:
establishing a communication channel between an electronic device and a wearable device;
capturing, by the wearable device, an image comprising a plurality of real objects through a camera module of a wearable device;
transmitting, by the wearable device, the image to the electronic device to generate augmented reality using the captured image, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;
in response to detecting, by the wearable device, at least one finger of a user, transmitting, by the wearable device a location of each of the at least one finger to the electronic device to display a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the finger is associated with the first window;
receiving and displaying, by the wearable device, the augmented reality on a display module of the wearable device; and
in response to detecting that a finger, of the at least one finger, is being bent to at least a predetermined angle, transitioning the first window into a second window different from the first window, displayed on the display module, that provides detailed information regarding the real object corresponding to the virtual object,
wherein a hand gesture of the user comprises at least one of bending the finger or moving the finger.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
detecting at least one finger further comprises detecting at least two fingers of the user within the augmented reality, and
in response to detecting a gesture of the at least two fingers, performing at least one of:
displaying a reduced form of the virtual object,
displaying an enlarged form of the virtual object, or
displaying a moved form of the virtual object.

### Claim 9 (depends on 7)

The method of claim 7, wherein displaying the augmented reality includes displaying, transparently or semitransparently, each of the virtual objects as an overlay on each of the real objects, respectively.

### Claim 10

A wearable device comprising:
a camera module configured to capturing an image through a camera module of the wearable device;
a transceiver configured to establish a communication channel between an electronic device and the wearable device;
a display module; and
a processor configured to:
transmit the image to the electronic device to generate augmented reality using the captured image through the transceiver, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;
in response to detecting at least one finger of a user, transmit, through the transceiver, a location of each of the at least one finger to the electronic device to display a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the finger is associated with the first window;
receive the augmented reality on a display module of the wearable device through the transceiver; and
in response to detecting that a finger, of the at least one finger, is being bent to at least a predetermined angle, control the display module to transition the first window into a second window, different from the first window and displayed on the display module, that provides detailed information regarding the real object corresponding to the virtual object,
wherein a hand gesture of the user comprises at least one of bending the finger or moving the finger.

### Claim 11 (depends on 10)

The wearable device of claim 10, wherein to detect at least one finger of the user, the processor is further configured to:
detect at least two fingers of the user within the augmented reality, and
in response to detecting at least two fingers, control the display to display at least one of:
a reduced form of the virtual object,
an enlarged form of the virtual object, or
a moved form of the virtual object.

### Claim 12 (depends on 10)

The wearable device of claim 10, wherein the processor is further configured to control the display module to display, transparently or semitransparently, each of the virtual objects as an overlay over each of the real objects, respectively.

### Claim 13 (depends on 1)

The method of claim 1, wherein the detailed information comprises at least one of a text or an image describing the real object that is associated with the virtual object.

### Claim 14 (depends on 1)

The method of claim 1, wherein the virtual object is an icon.

### Claim 15 (depends on 1)

The method of claim 1, wherein displaying the first window comprises transposing the window on a location of the virtual object within the augmented reality.

### Claim 16 (depends on 1)

The method of claim 1, wherein displaying the first window comprises displaying the first window according to a location of the at least one finger within the augmented reality.