# US 2012/0075345 A1 · Method, terminal and computer-readable recording medium for performing visual search based on movement or position of terminal

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0075345 A1
- Document ID: US20120075345A1
- Status: Published application
- Kind: A1
- Application number: 13/375,215
- Filed: 2010-09-06
- Published: 2012-03-29
- Earliest claimed: 2009-10-01
- Assignees: Olaworks, Inc.
- Inventors: Kyoung Suk Lee; Young Il Choi; Chan Jin Ju; Jung Hee Ryu
- Web page: https://patentgrep.com/patent/US20120075345A1

## CPC

- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 16/583: using metadata automatically derived from the content
- G06F 16/904: Browsing; Visualisation therefor (for navigating the web G06F16/954; browsing optimisation for the web G06F16/957)
- G06F 2200/1637: Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present invention includes a method for performing visual search based on a movement and/or an angular position of a terminal. The method includes the steps of: (a) sensing a movement and/or an angular position of a terminal by using at least one of sensors; (b) determining whether a triggering event occurs or not by referring to at least one of the sensed movement and the sensed angular position of the terminal; and (c) if the triggering event occurs, allowing visual search to be performed for at least one of objects included in an output image displayed on the terminal at the time of the occurrence of the triggering event; wherein the output image is generated in a form of augmented reality by combining an image inputted through the terminal in real time with information relevant thereto.

## Claims

### Claim 1

A method for performing visual search based on a movement and/or an angular position of a terminal comprising the steps of:
(a) sensing a movement and/or an angular position of a terminal by using at least one sensor;
(b) determining whether a triggering event has occurred by referring to at least one of the sensed movement and the sensed angular position of the terminal; and
(c) where the triggering event has occurred, allowing a visual search to be performed for at least an object included in an output image displayed on the terminal at the time of the occurrence of the triggering event;
wherein the output image is generated in a form of augmented reality by combining an image inputted through the terminal in real time with information relevant thereto.

### Claim 2 (depends on 1)

The method of claim 1, wherein the triggering event includes at least one of the following events: a first event of the terminal moving by a prefixed pattern, a second event of the terminal stopping moving for a predefined period of time, and a third event of the terminal taking a pre-established angular position.

### Claim 3 (depends on 2)

The method of claim 2, wherein the prefixed pattern is specified by at least one of the following factors: the terminal's moving distance, velocity, acceleration and moving direction.

### Claim 4 (depends on 2)

The method of claim 2, wherein the pre-established angular position is specified by an angle at which the terminal is tilted to at least one of axes.

### Claim 5 (depends on 1)

The method of claim 1, wherein, at the step (c), the at least an object includes a top n objects displayed on the output image closer to the center thereof.

### Claim 6 (depends on 1)

The method of claim 1, wherein, at the step (c), the visual search is performed by a remote operation equipment communicable with the terminal.

### Claim 7 (depends on 1)

The method of claim 1, further comprising the step of: (d) providing information on at least one of the objects obtained as a result of performing the visual search.

### Claim 8 (depends on 7)

The method of claim 7, wherein, at the step of (d), the image inputted by the terminal overlaid with the information on at least one of the objects is formed and provided in a form of augmented reality.

### Claim 9

A terminal for performing visual search based on a movement and/or angular position thereof comprising:
a movement and/or angular position sensing part for sensing information on a movement and/or angular position thereof by using at least one sensor;
a triggering event identifying part for determining when a triggering event has occurred by referring to at least one of the sensed movement and the sensed angular position thereof; and
a visual search part for performing a visual search for at least an object included in an output image displayed thereon when the triggering event occurs;
wherein the output image is generated in a form of augmented reality by combining an image inputted therethrough in real time with information relevant thereto.

### Claim 10 (depends on 9)

The terminal of claim 9, wherein the triggering event includes at least one of the following events: a first event of moving by a prefixed pattern, a second one of stopping moving for a predefined period of time, and a third one of taking a pre-established angular position.

### Claim 11 (depends on 10)

The terminal of claim 10, wherein the prefixed pattern is specified by at least one of the following factors: moving distance, velocity, acceleration and moving direction thereof.

### Claim 12 (depends on 10)

The terminal of claim 10, wherein the pre-established angular position is specified by an angle at which it is tilted to at least one of axes.

### Claim 13 (depends on 9)

The terminal of claim 9, wherein the at least an object includes a top n objects displayed on the output image closer to the center thereof.

### Claim 14 (depends on 9)

The terminal of claim 9, wherein the visual search is performed by remote operation equipment communicable therewith.

### Claim 15 (depends on 9)

The terminal of claim 9, wherein an output image generating part for providing information on at least one of the objects obtained as a result of performing the visual search is further included.

### Claim 16 (depends on 15)

The terminal of claim 15, wherein the output image generating part forms and provides the image inputted thereby overlaid with the information on at least one of object(s) in a form of augmented reality.

### Claim 17 (depends on 1)

The method of claim 1, wherein, when a key is inputted at a first angular position of the terminal as a first triggering event, a retrieval of information around a location of the terminal is performed at the time of the occurrence of the first triggering event; and when another key is inputted at a second angular position of the terminal as a second triggering event, said visual search for said at least an object included on the output image displayed through the terminal is performed at the time of the occurrence of the second triggering event.

### Claim 18 (depends on 7)

The method of claim 7, wherein the step (d) includes the steps of:
(d1) sensing the movement and/or the angular position of the terminal by using said at least one sensor;
(d2) determining whether a triggering event has occurred by referring to at least one of the sensed movement of the terminal and the sensed angular position thereof; and
(d3) when the triggering event for controlling the output image occurs, changing at least one of a method of displaying information on objects and an information type.

### Claim 19 (depends on 9)

The terminal of claim 9, wherein the triggering event identifying part determines the input of a key at its first angular position as a first triggering event and the input of another key at its second angular position as a second triggering event; and
wherein the visual search part performs a retrieval of information around the location of the terminal at the time of the occurrence of the first triggering event and then performs the visual search for said at least an object included on the output image displayed through the terminal at the time of the occurrence of the second triggering event.

### Claim 20 (depends on 15)

The terminal of claim 15, wherein the output image generating part senses its movement and/or angular position by using said at least one sensor, determines whether the triggering event has occurred by referring to at least one of the sensed movement and the sensed angular position, and changes at least one of a method of displaying information on objects and an information type, if the triggering event for controlling the output image occurs.

### Claim 21 (depends on 1)

A medium recording a computer readable program to execute the method of claim 1.
