# US 2008/0082208 A1 · Method, apparatus, and medium for controlling mobile device based on image of real space including the mobile device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0082208 A1
- Document ID: US20080082208A1
- Status: Published application
- Kind: A1
- Application number: 11/802,907
- Filed: 2007-05-25
- Published: 2008-04-03
- Earliest claimed: 2006-09-29
- Assignees: Samsung Electronics Co., Ltd.
- Inventors: Young-jin Hong; Ki-wan Choi; Yong-beom Lee; Sang-goog Lee; Hyoung-ki Lee
- Web page: https://patentgrep.com/patent/US20080082208A1

## CPC

- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0033: {by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES

## Abstract

Provided is a method, apparatus, and medium for controlling a mobile device based on the image of a real space including the mobile device, more particularly, a method and apparatus for moving a mobile device based on the image of a real space photographed by a remote control of a user. The apparatus includes an image acquisition unit acquiring the image of the real space including the mobile device using an image input unit of a remote control; an image output unit outputting the acquired image on an output unit of the remote control; a position detection unit recognizing the mobile device from the output image and detecting a position of the mobile device using the remote control; a target position detection unit converting a target position of the mobile device, which is selected by a user on the output unit, into a target position in the real space; and a movement control unit moving the mobile device to the target position in the real space.

## Claims

### Claim 1

A method of controlling a mobile device based on the image of a real space including the mobile device, the method comprising:
(a) acquiring the image of the real space including the mobile device using an image input unit of a remote control;
(b) outputting the acquired image on an output unit of the remote control;
(c) recognizing the mobile device from the output image and detecting a position of the mobile device using the remote control;
(d) selecting a target position of the mobile device on the output unit and converting the selected target position into a target position in the real space; and
(e) moving the mobile device to the target position in the real space.

### Claim 2 (depends on 1)

The method of claim 1, further comprising performing a function of the mobile device after moving the mobile device to the target position in the real space.

### Claim 3 (depends on 1)

The method of claim 1, wherein the image input unit is a camera which can photograph an image.

### Claim 4 (depends on 1)

The method of claim 1, wherein the output unit is a touch screen outputting an image photographed by the image input unit and receiving an input from the user or a display comprising a character input unit, by which the user can input coordinates, on a screen of the display.

### Claim 5 (depends on 1)

The method of claim 1, wherein (c) comprises recognizing the mobile device by recognizing a marker which is included in the mobile device and which has a known size and shape.

### Claim 6 (depends on 5)

The method of claim 5, wherein (c) further comprises sensing transformation of the marker detected from the output image, calculating a position of the marker in the real space, and thus detecting the position of the marker.

### Claim 7 (depends on 1)

The method of claim 1, wherein (d) comprises selecting the target position of the mobile device by touching the touch screen or inputting screen coordinates of the target position.

### Claim 8 (depends on 7)

The method of claim 7, wherein (d) further comprises perceiving the transformation of the marker attached to the mobile device from the acquired image in order to convert the target position selected from the acquired image into the target position in the real space and thus detecting the target position of the mobile device in the real space.

### Claim 9 (depends on 1)

The method of claim 1, wherein (e) comprises moving the mobile device to the target position in the real space by rotating the mobile device and driving the mobile device straight ahead using a driving unit.

### Claim 10

A method of controlling a mobile device based on the image of a real space including the mobile device, the method comprising:
(a) acquiring the image of the real space including the mobile device using an image input unit of a remote control;
(b) outputting the acquired image on an output unit of the remote control;
(c) recognizing the mobile device from the output image and detecting a position of the mobile device using the remote control;
(d) sequentially selecting a plurality of target positions of the mobile device on the output unit and converting the selected target positions into target positions in the real space; and
(e) sequentially moving the mobile device to the target positions in the real space.

### Claim 11

An apparatus for controlling a mobile device based on the image of a real space including the mobile device, the apparatus comprising:
an image acquisition unit to acquire the image of the real space including the mobile device using an image input unit of a remote control;
an image output unit to output the acquired image on an output unit of the remote control;
a position detection unit to recognize the mobile device from the output image and to detect a position of the mobile device using the remote control;
a target position detection unit to convert a target position of the mobile device, which is selected by a user on the output unit, into a target position in the real space; and
a movement control unit to move the mobile device to the target position in the real space.

### Claim 12 (depends on 11)

The apparatus of claim 11, further comprising a function execution unit to perform a function of the mobile device after moving the mobile device to the target position in the real space.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein the image input unit is a camera to photograph an image.

### Claim 14 (depends on 11)

The apparatus of claim 11, wherein the output unit is a touch screen to output an image photographed by the image input unit and to receive an input from the user or a display comprising a character input unit, by which the user can input coordinates, on a screen of the display.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the position detection unit recognizes the mobile device by recognizing a marker which is included in the mobile device and which has a known size and shape.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the position detection unit further comprises a position calculation unit to perceive transformation of the marker detected from the output image, to calculate a position of the marker in the real space, and to detect the position of the marker.

### Claim 17 (depends on 11)

The apparatus of claim 11, wherein the target position detection unit comprises a target position input unit to allow the user to input the target position of the mobile device by touching the touch screen or by inputting screen coordinates of the target position.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the target position detection unit further comprises a target position conversion unit to recognize the transformation of the marker attached to the mobile device from the acquired image in order to convert the target position selected from the acquired image into the target position in the real space and to detect the target position of the mobile device in the real space.

### Claim 19 (depends on 11)

The apparatus of claim 11, wherein the movement control unit moves the mobile device to the target position in the real space by rotating the mobile device and driving the mobile device straight ahead using a driving unit.

### Claim 20

An apparatus for controlling a mobile device based on the image of a real space including the mobile device, the apparatus comprising:
an image acquisition unit to acquire the image of the real space including the mobile device using an image input unit of a remote control;
an image output unit to output the acquired image on an output unit of the remote control;
a position detection unit to recognize the mobile device from the output image and to detect a position of the mobile device using the remote control;
a target position detection unit to convert a plurality of target positions of the mobile device, which are selected by a user on the output unit, into target positions in the real space; and
a movement control unit to sequentially move the mobile device to the target positions in the real space.

### Claim 21 (depends on 1)

At least one computer readable medium storing computer readable instructions that control at least one processor to implement the method of claim 1.

### Claim 22 (depends on 10)

At least one computer readable medium storing computer readable instructions that control at least one processor to implement the method of claim 10.

### Claim 23

A remote control for controlling a mobile device, comprising:
an image acquisition unit to acquire an image of a real space including the mobile device;
an image output unit to display the acquired image;
a position detection unit to recognize the mobile device in the real space and to detect a position of the mobile device; and
a target position detection unit to convert a target position of the mobile device, which is selected by a user on the output unit, into a target position in the real space in order to supply the target position in the real space to the mobile device.

### Claim 24

A remote control for controlling a mobile device, comprising:
an image acquisition unit to acquire an image of a real space including the mobile device;
an image output unit to display the acquired image;
a position detection unit to recognize the mobile device in the real space and to detect a position of the mobile device; and
a target position detection unit to convert a plurality of target positions of the mobile device, which are selected by a user on the output unit, into target positions in the real space in order to supply the target position in the real space to the mobile device for sequential movement of the mobile device.

### Claim 25

A method of controlling a mobile device based on the image of a real space including the mobile device, the method comprising:
(a) acquiring the image of the real space including the mobile device using an image input unit of a remote control;
(b) outputting the acquired image on an output unit of the remote control;
(c) recognizing the mobile device from the output image and detecting a position of the mobile device using the remote control;
(d) selecting a target position of the mobile device on the output unit and converting the selected target position into a target position in the real space; and
(e) outputting the target position in real space to the mobile device.

### Claim 26 (depends on 25)

At least one computer readable medium storing computer readable instructions that control at least one processor to implement the method of claim 25.

### Claim 27

A method of controlling a mobile device based on the image of a real space including the mobile device, the method comprising:
(a) acquiring the image of the real space including the mobile device using an image input unit of a remote control;
(b) outputting the acquired image on an output unit of the remote control;
(c) recognizing the mobile device from the output image and detecting a position of the mobile device using the remote control; and
(d) sequentially selecting a plurality of target positions of the mobile device on the output unit and converting the selected target positions into target positions in the real space; and
(e) outputting the target positions in real space to the mobile device.

### Claim 28 (depends on 27)

At least one computer readable medium storing computer readable instructions that control at least one processor to implement the method of claim 27.
