# US 2014/0198030 A1 · Image projection device, image projection system, and control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0198030 A1
- Document ID: US20140198030A1
- Status: Published application
- Kind: A1
- Application number: 14/133,853
- Filed: 2013-12-19
- Published: 2014-07-17
- Earliest claimed: 2013-01-16
- Assignees: Ricoh Company, Ltd.
- Inventors: Kazuhiro TAKAZAWA
- Web page: https://patentgrep.com/patent/US20140198030A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G03B 17/54: with projector
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image projection device for projecting an image on a projection object includes a detecting unit configured to detect a detection object present within an object space corresponding to the image projected by the image projection device; a recognizing unit configured to recognize an instruction motion of a user based on detection of the detection object by the detecting unit; and an output unit configured to generate an output corresponding to the instruction motion recognized within the object space by the recognizing unit from the instruction motion and the image projected by the image projection device.

## Claims

### Claim 1

An image projection device for projecting an image on a projection object, the image projection device comprising:
a detecting unit configured to detect a detection object present within an object space corresponding to the image projected by the image projection device;
a recognizing unit configured to recognize an instruction motion of a user based on detection of the detection object by the detecting unit; and
an output unit configured to generate an output corresponding to the instruction motion recognized within the object space by the recognizing unit from the instruction motion and the image projected by the image projection device.

### Claim 2 (depends on 1)

The image projection device according to claim 1, further comprising a deciding unit configured to decide, as an instruction region in which the instruction motion of the user is performed, a region within the object space, wherein
the region does not interfere with projection light projected from the image projection device, and faces a projection region indicating a region on which the image is projected within the projection object.

### Claim 3 (depends on 2)

The image projection device according to claim 2, further comprising:
an image sensor configured to capture an image of the object space that includes an upper edge of the projection region, the image sensor being disposed in a position different from a projection port of the image projection device from which the projection light is emitted;
a first calculating unit configured to calculate a distance between the image projection device and the projection object using a position of the projection region in data of the image captured by the image sensor; and
a second calculating unit configured to calculate a size of the projection region using the distance calculated by the first calculating unit, wherein
the detecting unit is configured to detect the detection object using the data of the image captured by the image sensor, and
the deciding unit is configured to decide a size of the instruction region using the size of the projection region calculated by the second calculating unit.

### Claim 4 (depends on 2)

The image projection device according to claim 2, wherein
the output unit includes
a generating unit configured to generate a UI image to be presented to the user according to the instruction motion performed within the instruction region;
a combining unit configured to combine an input video image and the UI image generated by the generating unit to generate a composite image; and
a projection unit configured to project the composite image generated by the combining unit on the projection object.

### Claim 5 (depends on 4)

The image projection device according to claim 4, wherein the generating unit is configured to generate the UI image according to a positional relation between the detection object and the projection region.

### Claim 6 (depends on 5)

The image projection device according to claim 5, wherein
the generating unit is configured to generate the UI image such that the UI image shows that a button is gradually pressed as the detection object approaches the projection region when an image of the button is displayed in a position corresponding to the detection object within the projection region, and
the generating unit is configured to generate the UI image such that the UI image shows completion of an operation when a distance between the detection object and the projection region becomes equal to or less than a threshold value.

### Claim 7 (depends on 5)

The image projection device according to claim 5, wherein
the generating unit is configured to generate the UI image such that the UI image shows that menus in a lower layer are gradually expanded as the detection object approach the projection region when an image of the menus overlapping with a layered structure is displayed in a position corresponding to the detection object within the projection region, and
the generating unit is configured to generate the UI image such that the UI image shows that all the menus with the layered structure are arranged in a single plane and an operation is completed when a distance between the detection object and the projection region becomes equal to or less than a threshold value.

### Claim 8 (depends on 5)

The image projection device according to claim 5, wherein
the generating unit is configured to change the UI image such that the UI image shows that an object displayed in a position corresponding to the detection object within the projection region gradually protrudes toward the user as the detection object approaches the projection region,
the generating unit is configured to change the UI image such that the object moves so as to follow movement of the detection object moving within a region in the instruction region in which a distance between the detection object and the projection region is equal to or less than the threshold value when the distance between the detection object and the projection region becomes equal to or less than a threshold value, and
the generating unit is configured to generate the UI image such that the UI image shows the object that has stopped following the movement of the detection object when the distance between the detection object and the projection region becomes larger than the threshold value.

### Claim 9 (depends on 5)

The image projection device according to claim 5, wherein
the generating unit is configured to generate the UI image such that the UI image shows that an object protrudes toward the user when the object is displayed in a position which is within the projection region and corresponds to the detection object and a distance between the detection object and the projection region is equal to or less than a threshold value,
the generating unit is configured to then generate the UI image such that the UI image shows that the object moves so as to follow movement of the detection object moving in a region within the instruction region in which the distance between the detection object and the projection region is larger than the threshold value, and
the generating unit is configured to generate the UI image such that the UI image shows that the object stops following the movement of the detection object and returns to a non-protruding state when the distance between the detection object and the projection region again becomes equal to or less than the threshold value.

### Claim 10 (depends on 1)

The image projection device according to claim 1, wherein a projection distance representing a distance between a projection port from which projection light is emitted from the image projection device and the projection object is set to a value smaller than a prescribed reference value.

### Claim 11

An image projection system, comprising:
an image projection device configured to project an image on a projection object; and
an information processing device connected to the image projection device, wherein
the information processing device includes
a detecting unit configured to detect a detection object present within an object space corresponding to the image projected by the image projection device;
a recognizing unit configured to recognize an instruction motion of a user based on detection of the detection object by the detecting unit; and
an output unit configured to generate an output corresponding to the instruction motion recognized within the object space by the recognizing unit from the instruction motion and the image projected by the image projection device.

### Claim 12 (depends on 11)

The image projection system according to claim 11, further comprising a deciding unit configured to decide, as an instruction region in which the instruction motion of the user is performed, a region within the object space, wherein
the region does not interfere with projection light projected from the image projection device, and faces a projection region indicating a region on which the image is projected within the projection object.

### Claim 13 (depends on 12)

The image projection system according to claim 12, further comprising:
an image sensor configured to capture an image of the object space that includes an upper edge of the projection region, the image sensor being disposed in a position different from a projection port of the image projection device from which the projection light is emitted;
a first calculating unit configured to calculate a distance between the image projection device and the projection object using a position of the projection region in data of the image captured by the image sensor; and
a second calculating unit configured to calculate a size of the projection region using the distance calculated by the first calculating unit, wherein
the detecting unit is configured to detect the detection object using the data of the image captured by the image sensor, and
the deciding unit is configured to decide a size of the instruction region using the size of the projection region calculated by the second calculating unit.

### Claim 14 (depends on 12)

The image projection system according to claim 12, wherein
the output unit includes
a generating unit configured to generate a UI image to be presented to the user according to the instruction motion performed within the instruction region;
a combining unit configured to combine an input video image and the UI image generated by the generating unit to generate a composite image; and
a projection unit configured to project the composite image generated by the combining unit on the projection object.

### Claim 15 (depends on 14)

The image projection system according to claim 14, wherein the generating unit is configured to generate the UI image according to a positional relation between the detection object and the projection region.

### Claim 16 (depends on 15)

The image projection system according to claim 15, wherein
the generating unit is configured to generate the UI image such that the UI image shows that a button is gradually pressed as the detection object approaches the projection region when an image of the button is displayed in a position corresponding to the detection object within the projection region, and
the generating unit is configured to generate the UI image such that the UI image shows completion of an operation when a distance between the detection object and the projection region becomes equal to or less than a threshold value.

### Claim 17 (depends on 15)

The image projection system according to claim 15, wherein
the generating unit is configured to generate the UI image such that the UI image shows that menus in a lower layer are gradually expanded as the detection object approach the projection region when an image of the menus overlapping with a layered structure is displayed in a position corresponding to the detection object within the projection region, and
the generating unit is configured to generate the UI image such that the UI image shows that all the menus with the layered structure are arranged in a single plane and an operation is completed when a distance between the detection object and the projection region becomes equal to or less than a threshold value.

### Claim 18 (depends on 15)

The image projection system according to claim 15, wherein
the generating unit is configured to change the UI image such that the UI image shows that an object displayed in a position corresponding to the detection object within the projection region gradually protrudes toward the user as the detection object approaches the projection region,
the generating unit is configured to change the UI image such that the object moves so as to follow movement of the detection object moving within a region in the instruction region in which a distance between the detection object and the projection region is equal to or less than the threshold value when the distance between the detection object and the projection region becomes equal to or less than a threshold value, and
the generating unit is configured to generate the UI image such that the UI image shows the object that has stopped following the movement of the detection object when the distance between the detection object and the projection region becomes larger than the threshold value.

### Claim 19 (depends on 15)

The image projection system according to claim 15, wherein
the generating unit is configured to generate the UI image such that the UI image shows that an object protrudes toward the user when the object is displayed in a position which is within the projection region and corresponds to the detection object and a distance between the detection object and the projection region is equal to or less than a threshold value,
the generating unit is configured to then generate the UI image such that the UI image shows that the object moves so as to follow movement of the detection object moving in a region within the instruction region in which the distance between the detection object and the projection region is larger than the threshold value, and
the generating unit is configured to generate the UI image such that the UI image shows that the object stops following the movement of the detection object and returns to a non-protruding state when the distance between the detection object and the projection region again becomes equal to or less than the threshold value.

### Claim 20

A control method, comprising:
detecting a detection object present within an object space corresponding to an image projected by an image projection device;
recognizing an instruction motion of a user based on detection of the detection object at the detecting; and
generating an output corresponding to the instruction motion recognized within the object space at the recognizing from the instruction motion and the image projected by the image projection device.