# US 2011/0254810 A1 · User interface device and method for recognizing user interaction using same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0254810 A1
- Document ID: US20110254810A1
- Status: Published application
- Kind: A1
- Application number: 13/087,897
- Filed: 2011-04-15
- Published: 2011-10-20
- Earliest claimed: 2010-04-15
- Assignees: Electronics and Telecommunications Research Institute
- Inventors: Dong Woo LEE; Yong Ki SON; Baesun KIM; IL YEON CHO; Hyun Tae JEONG; Jeong Mook LIM; Hee Sook SHIN
- Web page: https://patentgrep.com/patent/US20110254810A1

## CPC

- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0386: {for light pen}
- H04N 9/3173: {wherein the projection device is specially adapted for enhanced portability}
- H04N 9/3182: {Colour adjustment, e.g. white balance, shading or gamut (white balance per se H04N9/73; control of amplitude of colour signals H04N9/68; colour control circuits for displays in general G09G5/02, G09G3/2003, G09G3/3607)}
- H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- H04N 9/3194: {including sensor feedback}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A user interface device includes a frame replacement unit configured to replace a frame of an input image signal by a pattern frame at a frame time; a projector module configured to project an image of the image signal with the pattern frame onto a target; an image acquisition unit configured to capture a pattern image of the pattern frame from the image projected onto the target; and an image recognition unit configured to recognize a user interaction by using the captured pattern image.

## Claims

### Claim 1

A user interface device comprising:
a frame replacement unit configured to replace a frame of an input image signal by a pattern frame at a frame time;
a projector module configured to project an image of the image signal with the pattern frame onto a target;
an image acquisition unit configured to capture a pattern image of the pattern frame from the image projected onto the target; and
an image recognition unit configured to recognize a user interaction by using the captured pattern image.

### Claim 2 (depends on 1)

The user interface device of claim 1, wherein the pattern image includes a general-purpose image in unicolor and color and a constructed light in unicolor and color.

### Claim 3 (depends on 1)

The user interface device of claim 1, wherein the pattern image includes an infrared image.

### Claim 4 (depends on 1)

The user interface device of claim 1, wherein the pattern image has a grid pattern.

### Claim 5 (depends on 4)

The user interface device of claim 4, wherein the image recognition unit detects the change in the grid pattern of the pattern image caused by a motion event on the image projected onto the target to recognize the user interaction.

### Claim 6 (depends on 1)

The user interface device of claim 1, wherein the image recognition unit is further configured to detect a focus of the pattern image, and
the user interface device further comprises an optical controller configured to control the projector module to correct the focus of the projector module depending on the detected focus.

### Claim 7 (depends on 1)

The user interface device of claim 1, wherein the image recognition unit is further configured to detect brightness, color and distortion of the pattern image, and
the user interface device further comprises an image control unit configured to correct the brightness, color and distortion of the image to be projected onto the target depending on the detected brightness, color and distortion.

### Claim 8

A method for recognizing a user interaction, the method comprising:
replacing a frame of an input image signal by a pattern frame at a frame time;
projecting an image of the image signal with the pattern frame onto a target;
capturing a pattern image of the pattern frame from the image projected onto the target; and
recognizing the user interaction by using the captured pattern image.

### Claim 9 (depends on 8)

The method of claim 8, wherein the pattern image includes a general-purpose image in unicolor and color and a structured light in unicolor and color.

### Claim 10 (depends on 8)

The method of claim 8, wherein the pattern image includes an infrared image.

### Claim 11 (depends on 8)

The method of claim 8, wherein the pattern image has a grid pattern.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
detecting a focus of the pattern image; and
controlling said projecting the pattern image to correct the focus of the pattern image.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
detecting the brightness, color and distortion of the pattern image, and
controlling said projecting an image of the image signal with the pattern image to correct the brightness, color and distortion of the image to be projected onto the target.

### Claim 14 (depends on 11)

The method of claim 11, wherein said recognizing the user interaction comprises detecting the change in the grid pattern of the pattern image caused by a motion event on the image projected on the target.

### Claim 15

A user interface device comprising:
a projector module configured to project an image onto a target;
a pattern image generator configured to generate a laser beam having a pattern through diffraction to project a pattern image thereof onto the target;
an image acquisition unit, in synchronization with the projection of the pattern image, configured to capture the pattern image projected onto the target on which the image is projected; and
an image recognition unit configured to recognize a user interaction by using the captured pattern image.

### Claim 16 (depends on 15)

The user interface device of claim 15, wherein the pattern image generator includes a laser module to project the laser beam and a diffraction grating to pass the laser beam therethrough.

### Claim 17 (depends on 15)

The user interface device of claim 15, wherein the image recognition unit detects the change in the pattern of the pattern image caused by a motion event to recognize the user interaction.

### Claim 18 (depends on 15)

The user interface device of claim 15, wherein the image recognition unit is further configured to detect a focus of the pattern image, and
the user interface device further comprises an optical controller configured to control the projector module to correct the focus of the pattern image depending on the detected focus.

### Claim 19 (depends on 15)

The user interface device of claim 15, wherein the image recognition unit is further configured to detect brightness, color and distortion of the pattern image, and
the user interface device further comprises an image correction unit configured to correct the brightness, color and distortion of the image projected onto the target depending on the detected brightness, color and distortion.
