# US 2024/0004438 A1 · Enhanced field of view to augment three-dimensional (3d) sensory space for free-space gesture interpretation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004438 A1
- Document ID: US20240004438A1
- Status: Published application
- Kind: A1
- Application number: 18/370,338
- Filed: 2023-09-19
- Published: 2024-01-04
- Earliest claimed: 2013-10-03
- Assignees: Ultrahaptics IP Two Limited
- Applicants: Ultrahaptics IP Two Limited
- Inventors: David S. HOLZ; Paul DURDIK
- Web page: https://patentgrep.com/patent/US20240004438A1

## CPC

- G06F 1/1686: {the I/O peripheral being an integrated camera}
- G02B 5/04: Prisms
- G02B 5/045: {Prism arrays}
- G02B 7/1805: {for prisms (G02B7/181 takes precedence)}
- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- H04N 23/67: Focus control based on electronic image sensor signals
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06F 1/1629: {Protective covers or auxiliary enclosures for portable computers (for carrying with peripheral devices G06F1/1628; for storing A45C11/003)}
- G06F 2200/1633: Protecting arrangement for the entire housing of the computer
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The technology disclosed relates to enhancing the fields of view of one or more cameras of a gesture recognition system for augmenting the three-dimensional (3D) sensory space of the gesture recognition system. The augmented 3D sensory space allows for inclusion of previously uncaptured of regions and points for which gestures can be interpreted i.e. blind spots of the cameras of the gesture recognition system. Some examples of such blind spots include areas underneath the cameras and/or within 20-85 degrees of a tangential axis of the cameras. In particular, the technology disclosed uses a Fresnel prismatic element and/or a triangular prism element to redirect the optical axis of the cameras, giving the cameras fields of view that cover at least 45 to 80 degrees from tangential to the vertical axis of a display screen on which the cameras are mounted.

## Claims

### Claim 1

A method for enhancing a field of view of at least one camera mounted to a display of an electronic device, the display having a vertical axis, the method comprising:
capturing video images by a camera with an optical axis facing within 20 degrees of tangential to the vertical axis of the display;
redirecting by at least one optical element, the optical axis of the camera, thereby giving the camera a field of view that covers at least 45 to 80 degrees from tangential to the vertical axis of the display; and
compensating by a camera controller coupled to the camera, for redirection by the optical element; thereby enabling determining a position of at least one control object within the field of view of the camera.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
modifying the optical axis of the camera using a bender that bends or reorients the optical element of the camera.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
redirecting at least one illumination source to cover an effective area of the field of view of the camera.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
redirecting by the at least one optical element, the optical axis of the camera, giving the camera a field of view that covers at least 20 to 85 degrees from tangential to the vertical axis of the display.

### Claim 5 (depends on 1)

The method of claim 1, wherein a second camera is mounted to the display with an optical axis facing within 20 degrees of tangential to the vertical axis of the display, wherein the second camera is different from the camera of claim 1, the method further including:
not redirecting, by the optical element, the optical axis of the second camera.

### Claim 6 (depends on 1)

The method of claim 1, wherein the display is at least one of planar or cylindrical.

### Claim 7 (depends on 1)

The method of claim 1, wherein the optical element comprises an optical film.

### Claim 8

A method for enhancing a field of view of at least one camera mounted relative to a display of an electronic device, the display having a vertical axis, the method comprising:
capturing video images by a camera with optical axis facing within 20 degrees of tangential to the vertical axis of the display;
redirecting by at least one prismatic optical element, the optical axis of the camera, thereby giving the camera a field of view that covers at least 45 to 80 degrees from tangential to the vertical axis of the display; and
compensating by a camera controller coupled to the camera, for redirection by the at least one prismatic optical element; thereby enabling determining a position of at least one control object within the field of view of the camera.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
modifying the optical axis of the camera.

### Claim 10 (depends on 8)

The method of claim 8, further comprising:
redirecting at least one illumination source to cover an effective area of the field of view of the camera.

### Claim 11 (depends on 8)

The method of claim 8, further comprising:
redirecting by the at least one prismatic optical element, the optical axis of the camera, thereby giving the camera a field of view that covers at least 20 to 85 degrees from tangential to the vertical axis of the display.

### Claim 12 (depends on 8)

The method of claim 8, wherein at least one camera is mounted to the display with an optical axis facing within 20 degrees of tangential to the vertical axis of the display, wherein the camera is different from the camera of claim 8, the method further comprising:
not redirecting, by the prismatic optical element, the optical axis of the camera.

### Claim 13 (depends on 8)

The method of claim 8, wherein the display is at least one of planar or cylindrical.

### Claim 14 (depends on 8)

The method of claim 8, wherein the prismatic optical element comprises an optical film.

### Claim 15

A method for enhancing a field of view of at least one camera of a portable electronic device with a display screen, the display screen having a vertical axis, the method comprising:
capturing video images by one camera with an optical axis facing within 20 degrees of tangential to the vertical axis of the display screen;
redirecting by an optical element, the optical axis of the camera, thereby giving the camera a field of view that covers at least 45 to 80 degrees from tangential to the vertical axis of the display screen; and
compensating by a camera controller coupled to the camera, for redirection by the optical element; thereby enabling determining a position of at least one control object within the field of view of the camera.

### Claim 16 (depends on 15)

The method of claim 15, wherein the optical element comprises an optical film.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
modify the optical axis of the camera using a bender that bends or reorients the optical element of the camera.

### Claim 18 (depends on 15)

The method of claim 15, further comprising:
redirecting at least one illumination source to cover an effective area of the field of view of the camera.

### Claim 19 (depends on 15)

The method of claim 15, further comprising:
redirecting by the optical element, the optical axis of the camera, giving the camera a field of view that covers at least 20 to 85 degrees from tangential to the vertical axis of the display screen.

### Claim 20 (depends on 15)

The method of claim 15, wherein at least one other camera is mounted to the portable electronic device with an optical axis facing within 20 degrees of tangential to the vertical axis of the display screen, wherein the other camera is different from the camera of claim 15, the method further comprising:
not redirecting, by the optical element, the optical axis of the other camera.
