# US 10,372,209 B2 · Eye tracking enabling 3D viewing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,209 B2
- Document ID: US10372209B2
- Status: Granted
- Kind: B2
- Application number: 16/144,635
- Filed: 2018-09-27
- Published: 2019-01-24
- Granted: 2019-08-06
- Earliest claimed: 2010-08-31
- Assignees: Nintendo Co., Ltd.
- Applicants: Nintendo Co., Ltd.
- Inventors: Howard Cheng; William C. Newman, Jr.
- Web page: https://patentgrep.com/patent/US10372209B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 15/00: Three-dimensional [3D] image rendering
- H04N 13/279: the virtual viewpoint locations being selected by the viewers or determined by tracking
- H04N 13/366: using viewer tracking
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The exemplary illustrative non-limiting technology herein enables 3D viewing on conventional 2D displays such as home television sets by tracking a person's viewpoint. Detecting a player's viewpoint movement to change the viewing of the displayed object gives the illusion that the object is physically present in three-dimensional space. Viewpoint movement detection can provide collision-related game logic benefits such as allowing a player to dodge projectiles, giving a game character an ability to “see” the player when not behind line-of-sight obstacles, and other advantages.

## Claims

### Claim 1

A system for generating 3D images comprising:
at least one sensor configured to track a direction in which a user is looking;
a graphics engine operatively coupled to the at least one sensor, the graphics engine rendering 3D graphical images in real time response to the direction the at least one sensor senses;
the graphics engine modeling objects in real world coordinates and introducing parallax distortion to near field and far field objects to increase perception of parallax motion.

### Claim 2 (depends on 1)

The system of claim 1 wherein said sensor tracks the user's head orientation.

### Claim 3 (depends on 1)

The system of claim 1 wherein the introduced parallax enhances the 3D viewing a displayed image plane.

### Claim 4 (depends on 1)

The system of claim 1 wherein the graphics engine animates motion of said objects.

### Claim 5 (depends on 1)

The system of claim 1 wherein said graphics engine images said object as part of an animated interactive display.

### Claim 6 (depends on 1)

The system of claim 1 wherein said objects include a projectile the user is to dodge in the real world.

### Claim 7 (depends on 1)

The system of claim 1 wherein said sensor tracks plural points on the user.

### Claim 8 (depends on 7)

The system of claim 7 wherein said sensor tracks at least three points on the user.

### Claim 9 (depends on 1)

The system of claim 1 wherein said sensor comprises an infrared sensor.

### Claim 10 (depends on 1)

The system of claim 1 wherein the graphics engine performs 3D collision detection for portions of said objects that appear to extend beyond a viewing screen into the real world.

### Claim 11 (depends on 1)

The system of claim 1 wherein said graphics engine changes the viewing frustrum in response to said sensor.

### Claim 12

A system for providing 3D viewing comprising:
a tracking device that tracks a user's eyes; and
at least one processor coupled to said tracking device, said processor structured to project of at least one virtual object onto an image plane in response to the tracking device, and to use parallax responsive to the tracking device to generate a sequence of images that cause the virtual object to appear to extend beyond the image plane into the real world.

### Claim 13 (depends on 12)

The system of claim 12 wherein said tracking device tracks the user's head through use of a marker.

### Claim 14 (depends on 12)

The system of claim 12 wherein said processor introduces parallax to enhance parallax motion.

### Claim 15 (depends on 12)

The system of claim 12 wherein said processor animates motion of said virtual object.

### Claim 16 (depends on 12)

The system of claim 12 wherein said processor is adapted to cause display of said virtual object as part of an animated interactive application.

### Claim 17 (depends on 12)

The system of claim 12 wherein said virtual object comprises a projectile that the user is to dodge in the real world.

### Claim 18 (depends on 12)

The system of claim 12 wherein said tracking device comprises an infrared sensor.

### Claim 19 (depends on 12)

The system of claim 12 wherein said processor is structured to perform 3D collision detection for portions of said virtual object that appear to extend into the real world.

### Claim 20 (depends on 12)

The system of claim 12 wherein said processor is structured to change the viewing frustrum in response to said tracking device.