# US 12,217,356 B2 · Augmented reality wall with combined viewer and camera tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,356 B2
- Document ID: US12217356B2
- Status: Granted
- Kind: B2
- Application number: 18/503,043
- Filed: 2023-11-06
- Published: 2024-02-29
- Granted: 2025-02-04
- Earliest claimed: 2017-12-06
- Assignees: ARWall, Inc.
- Applicants: ARwall, Inc.
- Inventors: Leon Hui; Rene Amador; William Hellwarth; Michael Plescia
- Web page: https://patentgrep.com/patent/US12217356B2

## CPC

- G06T 15/20: Perspective computation
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/73: using feature-based methods
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- H04N 13/00: Stereoscopic video systems; Multi-view video systems; Details thereof
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06T 2207/10048: Infrared image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems, methods, and machine-readable media are disclosed. A computing device is in communications with a display and one or more trackers that determine a location of a camera relative to the display. The computing device generates a digital scene for presentation on the display such that one or more objects shown in the digital scene have suitable perspective for the location of the camera as determined by the tracker.

## Claims

### Claim 1

A system comprising:
a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display,
the computing device configured to:
receive the location of the moving camera from the one or more trackers, and
generate a real-time, rendered digital scene for presentation on the display;
wherein the computing device generates the digital scene such that one or more objects shown in the digital scene each have suitable perspective for the location of the moving camera as received from the one or more trackers; and
wherein the moving camera captures the digital scene.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
the display;
the one or more trackers; and
the moving camera.

### Claim 3 (depends on 1)

The system of claim 1, wherein
the moving camera moves relative to the display, the one or more trackers track the location of the moving camera when the moving camera moves, and
the computing device generates the digital scene such that the one or more objects shown in the digital scene each maintain suitable perspective for the location of the moving camera when the camera moves.

### Claim 4 (depends on 1)

The system of claim 1, wherein
the moving camera captures a composite image including a physical object disposed between the camera and the display,
the one or more trackers determine the location of the physical object with respect to the moving camera and the display, and
the computing device generates the digital scene such that at least one of the one or more objects shown in the digital scene is positioned within the scene based on the location of the physical object as determined by the one or more trackers.

### Claim 5 (depends on 4)

The system of claim 4, wherein
the physical object is movable with respect to the display,
the one or more trackers track the location of the physical object when the physical object moves, and
the computing device generates the digital scene such that the at least one of the one or more objects shown in the digital scene is repositioned within the scene based on the location of the physical object as the physical object moves.

### Claim 6 (depends on 5)

The system of claim 5, wherein
the physical object is an actor.

### Claim 7 (depends on 1)

The system of claim 1, wherein the one or more trackers comprises the moving camera.

### Claim 8

A method performed by a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, the method comprising:
receiving, from the one or more trackers, the location of the moving camera; and
generating a real-time, rendered digital scene for presentation on the display;
wherein one or more objects shown in the digital scene have suitable perspective for the location of the moving camera as received from the one or more trackers; and
capturing the digital scene using the moving camera.

### Claim 9 (depends on 8)

The method of claim 8, wherein the moving camera moves relative to the display, the method further comprising:
the one or more trackers tracking the location of the moving camera when the moving camera moves, and
the computing device generating the digital scene such that the one or more objects shown in the digital scene maintain suitable perspective for the location of the moving camera as the moving camera moves.

### Claim 10 (depends on 8)

The method of claim 8, wherein the moving camera captures a composite image including a physical object disposed between the camera and the display, the method further comprising:
the one or more trackers determining the location of the physical object with respect to the display, and
the computing device generating the digital scene such that at least one of the one or more objects shown in the digital scene is positioned within the scene based on the location of the physical object as determined by the one or more trackers.

### Claim 11 (depends on 10)

The method of claim 10, wherein the physical object is movable with respect to the display, the method further comprising:
the one or more trackers tracking the location of the physical object when the physical object moves, and
the computing device generating the digital scene such that the at least one of the one or more objects shown in the digital scene is repositioned within the scene based on the location of the physical object as the physical object moves.

### Claim 12 (depends on 11)

The method of claim 11, wherein
the physical object is an actor.

### Claim 13 (depends on 8)

The method of claim 8, wherein the one or more trackers comprises the moving camera.

### Claim 14

A non-transitory machine-readable medium storing instructions which, when executed by a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, will cause the computing device to perform actions comprising:
receiving the location of the moving camera from the one or more trackers; and
generating a real-time, rendered digital scene for presentation on the display;
wherein the computing device generates the digital scene such that one or more objects shown in the digital scene have suitable perspective for the location of the moving camera as received from the one or more trackers; and
capturing the digital scene using the moving camera.

### Claim 15 (depends on 14)

The machine-readable medium of claim 14, wherein the camera is movable with respect to the display, the actions performed further comprising:
the one or more trackers tracking the location of the moving camera when the moving camera moves, and
the computing device generating the digital scene such that the one or more objects shown in the digital scene maintain suitable perspective for the location of the moving camera as the moving camera moves.

### Claim 16 (depends on 14)

The machine-readable medium of claim 14, wherein the moving camera captures a composite image including a physical object disposed between the moving camera and the display, the actions performed further comprising:
the one or more trackers determining the location of the physical object with respect to the display, and
the computing device generating the digital scene such that at least one of the one or more objects shown in the digital scene is positioned within the scene based on the location of the physical object as determined by the one or more trackers.

### Claim 17 (depends on 16)

The machine-readable medium of claim 16, wherein the physical object is movable with respect to the display, the actions performed further comprising:
the one or more trackers tracking the location of the physical object when the physical object moves, and
the computing device generating the digital scene such that the at least one of the one or more objects shown in the digital scene is positioned within the scene based on the location of the physical object as the physical object moves.

### Claim 18 (depends on 16)

The machine-readable medium of claim 16, wherein
the physical object is an actor.

### Claim 19 (depends on 14)

The machine-readable medium of claim 14, wherein the one or more trackers comprises the moving camera.