# US 2019/0244431 A1 · Methods, devices, and systems for producing augmented reality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244431 A1
- Document ID: US20190244431A1
- Status: Published application
- Kind: A1
- Application number: 16/271,052
- Filed: 2019-02-08
- Published: 2019-08-08
- Earliest claimed: 2018-02-08
- Assignees: EDX Technologies, Inc.
- Applicants: EDX Technologies, Inc.
- Inventors: Roger Ray Skidmore
- Web page: https://patentgrep.com/patent/US20190244431A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06K 7/1417: {2D bar codes}
- G06K 19/06037: {multi-dimensional coding}
- G06T 15/20: Perspective computation
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS

## Abstract

AR engines modify a picture or video of the real world to include more than simply real world content. The modified image/frame content is passed along to one or multiple downstream AR engines which treat the content as original input. Modifications to images/frames may include the addition of virtual markers. Virtual markers may be visual content of virtual origin that is added to an image or frame. A virtual marker may trigger a predetermined reaction from a downstream AR engine when that downstream AR engine processes the modified content. For example, a virtual marker added to an image by an upstream AR engine may trigger a downstream AR engine to output a particular augmentation. The virtual marker may have a known meaning to both the upstream AR engine and the downstream AR engine. Accordingly, there may be “collaboration” among AR engines and a reduction in the processing requirements of downstream and overall image processing.

## Claims

### Claim 1

A method for producing an augmented reality (AR) for an image or video containing real world content, comprising
selecting one or more virtual markers from a virtual world based on a location associated with the real world objects depicted by the real world content;
applying, with an upstream AR engine, the one or more selected virtual markers to the image or video; and
triggering, by the one or more applied virtual markers, a downstream AR engine to augment the real world content of the image or video with one or more augmentations, wherein the downstream AR engine has a preexisting association among the one or more augmentations and the one or more applied virtual markers.

### Claim 2 (depends on 1)

The method of claim 1, wherein the selection in the selecting step is further based on a pose of a camera used to capture the image or video, wherein pose consists of a position and an orientation.

### Claim 3 (depends on 1)

The method of claim 1, wherein the applying step comprises modifying the real world content.

### Claim 4 (depends on 3)

The method of claim 3, wherein modifying the real world content consists of modifying one or more pixels.

### Claim 5 (depends on 1)

The method of claim 1, wherein the virtual world is a 3D virtual model that is modeled after a real world environment.

### Claim 6 (depends on 1)

The method of claim 1, wherein the location is a location of the real world objects or a location of a camera when the camera was used to capture the real world content depicting the real world content.

### Claim 7

A system for augmented reality (AR), comprising
one or more virtual markers stored in a virtual world that is modeled after a real world environment;
at least one upstream AR engine configured to
select one or more of the virtual markers from the virtual world based on a location associated with the real world objects depicted by the real world content of an image or video;
apply the one or more selected virtual markers to the image or video;
wherein the one or more applied virtual markers are configured to trigger a downstream AR engine to augment the real world content of the image or video with one or more augmentations, wherein the downstream AR engine has a preexisting association among the one or more augmentations and the one or more applied virtual markers.

### Claim 8 (depends on 7)

The system of claim 7, wherein the selection in the selecting step is further based on a pose of a camera used to capture the image or video, wherein pose consists of a position and an orientation.

### Claim 9 (depends on 7)

The system of claim 7, wherein the applying step comprises modifying the real world content.

### Claim 10 (depends on 9)

The system of claim 9, wherein modifying the real world content consists of modifying one or more pixels.

### Claim 11 (depends on 7)

The system of claim 7, wherein the virtual world is a 3D virtual model that is modeled after a real world environment.

### Claim 12 (depends on 7)

The system of claim 7, further comprising the downstream AR engine, the downstream AR engine being configured to recognize the one or more applied virtual markers and augment the real world content of the image or video with the one or more augmentations triggered by the one or more applied markers.

### Claim 13

A method for augmented reality (AR), comprising
based on a camera's location and pose, associating first virtual content with real world content captured by the camera, the first virtual content and real world content both being image or video content configured for concurrent display;
selecting second virtual content by processing an image or video frame containing both the first virtual content and the real world content, wherein selection of at least some of the second virtual content is triggered by the presence of the first virtual content, the second virtual content being usable for producing one or more AR augmentations; and
initiating a signal for controlling an augmented reality output by an output device, the output including the selected second virtual content and the real world content,
wherein the steps of associating, selecting, and initiating are performed by one or more AR engines.

### Claim 14 (depends on 13)

The method of claim 13, wherein the first virtual content is one or more markers.

### Claim 15 (depends on 14)

The method of claim 14, wherein the associating step comprises superimposing the one or more markers over selected parts of the real world content.

### Claim 16 (depends on 13)

The method of claim 13, wherein the associating step comprises adding the first virtual content to specific locations within the real world content.

### Claim 17 (depends on 16)

The method of claim 16, wherein the specific locations are specific pixels or groups of pixels.

### Claim 18 (depends on 13)

The method of claim 13, wherein the associating step is configured to associate the first virtual content with the real world content without extracting objects or features from the real world content captured by the camera.

### Claim 19 (depends on 18)

The method of claim 18, wherein the associating step is configured to associate the first virtual content with the real world content based only on the camera's location and pose.

### Claim 20 (depends on 13)

The method of claim 13, wherein the associating step is performed by a first AR engine and the selecting step is performed by a second AR engine.

### Claim at position 21

21-36. (canceled)
