# US 2020/0402313 A1 · System and method for attaching applications and interactions to static objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402313 A1
- Document ID: US20200402313A1
- Status: Published application
- Kind: A1
- Application number: 16/902,512
- Filed: 2020-06-16
- Published: 2020-12-24
- Earliest claimed: 2019-06-18
- Assignees: TMRW Foundation IP & Holding S. À R.L.
- Applicants: TMRW Foundation IP & Holding S. À R.L.
- Inventors: Cevat Yerli
- Web page: https://patentgrep.com/patent/US20200402313A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for attaching digital reality applications and interactions to static objects are disclosed. The system comprises a server configured to store and process input data, the server comprising a memory and a processor; and a plurality of client devices connected to the server via a network configured enable interaction with digital reality applications. The memory contains a database with structured data containing a persistent virtual world system storing virtual replicas of static objects. The virtual replicas are configured based on the static objects, comprising location and space, physics, and 3D structure. The system enables application developers to attach digital reality applications on the virtual replicas. When users employing client devices approach a digital reality application, the digital reality application detects the physical position and orientation of the client device, triggering the server to share digital reality application content with the client devices.

## Claims

### Claim 1

A system comprising:
one or more servers comprising a memory and a processor;
wherein the one or more servers contain a persistent virtual world system storing virtual replicas of static objects of the real world, the virtual replicas comprising a static object location and space, physics settings, and a 3D data structure, wherein the persistent virtual world system is configured to virtually attach digital reality applications on the virtual replicas, such that as a client device approaches a location, object, or entity to which a digital reality application is virtually attached, the digital reality application detects the physical position and orientation of the client device, triggering the one or more servers to transmit digital reality application content to the client device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the digital reality application content is presented in real reality, merged reality, or virtual reality.

### Claim 3 (depends on 1)

The system of claim 1, wherein interactions with the digital reality applications are personal or shared with other client devices.

### Claim 4 (depends on 1)

The system of claim 1, wherein the virtual replicas are created via computer vision techniques, computer-assisted drawings, computer-assisted engineering models, or combinations thereof.

### Claim 5 (depends on 4)

The system of claim 4, wherein the computer vision techniques comprise car or drone-based image-scanning through photo, video, depth measurements, simultaneous location and mapping (SLAM) scanning, or radar-imaging techniques, or a combination thereof.

### Claim 6 (depends on 1)

The system of claim 1, wherein the digital reality application content is streamed directly by the server to the client device and, as levels of interaction increase, the client device may download the digital reality applications or portions thereof and perform local computations and rendering.

### Claim 7 (depends on 1)

The system of claim 1, wherein digital content provided by the one or more digital reality applications comprises 3D image data, 3D geometries, 3D entities, 3D dynamic objects, video data, audio data, textual data, time data, positional data, orientational data, haptic data, or lighting data, or a combination thereof.

### Claim 8 (depends on 1)

The system of claim 1, wherein the client device comprises a combination of inertia measuring units and transceivers that enable sub-centimeter or sub-millimeter tracking of the computing devices.

### Claim 9 (depends on 8)

The system of claim 8, wherein the transceivers are millimeter-wave transceivers configured to wirelessly receive mmW radio waves from antennas and to send the data back to the antennas.

### Claim 10 (depends on 8)

The system of claim 8, wherein the tracking is performed through time of arrival (TOA), angle of arrival (AOA), visual imaging, GPS, or radar technology, or a combination thereof.

### Claim 11

A method comprising:
by a server computer system comprising a memory and a processor, providing a persistent virtual world system storing virtual replicas of static objects of the real world, the virtual replicas comprising a static object location and space, physics settings, and a 3D data structure; and
virtually attaching digital reality applications to the virtual replicas, such that as a client device approaches a location, object, or entity to which a digital reality application is virtually attached, the digital reality application detects the physical position and orientation of the client device, triggering the server computer system to transmit digital reality application content to the client device.

### Claim 12 (depends on 11)

The method of claim 11 further comprising:
creating, via a replica editor stored in a server, virtual replicas of the static objects selected from the real world;
adding virtual replica real-world properties, including location and space settings, physics settings, and 3D data structure;
virtually attaching, via the replica editor stored in the server, digital reality applications to the virtual replicas;

### Claim 13 (depends on 11)

The method of claim 11 further comprising:
retrieving and tracking, by a server, the viewing position and orientation of the client device; and
sending, by the server, digital reality application content to the client device.

### Claim 14 (depends on 11)

The method of claim 11, wherein the digital reality application content is presented in real reality, merged reality, or virtual reality.

### Claim 15 (depends on 11)

The method of claim 11, wherein interactions with the digital reality applications are personal or shared with other client devices.

### Claim 16 (depends on 11)

The method of claim 11, wherein the virtual replicas are created via computer vision techniques, computer-assisted drawings or computer-assisted engineering models, or combinations thereof.

### Claim 17 (depends on 16)

The method of claim 16, wherein the computer vision techniques comprise car or drone-based image-scanning through photo, video, depth measurements, simultaneous location and mapping (SLAM) scanning, or radar-imaging techniques, or a combination thereof.

### Claim 18 (depends on 11)

The method of claim 11, wherein the digital reality application content is streamed directly by the server to the client device and, as levels of interaction increase, the client device may download the digital reality applications and perform local computations and rendering.

### Claim 19 (depends on 11)

The method of claim 11, wherein digital content provided by the one or more digital reality applications comprises 3D image data, 3D geometries, 3D entities, 3D dynamic objects, video data, audio data, textual data, time data, positional data, orientational data, haptic data, or lighting data, or a combination thereof.

### Claim 20 (depends on 11)

The method of claim 11, wherein the client device comprises a combination of inertia measuring units and transceivers that enable sub-centimeter or sub-millimeter tracking of the computing devices.
