# US 2020/0401576 A1 · Interacting with real-world items and corresponding databases through a virtual twin reality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401576 A1
- Document ID: US20200401576A1
- Status: Published application
- Kind: A1
- Application number: 16/903,254
- Filed: 2020-06-16
- Published: 2020-12-24
- Earliest claimed: 2019-06-18
- Assignees: TMRW Foundation IP & Holding S. À R.L.
- Applicants: TMRW Foundation IP & Holding S. À R.L.
- Inventors: Cevat Yerli
- Web page: https://patentgrep.com/patent/US20200401576A1

## CPC

- G06F 9/46: Multiprogramming arrangements
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06F 9/453: {Help systems}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06N 20/00: Machine learning
- G06Q 30/0623: {by investigating goods or services}
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 15/205: {Image-based rendering}
- G06T 17/005: {Tree description, e.g. octree, quadtree}
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06)
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- G06Q 20/123: {Shopping for digital content}
- G06T 2219/024: Multi-user, collaborative environment
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A system comprises at least one cloud server of a cloud server computer system comprising at least one processor and memory storing a persistent virtual world system comprising one or more virtual objects including virtual data and models. The virtual objects comprise one or more of a virtual twin, a pure virtual object, or an application, wherein at least one of the virtual objects represents a store of real-world items connected to a periodically-updated database associated to the products of the at least one store. Users may access the store through the persistent virtual world system via a user device enabling interactions with and between elements within the store.

## Claims

### Claim 1

A system that enables updates of a database of real-world items via a virtual world system, the system comprising:
at least one server of a cloud server computer system comprising at least one processor and memory storing a persistent virtual world system comprising one or more virtual objects including virtual data and models, the virtual objects comprising one or more of a virtual twin, a pure virtual object, or an application, wherein at least one of the virtual objects represents a store of real-world items connected to a periodically-updated database associated with the real-world items of the store, and wherein the persistent virtual world system provides access to the database via a user interface of a user device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the store is a virtual store represented by a corresponding virtual object, wherein users can access the virtual store in virtual reality from a remote physical location.

### Claim 3 (depends on 2)

The system of claim 2, wherein the remote physical location comprises connected devices capturing data from the user that is transferred to an avatar through the persistent virtual world system.

### Claim 4 (depends on 1)

The system of claim 1, wherein the at least one virtual object that represents the store comprises a virtual twin stored in the persistent virtual world system and communicates with a plurality of connected devices configured to identify a plurality of real-world items or user interactions with the plurality of real-world items and to send corresponding state updates to the database, wherein users can access the store in augmented reality or remotely in virtual reality.

### Claim 5 (depends on 4)

The system of claim 4, wherein the plurality of connected devices include one or more cameras comprising RGBD cameras, depth cameras, LIDAR cameras, or CCTV cameras, or combinations thereof, wherein the cameras employ artificial intelligence-based machine vision to identify the plurality of real-world items or the user interactions with the real-world items.

### Claim 6 (depends on 4)

The system of claim 4, wherein the plurality of connected devices include sensors comprising one or more of temperature sensors, humidity sensors, moisture sensors, mass sensors, proximity sensors, inertial sensors, light sensors, gas and chemical sensors, pressure sensors, touch sensors, electrical sensors, or combinations thereof.

### Claim 7 (depends on 1)

The system of claim 1, wherein the at least one server stores separate layers for augmented and virtual realities that are activated through the user device connected to the at least one server, each layer virtual twins specific to each layer.

### Claim 8 (depends on 1)

The system of claim 1, wherein the store has an associated digital identifier stored in the at least one server, the digital identifier being retrieved by the user device from the at least one server in order to access the associated store through the persistent virtual world system.

### Claim 9 (depends on 1)

The system of claim 1, wherein the at least one virtual object that represents the store further comprises a personal assistant, and wherein the personal assistant is an artificial intelligence agent.

### Claim 10 (depends on 1)

The system of claim 1, wherein the user device has a personal identifier associated with a personal profile, and wherein a transaction or interaction performed by the user is recorded under the personal identifier.

### Claim 11 (depends on 1)

The system of claim 1, wherein entering or leaving a predetermined area of the store signals the at least one server to complete a transaction.

### Claim 12

A method enabling updates of a database of real-world items via a virtual world system, the method comprising:
providing at least one server of a cloud server computer system comprising at least one processor and memory storing a persistent virtual world system comprising one or more virtual objects including virtual data and models, the virtual objects comprising one or more of a virtual twin, a pure virtual object, or an application;
providing at least one virtual object representing a store of real-world items;
connecting, to the at least one virtual object representing the store, a database associated with the store;
receiving, from a user device, interaction or transaction instructions related to one or more of the real-world items, wherein an interaction or transaction associated with the received instructions is performed in augmented or virtual reality through the persistent virtual world system; and
updating the database based on the received instructions.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
receiving, from connected devices installed in a remote physical location, user data associated with a user of the user device, the user data comprising user movements, position and orientation in the remote physical location; and
updating a user avatar within the persistent virtual world system.

### Claim 14 (depends on 12)

The method of claim 12, wherein the at least one virtual object is a virtual twin of the store, the method further comprising:
providing corresponding virtual twins of each of the plurality of real-world items;
providing a plurality of connected devices configured to identify the plurality of real-world items or user interactions with the plurality of real-world items and to send corresponding state updates to the database;
receiving, by the persistent virtual world system, real-world data of the store from the connected devices; and
receiving interaction or transaction instructions from user devices.

### Claim 15 (depends on 12)

The method of claim 12, further comprising providing in the at least one server a digital identifier associated to each store that is retrieved by the user device in order to access the associated store through the persistent virtual world system.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
storing in the at least one server a user device personal identifier enabling identification of a user; and
recording in the memory of the server a transaction performed by the user under the user device personal identifier.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
detecting, through one or more sensors configured in one or more pre-determined areas, the user device personal identifier of the user entering or leaving one of the one or more predetermined areas;
completing a transaction based on the detecting; and
initiating an automated retrieval and delivery system.

### Claim 18

A computer-readable medium having stored thereon executable instructions configured to cause at least one server of a cloud server computer system comprising at least one processor and memory to perform steps comprising:
storing a persistent virtual world system comprising one or more virtual objects including virtual data and models, the virtual objects comprising one or more of a virtual twin, a pure virtual object, or an application;
providing at least one virtual object representing a store of real-world items;
connecting, to the at least one virtual object representing the store, a database associated with the store;
receiving, from a user device, interaction or transaction instructions related to one or more of the real-world items, wherein an interaction or transaction associated with the received instructions is performed in augmented or virtual reality through the persistent virtual world system; and
updating the database based on the received instructions.

### Claim 19 (depends on 18)

The computer-readable medium of claim 18, the steps further comprising:
receiving, from connected devices installed in a remote physical location, user data associated with a user of the user device, the user data comprising user movements, position and orientation in the remote physical location; and
updating a user avatar within the persistent virtual world system.

### Claim 20 (depends on 18)

The computer-readable medium of claim 18, wherein the at least one virtual object is a virtual twin of the store, the steps further comprising:
providing corresponding virtual twins of each of the plurality of real-world items; and
receiving, by the persistent virtual world system, real-world data of the store from a plurality of connected devices configured to identify the plurality of real-world items or user interactions with the plurality of real-world items and to send corresponding state updates to the database.