# US 2025/0165058 A1 · Multiparty object recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165058 A1
- Document ID: US20250165058A1
- Status: Published application
- Kind: A1
- Application number: 19/023,200
- Filed: 2025-01-15
- Published: 2025-05-22
- Earliest claimed: 2014-07-15
- Assignees: Nant Holdings IP, LLC
- Applicants: Nant Holdings IP, LLC
- Inventors: Bing Song; John Wiacek; David McKinnon; Matheen Siddiqui
- Web page: https://patentgrep.com/patent/US20250165058A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/215: comprising means for detecting acoustic signals, e.g. using a microphone
- A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- G06T 2219/024: Multi-user, collaborative environment
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Multiparty object recognition systems and methods are disclosed. A method of interactively manipulating virtual object data, wherein an object database is configured to store first party object data that corresponds to a first real-world object and is further configured to store second party object data that corresponds to a second real-world object, includes obtaining the first party object data and the second party object data for storage within the object database. Access to the object database is controlled such that the first party object data and the second party object data is accessible to the first party and the second party. Modification of the first party object data by the second party is facilitated to generate modified first party object data that is in accordance with at least one context parameter of the second party object data, and the modified first party object data is communicated to the first party.

## Claims

### Claim at position 1

1-30. (canceled)

### Claim 31

A multiparty object recognition system for an interactive game, the system comprising:
a processor;
an object database configured to store first player object data, obtained using a first device associated with a first player, that include first virtual object data and further configured to store second player object data, obtained using a second device associated with a second player, that include second virtual object data, wherein the first virtual object data and the second virtual object data each comprise one or more representations of virtual elements of virtual objects associated with an augmented reality environment, a virtual reality environment, or a mixed reality environment, at least one of the one or more representations being generated based on one or both of recognition of a real-world object within a scene in which the first device is located and recognition of a real-world object within a scene in which the second device is located; and
a memory storing software instructions that, when executed by a processor, cause the processor to operate as a portal engine configured to:
obtain the first player object data and the second player object data for storage within the object database;
control access to the object database such that the first player object data and the second player object data is accessible to at least the first player and the second player;
analyze matching features of the first player object data and the second player object data by using geographic recognition, wherein the analysis of matching features comprises determining a match of geographic features between the scene in which the first device is located and the scene in which the second device is located, and a match of at least one speech feature, geometric feature, or user-relationship feature;
generate modified first player object data by facilitating modification of at least one structure of the virtual object of the first player object data by the second player based on the analyzed matching features, wherein the modification represents a manipulation of the at least one structure of the virtual object of the first player object data within the augmented reality environment, virtual reality environment, or mixed reality environment, the modified first player object data being in accordance with at least one context parameter of the second player object data;
identify the modified first player object data as a recognized object;
retrieve, from the object database, an indication or information regarding an element of the game and associated with the recognized object; and
communicate the indication or information regarding the element of the game for presentation to at least one of the first party or the second party.

### Claim 32 (depends on 31)

The system of claim 31, wherein the indication or information regarding the element of the game comprises additional object data associated with the recognized object that is updatable over time.

### Claim 33 (depends on 31)

The system of claim 31, wherein the portal engine is further configured to:
communicate the modified first party object data for presentation to the first party, wherein the communication includes at least portions of the first party object data and second party object data such that, when displayed to the first party, the presentation comprises the at least a portion of first virtual object data and the at least a portion of second virtual object data, wherein the first virtual object data is rendered as being manipulated or otherwise modified by the second virtual object data in accordance with the modified first party object data.

### Claim 34 (depends on 31)

The system of claim 31, wherein the first player and the second player are remotely located from one another in a real-world environment.

### Claim 35 (depends on 31)

The system of claim 31, wherein one of the first or second virtual object data comprises one or more representations of a virtual writing surface.

### Claim 36 (depends on 35)

The system of claim 35, wherein the portal engine is further configured to:
based on the one or more representations of the virtual writing surface and location information, recognize the virtual writing surface as a correct one for the game, and obtain object data associated with use of the virtual writing surface in the game.

### Claim 37 (depends on 31)

The system of claim 31, wherein one of the first or second virtual object data comprises one or more representations of a virtual writing device.

### Claim 38 (depends on 37)

The system of claim 37, wherein the portal engine is further configured to:
based on the one or more representations of the virtual writing device and location information, recognize the virtual writing device as a correct one for the game, and obtain object data associated with use of the virtual writing device in the game.

### Claim 39 (depends on 31)

The system of claim 31, wherein at least one of background content of images can be used to recognize a location of the first player or the second player.

### Claim 40 (depends on 31)

The system of claim 31, wherein Global Positioning System (GPS) or other location information can be used to recognize a location of the scene in which the first device is located or a location of the scene in which the second device is located.

### Claim 41 (depends on 31)

The system of claim 31, wherein at least one of the first player object data or second player object data includes one or more instructions regarding the indication or information regarding the element of the game.

### Claim 42 (depends on 41)

The system of claim 41, wherein the instructions include instructions to write a correct phrase or password on a virtual writing surface using a virtual writing device.

### Claim 43 (depends on 42)

the system of claim 42, wherein after the correct phrase or password is written, the indication or information regarding the element of the game is presented to at least one of the first player or the second player.

### Claim 44 (depends on 31)

The system of claim 31, wherein the recognized object is a virtual game object.

### Claim 45 (depends on 31)

The system of claim 31, wherein the game comprises a scavenger hunt and the indication or information regarding the element of the game is a clue that relates to the scavenger hunt.

### Claim 46 (depends on 31)

The system of claim 31, wherein the at least one context parameter corresponds to at least one of a temporal context, a spatial context, a thematic context, an additive context, a subtractive context, and a geometric context of the second player object data.

### Claim 47 (depends on 31)

The system of claim 31, wherein at least one of the first party object data and the second party object data comprises metadata.

### Claim 48 (depends on 47)

The system of claim 47, wherein the metadata comprises at least one of first party-specific data or second party-specific data, location data, time data, user identification data, annotation data, social media data, product data, security data, and contextual data.

### Claim 49 (depends on 31)

The system of claim 31, wherein the one or more representations of virtual elements of a virtual object comprise one or more representations of virtual elements of the game.

### Claim 50 (depends on 49)

The system of claim 49, wherein the one or more representations of virtual elements of the game are generated based on respective perceptions of the scene by the first device and the second device.

### Claim 51 (depends on 49)

The system of claim 49, wherein the one or more representations of virtual elements of the game are generated based on a first scene in which the first device is located and a second scene in which the second device is located, wherein the first scene is different from the second scene.

### Claim 52 (depends on 31)

The system of claim 31, wherein the first player and the second player include at least one of a person, a group of people, and an automated device.

### Claim 53

A multiparty object recognition method for an interactive game, wherein an object database is configured to store first player object data, obtained using a first device associated with a first player, that include first virtual object data and further configured to store second player object data, obtained using a second device associated with a second player, that include second virtual object data, wherein the first virtual object data and the second virtual object data each comprise one or more representations of virtual elements of virtual objects associated with an augmented reality environment, a virtual reality environment, or a mixed reality environment, at least one of the one or more representations being generated based on one or both of recognition of a real-world object within a scene in which the first device is located and recognition of a real-world object within a scene in which the second device is located, the method comprising:
obtaining the first player object data and the second player object data for storage within the object database;
controlling access to the object database such that the first player object data and the second player object data is accessible to at least the first player and the second player;
analyzing matching features of the first player object data and the second player object data by using geographic recognition, wherein the analysis of matching features comprises determining a match of geographic features between the scene in which the first device is located and the scene in which the second device is located, and a match of at least one speech feature, geometric feature, or user-relationship feature;
generating modified first player object data by facilitating modification of at least one structure of the virtual object of the first player object data by the second player based on the analyzed matching features, wherein the modification represents a manipulation of the at least one structure of the virtual object of the first player object data within the augmented reality environment, virtual reality environment, or mixed reality environment, the modified first player object data being in accordance with at least one context parameter of the second player object data;
identifying the modified first player object data as a recognized object;
retrieving, from the object database, an indication or information regarding an element of the game and associated with the recognized object; and
communicating the indication or information regarding the element of the game for presentation to at least one of the first party or the second party.

### Claim 54

A computer program product embedded in a non-transitory computer readable medium comprising instructions for an interactive game, wherein an object database is configured to store first player object data, obtained using a first device associated with a first player, that include first virtual object data and further configured to store second player object data, obtained using a second device associated with a second player, that include second virtual object data, wherein the first virtual object data and the second virtual object data each comprise one or more representations of virtual elements of virtual objects associated with an augmented reality environment, a virtual reality environment, or a mixed reality environment, at least one of the one or more representations being generated based on one or both of recognition of a real-world object within a scene in which the first device is located and recognition of a real-world object within a scene in which the second device is located, the instructions, when executed by a processor, cause the processor to operate as a portal engine configured to:
obtain the first player object data and the second player object data for storage within the object database;
control access to the object database such that the first player object data and the second player object data is accessible to at least the first player and the second player;
analyze matching features of the first player object data and the second player object data by using geographic recognition, wherein the analysis of matching features comprises determining a match of geographic features between the scene in which the first device is located and the scene in which the second device is located, and a match of at least one speech feature, geometric feature, or user-relationship feature;
generate modified first player object data by facilitating modification of at least one structure of the virtual object of the first player object data by the second player based on the analyzed matching features, wherein the modification represents a manipulation of the at least one structure of the virtual object of the first player object data within the augmented reality environment, virtual reality environment, or mixed reality environment, the modified first player object data being in accordance with at least one context parameter of the second player object data;
identify the modified first player object data as a recognized object;
retrieve, from the object database, an indication or information regarding an element of the game and associated with the recognized object; and
communicate the indication or information regarding the element of the game for presentation to at least one of the first party or the second party.
