# US 2026/0228614 A1 · Interactive telemetry visualization of characters in a virtual environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0228614 A1
- Document ID: US20260228614A1
- Status: Published application
- Kind: A1
- Application number: 19/047,465
- Filed: 2025-02-06
- Published: 2026-08-06
- Earliest claimed: 2025-02-06
- Assignees: Epic Games, Inc.
- Applicants: Epic Games, Inc.
- Inventors: Nathan Martz; Stewart Miles; Leopold Haller; Chiaoling Hong
- Web page: https://patentgrep.com/patent/US20260228614A1

## CPC

- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 20/00: Machine learning
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system for visualizing telemetry of characters may capture telemetry data of characters within an environment, where the characters are controlled by human or agent players, and where the telemetry data for each of the characters includes a temporal stream of observations provided to the characters and actions taken by the characters. The system may further capture video data of the characters with the telemetry data, embed metadata into the telemetry data or the video data associated with a state of the environment, synchronize the telemetry data and the video data into a unified time stream based at least in part on the metadata, provide a user with an interactive interface allowing the user to select at least one of a portion of the unified time stream for visualization or a visualization axis, and provide a user with visualizations of the unified time stream based on selections.

## Claims

### Claim 1

A system comprising:
a controller including one or more processors configured to execute program instructions stored in a memory medium causing the one or more processors to:
capture telemetry data of one or more characters within an environment, wherein the one or more characters are controlled by at least one of human or agent players, wherein the telemetry data for each of the one or more characters includes a temporal stream of observations provided to the one or more characters and actions taken by the one or more characters;
capture video data of the one or more characters within the environment simultaneously with the telemetry data;
embed metadata into at least one of the telemetry data or the video data associated with a state of the environment;
synchronize the telemetry data and the video data into a unified time stream based at least in part on the metadata;
provide a user with an interactive interface allowing the user to select at least one of a portion of the unified time stream for visualization or a visualization axis, wherein the visualization axis comprises at least one of space or time in the environment; and
provide a user with one or more visualizations of the unified time stream based on selections in the interactive interface.

### Claim 2 (depends on 1)

The system of claim 1, wherein the observations provided to the one or more characters comprise one or more attributes of any of the one or more characters, wherein the interactive interface allows the user to display data associated with the one or more attributes for visualization.

### Claim 3 (depends on 2)

The system of claim 2, wherein the one or more attributes comprise at least one of health or status of any of the one or more characters along the visualization axis.

### Claim 4 (depends on 1)

The system of claim 1, wherein the interactive interface provides a graphical user interface with one or more panels.

### Claim 5 (depends on 4)

The system of claim 4, wherein at least one of the one or more panels provides at least one of a timeline view, a video view, or a spatial view of the unified time stream.

### Claim at position 6

6-7. (canceled)

### Claim 8 (depends on 4)

The system of claim 4, wherein the interactive interface provides a graphical user interface with two or more panels, wherein at least two of the two or more panels provide synchronized data.

### Claim 9

(canceled)

### Claim 10 (depends on 1)

The system of claim 1, wherein at least one of the one or more characters is controlled by an agent.

### Claim 11 (depends on 10)

The system of claim 10, wherein the agent includes a neural network.

### Claim 12 (depends on 10)

The system of claim 10, wherein the agent includes an application programming interface (API), a generative coding model configured to generate code based on the API, and executable code created by the generative coding model.

### Claim 13

(canceled)

### Claim 14 (depends on 1)

The system of claim 1, wherein the one or more characters include two or more characters, wherein the interactive interface provides data associated with a single one of the one or more characters.

### Claim 15 (depends on 1)

The system of claim 1, wherein the one or more characters include two or more characters, wherein the interactive interface provides one or more composite views of data associated with at least two of the two or more characters.

### Claim 16

A method comprising:
capturing telemetry data of one or more characters within an environment, wherein the one or more characters are controlled by at least one of human or agent players, wherein the telemetry data for each of the one or more characters includes a temporal stream of observations provided to the one or more characters and actions taken by the one or more characters;
capturing video data of the one or more characters within the environment simultaneously with the telemetry data;
embedding metadata into at least one of the telemetry data or the video data associated with a state of the environment;
synchronizing the telemetry data and the video data into a unified time stream based at least in part on the metadata;
providing a user with an interactive interface allowing the user to select at least one of a portion of the unified time stream for visualization or a visualization axis, wherein the visualization axis comprises at least one of space or time in the environment; and
providing a user with one or more visualizations of the unified time stream based on selections in the interactive interface.

### Claim 17 (depends on 16)

The method of claim 16, wherein the observations provided to the one or more characters comprise one or more attributes of any of the one or more characters, wherein the interactive interface allows the user to display data associated with the one or more attributes for visualization.

### Claim 18 (depends on 17)

The method of claim 17, wherein the one or more attributes comprise at least one of health or status of any of the one or more characters along the visualization axis.

### Claim 19

(canceled)

### Claim 20 (depends on 16)

The method of claim 16, wherein the interactive interface provides at least one of a timeline view, a video view, or a spatial view of the unified time stream.

### Claim at position 20

21-22. (canceled)

### Claim 23 (depends on 16)

The method of claim 16, wherein the interactive interface provides a graphical user interface with two or more panels, wherein at least two of the two or more panels provide synchronized data.

### Claim 24 (depends on 16)

The method of claim 16, wherein at least one of the one or more characters is controlled by an agent.

### Claim 25 (depends on 24)

The method of claim 24, wherein the agent includes a neural network.

### Claim 26 (depends on 24)

The method of claim 24, wherein the agent includes an application programming interface (API), a generative coding model configured to generate code based on the API, and executable code created by the generative coding model.

### Claim 27

(canceled)

### Claim 28 (depends on 16)

The method of claim 16, wherein the one or more characters include two or more characters, wherein the interactive interface provides data associated with a single one of the one or more characters.

### Claim 29 (depends on 16)

The method of claim 16, wherein the one or more characters include two or more characters, wherein the interactive interface provides one or more composite views of data associated with at least two of the two or more characters.
