US 2024/0253620 A1Published application

Image synthesis for discrete track prediction

AssigneeZoox, Inc.

InventorEthan Miller Pronovost

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques for generating discrete track data for object(s) in an environment are described herein. For example, the techniques may include a decoder of a variable autoencoder may generate, based on first latent variable data associated with an object and a first time, first output data representing a first discrete occupancy representation for the object at the first time. A diffusion model may generate, based on the first output data, second latent variable data associated with the object and a second time subsequent to the first time. The decoder may generate, based on the second latent variable data, second output data representing a second discrete occupancy representation for the object at the second time. A track component may generate, based on the first output data and the second output data, track data including a discrete track for the object over at least the first time and the second time.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:
generating, by a decoder of a variable autoencoder and based on map data and first latent variable data, first output data, wherein:
the first latent variable data is associated with a first object in an environment, a second object in the environment, and a first time;
and
the first output data represents a first bounding box for the first object at the first time and a second bounding box for the second object at the first time;
and
the first latent variable data represents a first behavior of the first object at the first time and a second behavior of the second object at the first time;
generating, by a diffusion model and based at least in part on the first output data, second latent variable data, wherein the second latent variable data is associated with the first object in the environment, the second object in the environment, and a second time subsequent to the first time;
generating, by the decoder and based at least in part on the second latent variable data, second output data representing a third bounding box for the first object at the second time and a fourth bounding box for the second object at the second time;
generating, by a track component and based at least in part on the first output data and the second output data, track data including a first discrete track for the first object and a second discrete track for the second object;
and
at least one of:
performing, based at least in part on the track data, a simulation between a vehicle, the first object, and the second object;
or
controlling, based at least in part on the track data, the vehicle in the environment relative to the first object and the second object.

14 more elements

One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:

generating, based on first latent variable data associated with an object in an environment and a first time, first output data representing a first discrete occupancy representation for the object at the first time, wherein the first latent variable data represents a behavior of the object at the first time;
generating, by a generative model and based at least in part on the first output data, second latent variable data, wherein the second latent variable data is associated with the object in the environment and a second time different from the first time;
generating, based at least in part on the second latent variable data, second output data representing a second discrete occupancy representation for the object at the second time;
and
generating, by a track component and based at least in part on the first output data and the second output data, track data including a discrete track for the object over at least the first time and the second time.

2 more elements

A method comprising:

generating, based on first latent variable data associated with an object in an environment and a first time, first output data representing a first discrete occupancy representation for the object at the first time, wherein the first latent variable data represents a behavior of the object at the first time;
generating, by a generative model and based at least in part on the first output data, second latent variable data, wherein the second latent variable data is associated with the object in the environment and a second time different from the first time;
generating, based at least in part on the second latent variable data, second output data representing a second discrete occupancy representation for the object at the second time;
and
generating, by a track component and based at least in part on the first output data and the second output data, track data including a discrete track for the object over at least the first time and the second time.

2 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/103,964Filed 31 Jan 2023
Published: 1 Aug 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

B60W 30/095
Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences: predicting travel path or…
B60W 30/18154
Related to particular drive situations: approaching an intersection
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks
G06N 3/047
Architecture, e.g. interconnection topology: probabilistic or stochastic networks

Family

US 12,286,104 B2Related patent
US 2024/0253620 A1This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key