US 2026/0264700 A1 · CONTRASTIVE TRAINING OF OBJECT TRAJECTORY ENCODERS AND TEXT ENCODERS

Abstract

Techniques are described herein for training contrastive models including object trajectory encoders and text encoders for evaluating, classifying, and/or predicting the movements and behaviors of dynamic objects in driving environments. A training system may receive sets of ground truth trajectory data describing movements of objects within driving environments, and associated text descriptions related to the trajectory data. The training system may jointly train the trajectory encoder and the text encoder, using contrastive loss, based on the related sets of trajectory data and text data. Once trained, the trajectory encoder and/or the text encoder may operate as pre-trained models for subsequently training and executing additional models with different output heads and/or various other downstream encoding tasks. In some examples, contrastive pre-trained trajectory encoders trained as described herein may be used for training and executing motion forecasting models within autonomous vehicles.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:
receiving input data including at least one of:
trajectory data associated with a first object in an environment, or a text description associated with the first object in the environment;
and
determining, using a trained encoder and based at least in part on the input data, a first encoding, wherein:
the trained encoder comprises a trajectory encoder and the first encoding comprises a trajectory encoding;
or
the trained encoder comprises a text encoder and the first encoding comprises a text encoding,
wherein the trajectory encoder and the text encoder are jointly trained, based at least in part on a similarity between a second trajectory encoding associated with a second object and a second text encoding associated with the second object.

8 more elements

One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:

receiving input data including at least one of:
trajectory data associated with a first object in an environment, or
a text description associated with the first object in the environment;
and
determining, using a trained encoder and based at least in part on the input data, a first encoding, wherein:
the trained encoder comprises a trajectory encoder and the first encoding comprises a trajectory encoding;
or
the trained encoder comprises a text encoder and the first encoding comprises a text encoding,
wherein the trajectory encoder and the text encoder are jointly trained, based at least in part on a similarity between a second trajectory encoding associated with a second object and a second text encoding associated with the second object.

6 more elements

Dependencies are inferred from the claim text.

Contrastive training of object trajectory encoders and text encoders

Life of this patent

Contrastive training of object trajectory encoders and text encoders

Life of this patent

Abstract

Claims20