US 12,612,073 B1 · Behavior modeling based on relative position data and absolute yaw data

Abstract

Techniques for performing prediction based on relative position data and/or absolute yaw data are described herein. A vehicle may detect an object in an environment. The vehicle may generate an embedding associated with the object and input the embedding into a machine learned model. The machine learned model may add the absolute yaw of the object to the embedding, generate a rotation matrix based on the pose of the object, and apply the rotation matrix to the embedding. Based on modifying the embedding (or generating a modified embedding), the attention layer of the machine learned model may perform attention on the modified embedding and respond by outputting output data. The vehicle may rotate the output data (or generate rotated output data) which may be used by one or more machine learned models to predict object behavior, generate vehicle actions, etc.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the system to perform operations comprising:
receiving, from a sensor device of a vehicle, sensor data on an environment;
detecting, based at least in part on the sensor data, an object, the object including a pose;
generating, based at least in part on the sensor data, an embedding associated with the object;
determining, based at least in part on inputting the embedding and the pose into a machine learned model, rotated output data, wherein determining the rotated output data comprises:
generating, based at least in part on the pose, a rotation matrix;
determining, based at least in part on the rotation matrix, a rotated embedding associated with the embedding;
receiving, in response to inputting the rotated embedding into an attention layer of the machine learned model, output data;
and
generating, based at least in part on the rotation matrix and the output data, the rotated output data;
and
controlling the vehicle based at least in part on the rotated output data.

11 more elements

Dependencies are inferred from the claim text.

Behavior modeling based on relative position data and absolute yaw data

Life of this patent

Behavior modeling based on relative position data and absolute yaw data

Life of this patent

Abstract

Claims20