# US 11,906,967 B1 · Determining yaw with learned motion model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,906,967 B1
- Document ID: US11906967B1
- Status: Granted
- Kind: B1
- Application number: 16/836,568
- Filed: 2020-03-31
- Granted: 2024-02-20
- Earliest claimed: 2020-03-31
- Assignees: Zoox, Inc.
- Applicants: Zoox, Inc.
- Inventors: Subhasis Das; Francesco Papi; Shida Shen
- Web page: https://patentgrep.com/patent/US11906967B1

## CPC

- B62D 15/025: {Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation (B62D1/28 takes precedence)}
- G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path
- B60W 60/0027: {using trajectory prediction for other traffic participants}
- B62D 15/0265: {Automatic obstacle avoidance by steering}
- G05D 1/247: using signals provided by artificial sources external to the vehicle, e.g. navigation beacons
- G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras
- G05D 1/628: following the obstacle profile, e.g. a wall or undulated terrain
- B62D 6/003: {in order to control vehicle yaw movement, i.e. around a vertical axis (B62D6/007 takes precedence)}
- B62D 6/005: {treating sensor outputs to obtain the actual yaw rate}
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

Techniques to use a trained model to determine a yaw of an object are described. For example, a system may implement various techniques to generate multiple representations for an object in an environment. Each representation vary based on the technique and data used. An estimation component may estimate a representation from the multiple representations. The model may be implemented to output a yaw for the object using the multiple representations, the estimated representation, and/or additional information. The output yaw may be used to track an object, generate a trajectory, or otherwise control a vehicle.

## Claims

### Claim 1

A method comprising:
receiving sensor data from one or more sensors associated with an autonomous vehicle, the sensor data representative of an object in an environment;
determining, based at least in part on a first portion of the sensor data and using a first technique, a first representation of the object, the first representation including a first measured yaw for the object;
determining, based at least in part on a second portion of the sensor data and using a second technique different than the first technique, a second representation of the object, the second representation including a second measured yaw for the object;
determining, based at least in part on the first measured yaw and the second measured yaw, an estimated yaw for the object;
inputting the first measured yaw, the second measured yaw, and the estimated yaw into a machine learned model;
receiving, from the machine learned model, an output yaw for the object; and
controlling the autonomous vehicle relative to the object based at least in part on the output yaw.

### Claim 2 (depends on 1)

The method of claim 1, wherein the inputting further comprises inputting additional data into the machine learned model, the additional data comprising at least one of:
a distance of the object to the autonomous vehicle;
a classification of the object;
information about a road segment associated with the object;
a first confidence associated with the first measured yaw;
a second confidence associated with the second measured yaw; or
a covariance associated with the estimated yaw.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving additional sensor data, the additional sensor data comprising additional instances of the sensor data;
determining, based at least in part on the additional sensor data and with the first technique, additional instances of the first measured yaw;
determining, based at least in part on the additional sensor data and with the second technique, additional instances of the second measured yaw;
determining, based at least in part on the additional instances of the first measured yaw and the additional instances of the second measured yaw, additional instances of the estimated yaw; and
inputting the additional instances of the first measured yaw, the additional instances of the second measured yaw, and the additional instances of the estimated yaw into the machine learned model with the first measured yaw, the second measured yaw, and the estimated yaw.

### Claim 4 (depends on 1)

The method of claim 1, wherein the determining the estimated yaw comprises using a Bayesian filter.

### Claim 5 (depends on 1)

The method of claim 1, wherein the inputting further comprises inputting a first confidence associated with the first measured yaw and a second confidence associated with the second measured yaw.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first portion of the sensor data is generated by a first sensor and the second portion of the sensor data is generated by a second sensor different than the first sensor.

### Claim 7

A system comprising:
one or more processors; and
memory storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving sensor data from one or more sensors;
determining, based at least in part on a first portion of the sensor data and using a first technique, a first measured yaw associated with an object in an environment;
determining, based at least in part on a second portion of the sensor data and using a second technique different than the first technique, a second measured yaw associated with the object;
determining, based at least in part on the first measured yaw and the second measured yaw, an estimated yaw associated with the object;
inputting the first measured yaw, the second measured yaw, and the estimated yaw into a machine learned model;
receiving, from the machine learned model, an output yaw; and
determining a track of the object in the environment based at least in part on the output yaw.

### Claim 8 (depends on 7)

The system of claim 7, wherein:
the sensor data comprises first sensor data from a first sensor modality and second sensor data from a second sensor modality;
the first measured yaw is determined based at least in part on the first sensor data; and
the second measured yaw is determined based at least in part on the second sensor data.

### Claim 9 (depends on 8)

The system of claim 8, wherein the first sensor data and the second sensor data comprise one or more of LiDAR data, image data, or radar data.

### Claim 10 (depends on 7)

The system of claim 7, wherein the sensor data is associated with a first time, the operations further comprising:
receiving additional sensor data associated with one or more additional times;
determining, based at least in part on the additional sensor data, one or more additional instances of the measured yaw;
determining, based at least in part on the one or more additional instances of the measured yaw, one or more additional instances of the estimated yaw; and
inputting the one or more additional instances of the measured yaw and the one or more additional instances of the estimated yaw into the machine learned model with the measured yaw and the estimated yaw.

### Claim 11 (depends on 7)

The system of claim 7, the operations further comprising:
inputting map information into the machine learned model, the map information comprising at least one of an identification of a road segment proximate the object or a yaw of the road segment.

### Claim 12 (depends on 7)

The system of claim 7, the operations further comprising:
inputting, into the machine learned model, one or more of a classification of the object, a distance to the object, a first confidence associated with the measured yaw, or a second confidence associated with the estimated yaw.

### Claim 13 (depends on 7)

The system of claim 7, wherein the determining the estimated yaw is based at least in part on using a Bayesian filter and the Bayesian filter further determines a first covariance.

### Claim 14 (depends on 13)

The system of claim 13, the operations further comprising:
receiving, from the machine learned model, a second covariance associated with the estimated yaw, wherein the second covariance is different from the first covariance.

### Claim 15 (depends on 7)

The system of claim 7, wherein determining the track comprises:
updating, as an updated track, previous track based at least in part on the output yaw, the updated track comprising one or more of historical positions, historical velocities, historical orientations, or historical accelerations of the object, the operations further comprising:
determining a trajectory relative to the object based at least in part on the updated track; and
controlling an autonomous vehicle based at least in part on the trajectory.

### Claim 16

One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform acts comprising:
receiving sensor data from one or more sensors associated with a vehicle;
determining, based at least in part on a first portion of the sensor data and using a first technique, a first measured yaw associated with an object in an environment;
determining, based at least in part on a second portion of the sensor data and using a second technique different than the first technique, a second measured yaw associated with the object;
determining, based at least in part on the first measured yaw and the second measured yaw, an estimated yaw associated with the object;
inputting the first measured yaw, the second measured yaw, and the estimated yaw into a machine learned model;
receiving, from the machine learned model, an output yaw; and
determining a track of the object.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, wherein the sensor data is associated with a first time, the acts further comprising:
receiving additional sensor data associated with one or more additional times;
determining, based at least in part on the additional sensor data, one or more additional instances of the first measured yaw and one or more additional instances of the second measured yaw;
determining, based at least in part on the one or more additional instances of the first measured yaw and the one or more additional instances of the second measured yaw, one or more additional instances of the estimated yaw; and
inputting the one or more additional instances of the first measured yaw, the one or more additional instances of the second measured yaw, and the one or more additional instances of the estimated yaw into the machine learned model with the first measured yaw, the second measured yaw, and the estimated yaw.

### Claim 18 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, the acts further comprising:
inputting, into the machine learned model, one or more of a classification of the object, a distance of the object to the vehicle, a first confidence associated with the first measured yaw, a second confidence associated with the second measured yaw, or a third confidence associated with the estimated yaw.

### Claim 19 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, the acts further comprising:
inputting map information into the machine learned model, the map information comprising at least one of an identification of a road segment proximate the object or a yaw of the road segment.

### Claim 20 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, wherein the determining the estimated yaw is based at least in part on using a Bayesian filter.
