# US 11,299,150 B2 · Using wheel orientation to determine future heading

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,299,150 B2
- Document ID: US11299150B2
- Status: Granted
- Kind: B2
- Application number: 16/790,818
- Filed: 2020-02-14
- Published: 2020-08-06
- Granted: 2022-04-12
- Earliest claimed: 2017-02-10
- Assignees: Waymo LLC
- Applicants: Waymo LLC
- Inventors: Jens-Steffen Ralf Gutmann
- Web page: https://patentgrep.com/patent/US11299150B2

## CPC

- B60W 30/0956: {the prediction being responsive to traffic or environmental parameters}
- B60W 10/04: including control of propulsion units
- B60W 10/20: including control of steering systems
- B60W 40/04: Traffic conditions
- B60W 50/0097: {Predicting future conditions}
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G01S 13/931: of land vehicles
- G01S 15/931: of land vehicles
- G01S 17/931: of land vehicles
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

The technology relates to determining a future heading of an object. In order to do so, sensor data, including information identifying a bounding box representing an object in a vehicle's environment and locations of sensor data points corresponding to the object, may be received. Based on dimensions of the bounding box, an area corresponding to a wheel of the object may be identified. An orientation of the wheel may then be estimated based on the sensor data points having locations within the area. The estimation may then be used to determine a future heading of the object.

## Claims

### Claim 1

A method of determining a future behavior of an object, the method comprising:
receiving, by one or more processors, sensor data including information identifying an object in a vehicle's environment and sensor data corresponding to the object;
identifying, by the one or more processors, an area corresponding to an expected location of a wheel of the object based on a type of the object;
identifying, by the one or more processors, a portion of the sensor data that is within the area; and
determining, by the one or more processors, a future behavior of the object based on the portion of the sensor data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sensor data includes laser sensor data points.

### Claim 3 (depends on 1)

The method of claim 1, wherein the sensor data includes a camera image.

### Claim 4 (depends on 1)

The method of claim 1, wherein identifying the portion of the sensor data includes using a three-dimensional model estimation from two or more images of the wheel.

### Claim 5 (depends on 1)

The method of claim 1, wherein the sensor data includes a camera image and wherein identifying the portion of the sensor data includes using three-dimensional model data generated for the camera image using laser data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the type of the object includes one of a passenger vehicle, a tractor trailer, a bus or a bicycle.

### Claim 7 (depends on 1)

The method of claim 1, wherein the future behavior includes the object changing lanes.

### Claim 8 (depends on 1)

The method of claim 1, wherein the future behavior includes the object moving from a stationary position.

### Claim 9 (depends on 1)

The method of claim 1, wherein the expected location of the wheel corresponds to a two-dimensional area.

### Claim 10 (depends on 1)

The method of claim 1, wherein the portion of the sensor data includes sensor data outside of a bounding box for the object.

### Claim 11 (depends on 1)

The method of claim 1, wherein determining the future behavior includes determining an aspect ratio of width and height of the wheel.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining the future behavior includes using the aspect ratio to determine an orientation of the wheel.

### Claim 13 (depends on 1)

The method of claim 1, wherein determining the future behavior includes determining an angle of the wheel relative to a side of the vehicle.

### Claim 14 (depends on 1)

The method of claim 1, wherein identifying the portion of the sensor data includes using intensity information of the sensor data to identify an outer rim of the wheel.

### Claim 15 (depends on 1)

The method of claim 1, wherein identifying the portion of the sensor data includes using a camera image to determine an area where the wheel is expected to be located, and determining the future behavior includes using laser data points to determine an orientation of the wheel based on the area.

### Claim 16 (depends on 1)

The method of claim 1, further comprising, controlling a vehicle in an autonomous driving mode based on the future behavior.

### Claim 17 (depends on 16)

The method of claim 16, wherein controlling the vehicle includes changing lanes.

### Claim 18 (depends on 16)

The method of claim 16, wherein controlling the vehicle includes increasing a speed of the vehicle.

### Claim 19 (depends on 1)

The method of claim 1, further comprising, sending, based on the future behavior, a notification to an occupant of the vehicle to take control of one or more of steering, acceleration, or deceleration of the vehicle.