# US 2021/0064889 A1 · Detecting potentially occluded objects for autonomous vehicles

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0064889 A1
- Document ID: US20210064889A1
- Status: Published application
- Kind: A1
- Application number: 16/552,426
- Filed: 2019-08-27
- Published: 2021-03-04
- Earliest claimed: 2019-08-27
- Assignees: Waymo LLC
- Applicants: Waymo LLC
- Inventors: Jennifer Iglesias
- Web page: https://patentgrep.com/patent/US20210064889A1

## CPC

- B60W 60/0027: {using trajectory prediction for other traffic participants}
- B60W 30/146: {Speed limiting}
- B60W 30/18154: {Approaching an intersection}
- B60W 50/0097: {Predicting future conditions}
- B60W 60/0017: {of other traffic participants}
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G05D 1/0223: {involving speed control of the vehicle (vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00)}
- G05D 1/622: Obstacle avoidance (predicting or avoiding probable or impending collision of road vehicles B60W30/08)
- G05D 1/65: Following a desired speed profile
- G06V 10/273: {removing elements interfering with the pattern to be recognised}
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- B60W 2552/53: Road markings, e.g. lane marker or crosswalk
- B60W 2554/00: Input parameters relating to objects
- B60W 2554/4029: Pedestrians
- B60W 2720/10: Longitudinal speed
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

Aspects of the disclosure relate to controlling a vehicle having an autonomous driving mode. For instance, that the vehicle is approaching a crosswalk may be determined. A set of segments may be identified for the crosswalk. A set of potential occluded pedestrians may be generated. Each potential occluded pedestrian of the set is assigned a speed characteristic and a segment. The segments of the set of potential occluded pedestrians may be updated over time using the assigned speed characteristics. Sensor data from a perception system of the vehicle is received, and one or more potential occluded pedestrians an having an updated assigned segment corresponding to a segment that is visible to a perception system of the vehicle may be removed from the set of potential occluded pedestrians. After the removing, the set may be used to control the vehicle in the autonomous driving mode.

## Claims

### Claim 1

A method of controlling a vehicle having an autonomous driving mode, the method comprising:
determining, by one or more processors, that the vehicle is approaching a crosswalk;
identifying, by the one or more processors, a set of segments for the crosswalk;
generating, by the one or more processors, a set of potential occluded pedestrians, each potential occluded pedestrian of the set being assigned a speed characteristic defining a speed of the potential occluded pedestrian and an assigned segment of the set of potential occluded pedestrians;
updating the assigned segments of the set of potential occluded pedestrians over time using the assigned speed characteristics;
receiving, by the one or more processors, sensor data from a perception system of the vehicle;
identifying, by the one or more processors, a segment of the set of segments that is visible to the perception system based on the sensor data;
removing, by the one or more processors, one or more potential occluded pedestrians having an updated assigned segment corresponding to the identified segment from the set of potential occluded pedestrians; and
after the removing, using, by the one or more processors, the set to control the vehicle in the autonomous driving mode.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining that the vehicle is approaching a crosswalk is based on a current route of the vehicle and map information identifying a location of the crosswalk.

### Claim 3 (depends on 1)

The method of claim 1, wherein the generating is performed when the vehicle has reached a predetermined distance.

### Claim 4 (depends on 3)

The method of claim 3, wherein the predetermined distance corresponds to a distance from the crosswalk along a current route of the vehicle.

### Claim 5 (depends on 3)

The method of claim 3, wherein the predetermined distance corresponds to a forward edge of a field of view of a perception system of the vehicle including one or more sensors.

### Claim 6 (depends on 1)

The method of claim 1, wherein identifying the segments includes segmenting an area of the crosswalk into a predetermined number of segments.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying the segments includes retrieving the segments from pre-stored map information.

### Claim 8 (depends on 1)

The method of claim 1, wherein generating the set of potential occluded pedestrians includes associating each segment with a plurality of pedestrians having different speed characteristics.

### Claim 9 (depends on 8)

The method of claim 8, wherein one of the different speed characteristics indicates that a pedestrian is stationary.

### Claim 10 (depends on 8)

The method of claim 8, wherein one of the different speed characteristics includes a pedestrian moving in a first direction towards a first edge of the crosswalk at a first speed, and wherein another of the different speed characteristics includes a pedestrian moving in a second direction opposite of the first direction towards a second edge of the crosswalk opposite of the first edge at the first speed.

### Claim 11 (depends on 1)

The method of claim 1, wherein the one or more potential occluded pedestrians are removed from the set based on a determination that the one or more potential occluded pedestrians would have left the crosswalk.

### Claim 12 (depends on 1)

The method of claim 1, wherein the one or more potential occluded pedestrians are removed from the set further based on whether an area corresponding to a current segment for that one or more potential occluded pedestrians has been visible to the perception system for a minimum number of iterations of received sensor data while the one or more potential occluded pedestrians would have occupied the current segment.

### Claim 13 (depends on 1)

The method of claim 1, wherein the one or more potential occluded pedestrians are removed from the set further based on whether an area corresponding to a current segment for that one or more potential occluded pedestrians has been visible to the perception system for at least a predetermined period of time during which the one or more potential occluded pedestrians would have occupied the current segment.

### Claim 14 (depends on 1)

The method of claim 1, wherein the one or more potential occluded pedestrians are removed from the set further based on whether a current segment for that one or more potential occluded pedestrians has been occupied by a non-pedestrian object for a minimum number of iterations of received sensor data while the one or more potential occluded pedestrians would have occupied the current segment.

### Claim 15 (depends on 1)

The method of claim 1, wherein using the set of potential occluded pedestrians to control the vehicle in the autonomous driving mode includes, generating a speed constraint based on the set of potential occluded pedestrians, the speed constraint which limits the speed of the vehicle based on a distance of the vehicle from the crosswalk to ensure that the vehicle is able to stop before reaching the crosswalk.

### Claim 16 (depends on 1)

The method of claim 1, further comprising:
determining whether one or more edges of the crosswalk are occluded; and
adding one or more additional potential occluded pedestrians to the set of potential occluded pedestrians based on the determination of whether one or more edges of the crosswalk are occluded, and wherein the removing is performed after the adding.

### Claim 17 (depends on 16)

The method of claim 16, wherein when the one or more edges of the crosswalk are not occluded, the one or more additional potential occluded pedestrians are not added to the set.

### Claim 18 (depends on 16)

The method of claim 16, wherein each of the one or more additional potential occluded pedestrians is assigned a speed characteristic.

### Claim 19 (depends on 1)

The method of claim 1, wherein using the set of potential occluded pedestrians to control the vehicle in the autonomous driving mode includes generating a speed constraint based on the set of potential occluded pedestrians, the speed constraint identifies a maximum speed limit for the vehicle when approaching the crosswalk.

### Claim 20 (depends on 19)

The method of claim 19, wherein the speed constraint is a slow region and is generated based on a distance between one of the set of potential occluded pedestrians and the vehicle as well as the assigned speed characteristic for the one of the set of potential occluded pedestrians.
