# US 2023/0067538 A1 · Autonomous vehicle maneuver in response to emergency personnel hand signals

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0067538 A1
- Document ID: US20230067538A1
- Status: Published application
- Kind: A1
- Application number: 17/823,698
- Filed: 2022-08-31
- Published: 2023-03-02
- Earliest claimed: 2021-09-01
- Assignees: Tusimple, Inc.
- Applicants: TuSimple, Inc.
- Inventors: Hunter Scott Willoughby; Scott Douglas Foster; Dishi Li; Mohammad Poorsartep
- Web page: https://patentgrep.com/patent/US20230067538A1

## CPC

- B60W 60/0015: {specially adapted for safety}
- B60W 60/001: {Planning or execution of driving tasks}
- B60W 60/00186: {related to the vehicle}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 40/113: {Recognition of static hand signs}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G08G 1/09: Arrangements for giving variable traffic instructions ({railroad crossing signals B61L; reflectors E01F, G08B}; indicating arrangements for variable information by selection or combination of individual elements G09F9/00)
- G08G 1/0955: transportable
- G08G 1/09623: {Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence)}
- B60W 2300/145: Semi-trailers
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W 2552/50: Barriers
- B60W 2554/402: Type
- B60W 2554/4026: Cycles
- B60W 2554/4029: Pedestrians
- B60W 2554/4044: Direction of movement, e.g. backwards
- B60W 2554/406: Traffic density
- B60W 2555/00: Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
- B60W 2555/60: Traffic rules, e.g. speed limits or right of way
- B60W 2556/40: High definition maps
- B60W 2556/45: External transmission of data to or from the vehicle
- B60W 2556/65: Data transmitted between vehicles
- B60W 2720/10: Longitudinal speed
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A control device associated with an autonomous vehicle detects that an emergency personnel is altering traffic on a road using an emergency-related hand signal to divert the traffic from a road anomaly, such as a road accident. The control device determines an interpretation of the emergency-related hand signal. The control device determines a proposed trajectory for the autonomous vehicle according to the interpretation of the emergency-related hand signal. In certain embodiments, the control device may navigate the autonomous vehicle according to the interpretation of the emergency-related hand signal. In certain embodiments, the control device may transmit the proposed trajectory to an oversight server for confirmation. In certain embodiments, the oversight may confirm or override the proposed trajectory.

## Claims

### Claim 1

A system comprising:
an autonomous vehicle configured to travel along a road, wherein the autonomous vehicle comprises at least one sensor;
a control device associated with the autonomous vehicle and comprising a first processor configured to:
access sensor data captured by the at least one sensor, wherein the sensor data provides information about at least a portion of an area in front of the autonomous vehicle;
detect, from the sensor data, a road anomaly comprising one of a road accident, a road closure, or congested traffic;
detect, from the sensor data, that an emergency personnel is altering a traffic flow using an emergency-related hand signal, wherein the emergency personnel is on a traffic lane adjacent to the road anomaly and facing oncoming traffic;
determine an interpretation of the emergency-related hand signal;
determine a proposed trajectory for the autonomous vehicle according to the interpretation of the emergency-related hand signal; and
navigate the autonomous vehicle according to the proposed trajectory.

### Claim 2 (depends on 1)

The system of claim 1, wherein the emergency personnel is a law enforcement officer, a firefighter, or a paramedic.

### Claim 3 (depends on 1)

The system of claim 1, wherein detecting, from the sensor data, that the emergency personnel is altering the traffic flow using the emergency-related hand signal comprises determining that the emergency personnel is wearing a law enforcement uniform, a firefighter uniform, or a paramedic uniform.

### Claim 4 (depends on 1)

The system of claim 1, wherein:
the emergency-related hand signal is hand motions or waving a flag that indicate all vehicles stop; and
the proposed trajectory follows the interpretation of the emergency-related hand signal, such that if the interpretation of the emergency-related hand signal is all vehicles stop, the proposed trajectory is stopping.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the emergency-related hand signal is hand motions or waving a flag that indicate all vehicles proceed; and
the proposed trajectory follows the interpretation of the emergency-related hand signal, such that if the interpretation of the emergency-related hand signal is all vehicles proceed, the proposed trajectory is moving forward.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the emergency-related hand signal is hand motions or waving a flag that indicate vehicles on right, front, and behind to stop, and vehicles on a left side to proceed;
the proposed trajectory follows the interpretation of the emergency-related hand signal, such that if the interpretation of the emergency-related hand signal is vehicles on right, front, and behind of the emergency personnel to stop, and vehicles on the left side to proceed:
the proposed trajectory is moving forward if the autonomous vehicle is on the left side of the emergency personnel; and
the proposed trajectory is stopping if the autonomous vehicle is on any of the right, front, and behind of the emergency personnel.

### Claim 7 (depends on 1)

The system of claim 1, wherein determining the interpretation of the emergency-related hand signal comprises:
accessing a training dataset comprising a plurality of images, wherein a respective image, from among the plurality of images, is labeled with an interpretation of a hand signal shown in the respective image;
extracting a first set of features from the sensor data where the emergency-related hand signal is detected, wherein:
the first set of features indicates a type of the emergency-related hand signal;
the first set of features is represented by a first vector comprising numerical values;
extracting a second set of features from an image of the plurality of images, wherein:
the image shows a particular hand signal;
the image is labeled with a particular interpretation of the particular hand signal;
the second set of features indicates a type of the particular hand signal;
the second set of features is represented by a second vector comprising numerical values;
determining a distance between the first vector and the second vector; and
in response to determining that the distance between the first vector and the second vector is less than a threshold percentage, determining that the interpretation of the emergency-related hand signal corresponds to the particular interpretation of the particular hand signal.

### Claim 8 (depends on 1)

The system of claim 1, wherein detecting, from the sensor data, the road anomaly comprises:
extracting a set of features from the sensor data, wherein the set of features indicates physical attributes of objects indicated in the sensor data; and
determining that the objects include an object associated with the road anomaly, wherein the object comprises a traffic cone or a traffic barrier.

### Claim 9 (depends on 1)

The system of claim 1, wherein the first processor is further configured to transmit the proposed trajectory to one or more other autonomous vehicles.

### Claim 10 (depends on 1)

The system of claim 1, wherein the sensor data comprises at least one of a camera video, a camera image data, and a light detection and ranging (LiDAR) cloud data.

### Claim 11

A system comprising:
an autonomous vehicle configured to travel along a road, wherein the autonomous vehicle comprises at least one sensor;
a control device associated with the autonomous vehicle and comprising a first processor configured to:
access sensor data captured by the at least one sensor, wherein the sensor data provides information about at least a portion of an area in front of the autonomous vehicle;
detect, from the sensor data, a road anomaly comprising one of a road accident, a road closure, or congested traffic;
detect, from the sensor data, that an emergency personnel is altering a traffic flow using an emergency-related hand signal, wherein the emergency personnel is on a traffic lane adjacent to the road anomaly and facing oncoming traffic;
determine an interpretation of the emergency-related hand signal;
determine a proposed trajectory for the autonomous vehicle according to the interpretation of the emergency-related hand signal; and
transmit at least one of the proposed trajectory and the sensor data to an oversight server;
the oversight server operably coupled with the control device, and comprising a second processor configured to:
receive the at least one of the proposed trajectory and the sensor data;
determine whether the emergency-related hand signal is in use to alter the traffic flow;
in response to determining that the emergency-related hand signal is in use to alter the traffic flow, determine whether the proposed trajectory causes the autonomous vehicle to go outside of an operational design domain that indicates pre-mapped geographical areas where the autonomous vehicle is able to autonomously travel; and
in response to determining that the proposed trajectory does not cause the autonomous vehicle to go outside of the operational design domain, transmit, to the control device, an instruction that indicates to perform the proposed trajectory.

### Claim 12 (depends on 11)

The system of claim 11, wherein the first processor is further configured to:
receive the instruction from the oversight server; and
navigate the autonomous vehicle according to the proposed trajectory.

### Claim 13 (depends on 11)

The system of claim 11, wherein the second processor is further configured, in response to determining that the emergency-related hand signal is not in use to alter the traffic flow, to:
determine a second proposed trajectory for the autonomous vehicle; and
transmit the second proposed trajectory to the control device.

### Claim 14 (depends on 13)

The system of claim 13, wherein the first processor is further configured to:
receive the second proposed trajectory from the oversight server; and
navigate the autonomous vehicle according to the second proposed trajectory.

### Claim 15 (depends on 11)

The system of claim 11, wherein the second processor is further configured, in response to determining that the proposed trajectory causes the autonomous vehicle to go outside of the operational design domain, to:
determine a third proposed trajectory for the autonomous vehicle; and
transmit the third proposed trajectory to the control device.

### Claim 16 (depends on 15)

The system of claim 15, wherein the first processor is further configured to:
receive, from the oversight server, the third proposed trajectory; and
navigate the autonomous vehicle according to the third proposed trajectory.

### Claim 17 (depends on 11)

The system of claim 11, wherein the second processor is further configured to transmit the proposed trajectory to one or more other autonomous vehicles.

### Claim 18 (depends on 11)

The system of claim 11, wherein determining whether the emergency-related hand signal is in use to alter the traffic flow comprises:
accessing map data that comprises at least a portion of a map of a city that includes the road;
determining, from the map data, that the autonomous vehicle is traveling within a particular area where is known hand signals are used to control traffic, wherein the particular area comprises one of a school road crossing area, a construction area, or a road accident area; and
prioritizing an analysis of the sensor data for hand signal detection, wherein the sensor data is captured when the autonomous vehicle is traveling within the particular area.

### Claim 19 (depends on 18)

The system of claim 18, wherein determining whether the emergency-related hand signal is in use to alter the traffic flow comprises:
determining that the autonomous vehicle is traveling within the particular area during a particular time window, wherein the particular time window comprises one of active hours of a construction site, school opening hours, or school closing hours; and
prioritizing an analysis of the sensor data for hand signal detection, wherein the sensor data is captured during the particular time window.

### Claim 20 (depends on 11)

The system of claim 11, wherein the second processor is further configured to:
communicate the sensor data to a third party;
communicate the proposed trajectory to the third party;
receive an input from the third party regarding one or more traveling parameters to prioritize, wherein the one or more traveling parameters comprise a speed, a fuel-saving parameter, or maintaining an originally planned route;
update the proposed trajectory based on the received input; and
transmit the updated trajectory to the control device.
