# US 10,768,626 B2 · System and method for providing multiple agents for decision making, trajectory planning, and control for autonomous vehicles

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,768,626 B2
- Document ID: US10768626B2
- Status: Granted
- Kind: B2
- Application number: 15/721,781
- Filed: 2017-09-30
- Published: 2019-04-04
- Granted: 2020-09-08
- Earliest claimed: 2017-09-30
- Assignees: Tusimple, Inc.
- Applicants: TuSimple, Inc.
- Inventors: Xing Sun; Yufei Zhao; Wutu Lin; Zijie Xuan; Liu Liu; Kai-Chieh Ma
- Web page: https://patentgrep.com/patent/US10768626B2

## CPC

- G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B60W 30/00: Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W 60/001: {Planning or execution of driving tasks}
- G05D 1/20: Control system inputs
- G05D 1/81: Handing over between on-board automatic and on-board manual control
- G06F 9/00: Arrangements for program control, e.g. control units (program control for peripheral devices G06F13/10)
- B60W 2050/0006: {Digital architecture hierarchy}
- G05D 2101/10: using artificial intelligence [AI] techniques
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

A system and method for providing multiple agents for decision making, trajectory planning, and control for autonomous vehicles are disclosed. A particular embodiment includes: partitioning a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem; receiving a task request from a vehicle subsystem; dispatching the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle; causing execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of a data processor to produce a vehicle control output; and providing the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

## Claims

### Claim 1

A system comprising:
a data processor; and
a memory for storing a multiple agent autonomous vehicle control module, executable by the data processor, the multiple agent autonomous vehicle control module being a data processing module configured to perform an autonomous vehicle control operation for an autonomous vehicle, the multiple agent autonomous vehicle control module being partitioned into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem, the fast response vehicle control subsystem configured to preempt the deep computing vehicle control subsystem if the fast response vehicle control subsystem and the deep computing vehicle control subsystem are executed by the same data processor, the autonomous vehicle control operation being configured to:
receive a task request from a vehicle subsystem;
determine, by use of the data processor, if the task request is appropriate for the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle;
dispatch, by use of the data processor, the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the determination;
cause execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of the data processor to produce a vehicle control output; and
provide the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

### Claim 2 (depends on 1)

The system of claim 1 being further configured to receive sensor data from a plurality of sensors on the autonomous vehicle.

### Claim 3 (depends on 1)

The system of claim 1 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem by use of the data processor is performed in parallel.

### Claim 4 (depends on 1)

The system of claim 1 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed by different data processors.

### Claim 5 (depends on 1)

The system of claim 1 wherein the multiple agent autonomous vehicle control module being further partitioned into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, a trajectory generation subsystem, and a vehicle control subsystem.

### Claim 6 (depends on 1)

The system of claim 1 wherein the multiple agent autonomous vehicle control module being further partitioned into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, a trajectory generation subsystem, and a vehicle control subsystem, the execution of the decision making subsystem, the trajectory generation subsystem, and the vehicle control subsystem by use of the data processor is performed in parallel.

### Claim 7 (depends on 1)

The system of claim 1 wherein the vehicle control output is provided to a vehicle control subsystem causing the autonomous vehicle to follow a routing and perform vehicle motion control operations corresponding to the vehicle control output.

### Claim 8

A method comprising:
partitioning a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem, the multiple agent autonomous vehicle control module being a data processing module storable in a memory and executable by a data processor, the fast response vehicle control subsystem configured to preempt the deep computing vehicle control subsystem if the fast response vehicle control subsystem and the deep computing vehicle control subsystem are executed by the same data processor;
receiving a task request from a vehicle subsystem;
determining, by use of the data processor, if the task request is appropriate for the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle;
dispatching, by use of the data processor, the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the determination;
causing execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of the data processor to produce a vehicle control output; and
providing the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

### Claim 9 (depends on 8)

The method of claim 8 including receiving sensor data from a plurality of sensors on the autonomous vehicle, the sensors being from the group consisting of: an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, a laser range finder/LIDAR unit, and one or more cameras or image capture devices.

### Claim 10 (depends on 8)

The method of claim 8 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed in parallel by different data processors.

### Claim 11 (depends on 8)

The method of claim 8 wherein the completion of a task by execution of the deep computing vehicle control subsystem occurs more slowly than the completion of a task by execution of the fast response vehicle control subsystem.

### Claim 12 (depends on 8)

The method of claim 8 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed by the same data processor.

### Claim 13 (depends on 8)

The method of claim 8 including partitioning the multiple agent autonomous vehicle control module into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, and a trajectory generation subsystem, the execution of the decision making subsystem, and the trajectory generation subsystem by use of the data processor is performed in parallel.

### Claim 14 (depends on 8)

The method of claim 8 wherein the vehicle control output is provided to a vehicle control subsystem causing the autonomous vehicle to follow a routing and perform vehicle motion control operations corresponding to the vehicle control output, the vehicle control subsystem including a steering and throttle control subsystem.

### Claim 15

A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
partition a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem, the multiple agent autonomous vehicle control module being a data processing module storable in a memory and executable by a data processor, the fast response vehicle control subsystem configured to preempt the deep computing vehicle control subsystem if the fast response vehicle control subsystem and the deep computing vehicle control subsystem are executed by the same data processor;
receive a task request from a vehicle subsystem;
determine if the task request is appropriate for the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle;
dispatch the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the determination;
cause execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of the data processor to produce a vehicle control output; and
provide the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

### Claim 16 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 including receiving sensor data from a plurality of sensors on the autonomous vehicle, the sensors including one or more cameras or image capture devices.

### Claim 17 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed by the same data processor.

### Claim 18 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 wherein the plurality of subsystem agents including a deep computing decision making subsystem and a fast response decision making subsystem.

### Claim 19 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 including partitioning the multiple agent autonomous vehicle control module into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, a trajectory generation subsystem, and a vehicle control subsystem.

### Claim 20 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 wherein the vehicle control output is provided to a vehicle control subsystem causing the autonomous vehicle to follow a routing and perform vehicle motion control operations corresponding to the vehicle control output, the vehicle control subsystem including a braking and navigation control subsystem.