# US 2022/0032935 A1 · System and method for managing flexible control of vehicles by diverse agents in autonomous driving simulation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0032935 A1
- Document ID: US20220032935A1
- Status: Published application
- Kind: A1
- Application number: 16/989,776
- Filed: 2020-08-10
- Published: 2022-02-03
- Earliest claimed: 2020-07-28
- Assignees: Huawei Technologies Co., Ltd.
- Inventors: Jun LUO; Julian VILLELLA; Mohsen ROHANI; David RUSU; Montgomery ALBAN; Seyed Ershad BANIJAMALI
- Web page: https://patentgrep.com/patent/US20220032935A1

## CPC

- B60W 30/18154: {Approaching an intersection}
- B60W 50/082: {Selecting or switching between different modes of propelling}
- B60W 30/18159: {Traversing an intersection}
- B60W 40/06: Road conditions
- B60W 50/085: {Changing the parameters of the control units, e.g. changing limit values, working points by control input}
- B60W 60/001: {Planning or execution of driving tasks}
- B60W 60/0027: {using trajectory prediction for other traffic participants}
- G06F 30/15: Vehicle, aircraft or watercraft design
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G08G 1/0129: {for creating historical data or processing based on historical data}
- G08G 1/165: {for passive traffic, e.g. including static obstacles, trees}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data
- G06N 20/00: Machine learning
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

Method and system for controlling the behavior of an object. Behavior of the object is controlled during a first time period by using a first agent that applies a first behavior policy to map observations about the object and the environment in the first time period to a corresponding control action. Control is transitioned from the first agent to a second agent during a transition period following the first time period. Behavior of the object during a second time period following the transition period is controlled by using a second agent that applies a second behavior policy to map observations about the object and the environment in the second time period to a corresponding control action that is applied to the object. During transition the first agent applies the first behavior policy control the object and the second agent applies the second behavior policy to map observations about the object and the environment to corresponding control actions that are not applied to the object.

## Claims

### Claim 1

A computer implemented method for controlling the behavior of an object, comprising:
controlling the behavior of the object during a first time period by using a first agent that applies a first behavior policy to map observations about the object and the environment in the first time period to a corresponding control action that is applied to the object;
transitioning control of the behavior of the object from the first agent to a second agent during a transition period following the first time period; and
controlling the behavior of the object during a second time period following the transition period by using a second agent that applies a second behavior policy to map observations about the object and the environment in the second time period to a corresponding control action that is applied to the object;
wherein during the transition period the first agent applies the first behavior policy to map observations about the object and the environment in the transition period to a corresponding control action that is applied to the object and the second agent applies the second behavior policy to map observations about the object and the environment in the transition period to a corresponding control action that is not applied to the object.

### Claim 2 (depends on 1)

The method of claim 1 wherein the observations mapped by the first behavior policy and the observations mapped by the second behavior policy are each from respective, different, observation spaces.

### Claim 3 (depends on 1)

The method of claim 1 wherein during the transition period a set of observations about the object and the environment is modified to include observations about the object and the environment required by the second behavior policy.

### Claim 4 (depends on 1)

The method of claim 1 wherein the first time period corresponds to a time that the object is present in a first zone defined by a first spatiotemporal boundary, the second time period corresponds to a time when the object is present in a second zone defined by a second spatiotemporal boundary, and the transition period corresponds to a time when the object is present in a transition zone between the first zone and the second zone, the method including performing the transitioning upon detecting presence of the object in the transition zone following presence of the object in the first zone.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
transitioning control of the behavior of the object from the second agent to the first agent during a further transition period following the second time period; and
wherein during the further transition period the second agent applies the second behavior policy to map observations about the object and the environment in the second transition period to a corresponding control action that is applied to the object and the first agent applies the first behavior policy to map observations about the object and the environment in the further transition period to a corresponding control action that is not applied to the object.

### Claim 6 (depends on 1)

The method of claim 1 wherein the method is applied during a simulation run, the object is a simulated object and the observations about the object and the environment are simulated observations about the object and the environment.

### Claim 7 (depends on 6)

The method of claim 6 wherein the object is a simulated social vehicle operating in a simulated environment that also includes a simulated ego vehicle that is controlled throughout the first time period, transition period and second time period by a respective ego agent that applies an ego behavior policy to map ongoing observations about the ego vehicle and the environment to corresponding ego vehicle control actions that are applied to the ego vehicle.

### Claim 8 (depends on 7)

The method of claim 7 wherein the first time period corresponds to a time that the object is present in a first zone defined by a first spatiotemporal boundary, the second time period corresponds to a time when the object is present in a second zone defined by a second spatiotemporal boundary, and the transition period corresponds to a time when the object is present in a transition zone between the first bubble and the second bubble, the method including performing the transitioning upon detecting presence of the object in the transition zone following presence of the object in the first zone, and wherein the second zone and the transition zone are fixed in a virtual position that moves with the virtual location of the simulated ego vehicle within the simulated environment.

### Claim 9 (depends on 7)

The method of claim 7 wherein the first time period corresponds to a time that the object is present in a first zone defined by a first spatiotemporal boundary, the second time period corresponds to a time when the object is present in a second zone defined by a second spatiotemporal boundary, and the transition period corresponds to a time when the object is present in a transition zone between the first zone and the second zone, the method including performing the transitioning upon detecting presence of the object in the transition zone following presence of the object in the first zone, and wherein the second zone and the transition zone are fixed in a virtual position that is stationary with a virtual physical location within the simulated environment.

### Claim 10 (depends on 1)

The method of claim 1 wherein the first behavior policy is less computationally intensive than the second behavior policy.

### Claim 11 (depends on 10)

The method of claim 10 wherein the second behavior policy is configured to map observations about the object and the environment from an observation space that is enriched relative to an observation space that the first behavior policy is configured to map observations from.

### Claim 12 (depends on 11)

The method of claim 11 wherein the second behavior policy is configured to map observations about the object and the environment to control actions from an action space that is enriched relative to an action space that the first behavior policy is configured to map observations about the object and the environment to control actions from.

### Claim 13

A computer system comprising:
a processor; and
a non-transitory memory coupled to the processor, the memory storing instructions that, when executed by the processor, configure the computer system to:
control the behavior of the object during a first time period by using a first agent that applies a first behavior policy to map observations about the object and the environment in the first time period to a corresponding control action that is applied to the object;
transition control of the behavior of the object from the first agent to a second agent during a transition period following the first time period; and
control the behavior of the object during a second time period following the transition period by using a second agent that applies a second behavior policy to map observations about the object and the environment in the second time period to a corresponding control action that is applied to the object;
wherein during the transition period the first agent applies the first behavior policy to map observations about the object and the environment in the transition period to a corresponding control action that is applied to the object and the second agent applies the second behavior policy to map observations about the object and the environment in the transition period to a corresponding control action that is not applied to the object.

### Claim 14 (depends on 13)

The computer system of claim 13 wherein the observations mapped by the first behavior policy and the observations mapped by the second behavior policy are each from respective, different, observation spaces.

### Claim 15 (depends on 13)

The computer system of claim 13 wherein during the transition period a set of observations about the object and the environment are modified to include observations about the object and the environment required by the second behavior policy.

### Claim 16 (depends on 13)

The computer system of claim 13 wherein the first time period corresponds to a time that the object is present in a first zone defined by a first spatiotemporal boundary, the second time period corresponds to a time when the object is present in a second zone defined by a second spatiotemporal boundary, and the transition period corresponds to a time when the object is present in a transition zone between the first zone and the second zone, and further including performing the transitioning upon detecting presence of the object in the transition zone following presence of the object in the first zone.

### Claim 17 (depends on 13)

The computer system of claim 13 wherein the memory stores further instructions that, when executed by the processor, configure the computer system to transition control of the behavior of the object from the second agent to the first agent during a further transition period following the second time period; and wherein during the further transition period the second agent applies the second behavior policy to map observations about the object and the environment in the second transition period to a corresponding control action that is applied to the object and the first agent applies the first behavior policy to map observations about the object and the environment in the further transition period to corresponding control actions that are not applied to the object.

### Claim 18 (depends on 13)

The computer system of claim 13 wherein the instructions, when executed by the processor, configure the computer system to perform a simulation run, wherein the object is a simulated object and the observations about the object and the environment are simulated observations about the object and the environment.

### Claim 19 (depends on 18)

The method of claim 18 wherein the object is a simulated social vehicle operating in a simulated environment that also includes a simulated ego vehicle that is controlled throughout the first time period, transition period and second time period by a respective ego agent that applies an ego behavior policy to map ongoing observations about the ego vehicle and the environment to corresponding ego vehicle control actions that are applied to the ego vehicle.

### Claim 20

A computer program product comprising a non-transitory computer medium storing instructions for configuring a computer system to:
control the behavior of the object during a first time period by using a first agent that applies a first behavior policy to map observations about the object and the environment in the first time period to a corresponding control action that is applied to the object;
transition control of the behavior of the object from the first agent to a second agent during a transition period following the first time period; and
control the behavior of the object during a second time period following the transition period by using a second agent that applies a second behavior policy to map observations about the object and the environment in the second time period to a corresponding control action that is applied to the object;
wherein during the transition period the first agent applies the first behavior policy to map observations about the object and the environment in the transition period to a corresponding control action that is applied to the object and the second agent applies the second behavior policy to map observations about the object and the environment in the transition period to a corresponding control action that is not applied to the object.