# US 11,307,585 B2 · Introspective competence modeling for AV decision making

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,307,585 B2
- Document ID: US11307585B2
- Status: Granted
- Kind: B2
- Application number: 16/668,584
- Filed: 2019-10-30
- Published: 2021-05-06
- Granted: 2022-04-19
- Earliest claimed: 2019-10-30
- Assignees: Nissan North America, Inc.; University of Massachusetts; RENAULT s.a.s.
- Applicants: Nissan North America, Inc.; The University of Massachusetts; Renault S.A.S.
- Inventors: Connor Basich; Kyle Hollins Wray; Stefan Witwicki; Shlomo Zilberstein
- Web page: https://patentgrep.com/patent/US11307585B2

## CPC

- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- B60W 60/005: {Handover processes (Handing over between remote control and on-board control or handing over between remote control arrangements G05D1/227)}
- G08G 1/163: {involving continuous checking}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- G05D 1/0061: {for transition from automatic pilot to manual pilot and vice versa}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

A first method includes detecting, based on sensor data, an environment state; selecting an action based on the environment state; determining an autonomy level associated with the environment state and the action; and performing the action according to the autonomy level. The autonomy level can be selected based at least on an autonomy model and a feedback model. A second method includes calculating, by solving an extended Stochastic Shortest Path (SSP) problem, a policy for solving a task. The policy can map environment states and autonomy levels to actions and autonomy levels. Calculating the policy can include generating plans that operate across multiple levels of autonomy.

## Claims

### Claim 1

A method of autonomous driving by an autonomous vehicle (AV), comprising:
detecting, based on sensor data, an environment state;
selecting an action based on the environment state;
determining an autonomy level associated with the environment state and the action,
wherein the autonomy level indicates an extent to which the action is to be performed autonomously by the AV,
wherein the AV determines respective autonomy levels for actions to be performed,
wherein the autonomy level is selected based at least on an autonomy model and a feedback model, and
wherein the feedback model models feedback types received from a human with respect to the action, how costly each type of feedback is, and how likely is each type of feedback to be received from the human; and
performing the action according to the autonomy level, wherein performing the action according to the autonomy level comprises:
in response to determining that the autonomy level indicates “verified autonomy” and to determining that the AV is being monitored:
performing the action; and
in response to receiving an override signal while the action is being performed, stopping the action and switching to a manual operation mode of the AV.

### Claim 2 (depends on 1)

The method of claim 1, wherein the autonomy level is selected from a set comprising a first autonomy level indicating “no autonomy”, a second autonomy level indicating “verified autonomy”, a third autonomy level indicating “supervised autonomy”, and a fourth autonomy level indicating “unsupervised autonomy”.

### Claim 3 (depends on 2)

The method of claim 2,
wherein the autonomy level is the second autonomy level indicating “verified autonomy”, and wherein performing the action according to the autonomy level comprises:
receiving, for the action, an approval feedback signal or a disapproval feedback signal.

### Claim 4 (depends on 3)

The method of claim 3, wherein performing the action according to the autonomy level comprises:
querying for the approval feedback signal prior to receiving the approval feedback signal.

### Claim 5 (depends on 3)

The method of claim 3,
wherein the autonomy level is the third autonomy level indicating “supervised autonomy”, and
wherein performing the action according to the autonomy level comprises:
determining that the AV is being monitored by a human before performing the action.

### Claim 6 (depends on 3)

The method of claim 3, further comprising:
receiving the approval feedback signal.

### Claim 7 (depends on 1)

The method of claim 1, wherein performing the action according to the autonomy level comprises:
determining, based on the autonomy level, whether to request approval from a human for the action before performing the action.

### Claim 8 (depends on 1)

The method of claim 1,
wherein the autonomy level is an “no autonomy” such that the AV is not allowed to perform autonomous actions, and
wherein performing the action according to the autonomy level comprises:
enabling the AV to be manually controlled by a human.

### Claim 9 (depends on 1)

The method of claim 1,
wherein the autonomy model comprises a utility model and an autonomy profile,
wherein utility model describes a utility of performing a first action in a first autonomy level with respect to a first environment state given that the AV just operated in second autonomy level, and
wherein the autonomy profile maps respective environment states to respective actions and prescribing constraints on allowed levels of autonomy for particular environment states.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
updating at least one of an autonomy profile, a feedback profile, or a human transition function in response to the performing the action.

### Claim 11

An autonomous vehicle (AV), comprising:
a processor configured to execute instructions to:
detect, based on sensor data, an environment state;
select an action based on the environment state;
determine an autonomy level associated with the environment state and the action,
wherein the autonomy level indicates an extent to which the action is to be performed autonomously,
wherein the AV determines respective autonomy levels for actions to be performed,
wherein the autonomy level is selected based at least on an autonomy model and a feedback model, and
wherein the feedback model models feedback types received from a human with respect to the action, how costly each type of feedback is, and how likely is each type of feedback to be received from the human; and
cause the action to be performed according to the autonomy level, wherein to cause the action to be performed according to the autonomy level comprises to:
in response to determining that the autonomy level indicates “verified autonomy” and to determining that the AV is being monitored,
cause the action to be performed; and
in response to receiving an override signal while the action is being performed, cause the action to stop and cause a switch to a manual operation mode of the AV.

### Claim 12 (depends on 11)

The AV of claim 11, wherein the autonomy level is selected from a set comprising a first autonomy level indicating “no autonomy”, a second autonomy level indicating “verified autonomy”, a third autonomy level indicating “supervised autonomy”, and a fourth autonomy level indicating “unsupervised autonomy”.

### Claim 13 (depends on 12)

The AV of claim 12,
wherein the autonomy level is the second autonomy level indicating “verified autonomy”, and wherein to perform the action according to the autonomy level comprises to:
receive, for the action, an approval feedback signal or a disapproval feedback signal.

### Claim 14 (depends on 13)

The AV of claim 13, wherein to perform the action according to the autonomy level comprises to:
query for the approval feedback signal prior to receiving the approval feedback signal.

### Claim 15 (depends on 13)

The AV of claim 13,
wherein the autonomy level is the third autonomy level indicating “supervised autonomy”, and
wherein to performing the action according to the autonomy level comprises to:
determine that the AV is being monitored by a human before performing the action.

### Claim 16 (depends on 13)

The AV of claim 13, wherein the processor is further configured to execute instructions to:
receive the approval feedback signal.

### Claim 17 (depends on 11)

The AV of claim 11, wherein to perform the action according to the autonomy level comprises to:
determine, based on the autonomy level, whether to request approval from a human for the action before performing the action.

### Claim 18 (depends on 11)

The AV of claim 11,
wherein the autonomy level is an “no autonomy” such that the AV is not allowed to perform autonomous actions, and
wherein to perform the action according to the autonomy level comprises to:
enable the AV to be manually controlled by a human.

### Claim 19 (depends on 11)

The AV of claim 11,
wherein the autonomy model comprises a utility model and an autonomy profile,
wherein utility model describes a utility of performing a first action in a first autonomy level with respect to a first environment state given that the AV just operated in second autonomy level, and
wherein the autonomy profile maps respective environment states to respective actions and prescribing constraints on allowed levels of autonomy for particular environment states.

### Claim 20 (depends on 11)

The AV of claim 11, wherein the processor is further configured to execute instructions to:
update at least one of an autonomy profile, a feedback profile, or a human transition function in response to performing the action.