# US 2025/0083702 A1 · Method Of Machine-Learned Verification And Advance Notice Oracles For Autonomous Systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0083702 A1
- Document ID: US20250083702A1
- Status: Published application
- Kind: A1
- Application number: 18/809,958
- Filed: 2024-08-20
- Published: 2025-03-13
- Earliest claimed: 2023-09-12
- Assignees: The Government of the United States of America, as represented by the Secretary of the Navy
- Applicants: The United States of America AS Represented By the Secretary of the Navy
- Inventors: Doron Drusinsky; Matthew Leonard Litton; James Bret Michael
- Web page: https://patentgrep.com/patent/US20250083702A1

## CPC

- B60W 60/0015: {specially adapted for safety}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- B60W 2520/105: Longitudinal acceleration
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method of training, verification, and advanced notice for an autonomous system, comprising: by a machine learning classifier, wherein an adversary vehicle, and a primary vehicle are on the same path; creating a path position probability transition matrix, wherein the path position probability transition matrix comprises an acceleration parameters array of one or more acceleration parameters for a potential position, sorting a sample path, and using Ne elite path; creating a samples array, wherein the sample array is one or more paths; evaluating a sample path; sorting the sample path, based at least in part a custom score of a custom score function and selecting a subset of the sample paths as Ne elite paths; using the Ne elite paths to update the path probability transition matrix and the acceleration parameters array; and repeating until the Ne elite paths stabilizes for a predetermined number of iterations.

## Claims

### Claim 1

A method of training, verification, and advanced notice for an autonomous system, comprising:
by a machine learning classifier, wherein an adversary vehicle, and a primary vehicle are on the same path;
creating a path position probability transition matrix, wherein the path position probability transition matrix comprises an acceleration parameters array of one or more acceleration parameters for a potential position, wherein the path position probability transition matrix comprises equal probabilities for creating a samples array, evaluating a sample path, sorting a sample path, and using Ne elite path;
creating a samples array, wherein the sample array is one or more paths, wherein the one or more paths are a sequence of grid points of a two-dimensional grid, wherein the grid points comprise: a position and an acceleration, wherein the position is a sampled position from a probability distribution of the path probability transition matrix, and wherein the acceleration is a sampled acceleration from an acceleration probability distribution associated with the sampled position;
evaluating a sample path, wherein a custom score function asserts a potential position and a potential acceleration to identify a collision path of the perturbation-adversary and the primary vehicle, a no collision path of the adversary vehicle and the primary vehicle, a small position and acceleration distance between the adversary vehicle and the primary vehicle;
sorting the sample path, based at least in part a custom score of a custom score function and selecting a subset of the sample paths as Ne elite paths;
using the Ne elite paths to update the path probability transition matrix and the acceleration parameters array; and
repeating until the Ne elite paths stabilizes.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
updating the path probability transition matrix by the number of times the Ne elite paths transitions from location i to location j, divided by N2; and
updating Gaussian parameters pairs of the acceleration parameters array as follows:
μ
i
=
1
N
e
⁢
∑
K
=
0
N
e
⁢
accek
[
k
,
i
]
⁢
σ
i
2
=
1
N
e
⁢
∑
k
=
0
N
e
⁢
(
μ
i
-
accel
[
k
,
i
]
)
2
wherein accel[k, i] is the acceleration in cell i of the two-dimensional grid according to a k elite path.

### Claim 3 (depends on 1)

The method of claim 1, wherein the custom score function prioritizes a high priority sample path as a slight deviation from a non-colliding path that result in a collision in the Ne elite paths;
wherein the custom score function prioritizes successively different colliding or non-colliding paths to generate a high-variance dataset; and
wherein the high-variance data set trains a machine learning model.

### Claim 4 (depends on 1)

The method of claim 1, further comprising an independent adversary vehicle, wherein the independent adversary vehicle, the vanilla adversary vehicle, the adversary vehicle, and the primary vehicle are on the same path; and
wherein the custom score function asserts a potential position and a potential acceleration to identify a no collision between the adversary vehicle and the independent adversary vehicle, and a small position and acceleration distance between the adversary vehicle and independent adversary vehicle.

### Claim 5 (depends on 4)

The method of claim 4, wherein the sample paths of the independent adversary vehicle and the primary vehicle should not collide.

### Claim 6 (depends on 1)

The method of claim 1, wherein realistic scenarios are simulated from one or more autonomous vehicle reporting databases.

### Claim 7 (depends on 1)

The method of claim 1, wherein one or more random sample paths are added to the high-variance dataset; and
wherein the one or more random sample paths represent one or more naturalistic paths.

### Claim 8 (depends on 1)

The method of claim 1, wherein one or more dissimilar collision and non-collision paths contribute to model robustness.

### Claim 9

A system for training, verifying, and providing advance notice, the system comprising a non-transitory computer readable medium storing instruction executable by a processor, wherein the instructions comprise instructions to:
by a machine learning classifier, wherein an adversary vehicle, and a primary vehicle are on the same path;
creating a path position probability transition matrix, wherein the path position probability transition matrix comprises an acceleration parameters array of one or more acceleration parameters for a potential position, wherein the path position probability transition matrix comprises equal probabilities for creating a samples array, evaluating a sample path, sorting a sample path, and using Ne elite path;
creating a samples array, wherein the sample array is one or more paths, wherein the one or more paths are a sequence of grid points of a two-dimensional grid, wherein the grid points comprise: a position and an acceleration, wherein the position is a sampled position from a probability distribution of the path probability transition matrix, and wherein the acceleration is a sampled acceleration from an acceleration probability distribution associated with the sampled position;
evaluating a sample path, wherein a custom score function asserts a potential position and a potential acceleration to identify a collision path of the adversary vehicle and the primary vehicle, a no collision path of adversary vehicle and the primary vehicle, a small position and acceleration distance between the adversary vehicle and primary vehicle;
sorting the sample path, based at least in part a custom score of a custom score function and selecting a subset of the sample paths as Ne elite paths;
using the Ne elite paths to update the path probability transition matrix and the acceleration parameters array; and
repeating until the Ne elite paths stabilizes.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
updating the path probability transition matrix by the number of times the Ne elite paths transitions from location i to location j, divided by N2; and
updating Gaussian parameters pairs of the acceleration parameters array as follows:
μ
i
=
1
N
e
⁢
∑
K
=
0
N
e
⁢
accek
[
k
,
i
]
⁢
σ
i
2
=
1
N
e
⁢
∑
k
=
0
N
e
⁢
(
μ
i
-
accel
[
k
,
i
]
)
2
wherein accel[k, i] is the acceleration in cell i of the two-dimensional grid according to a k elite path.

### Claim 11 (depends on 9)

The method of claim 9, wherein the custom score function prioritizes a high priority sample path as a slight deviation from a non-colliding path that result in a collision in the Ne elite paths;
wherein the custom score function prioritizes successively different colliding or non-colliding paths to generate a high-variance dataset; and
wherein the high-variance data set trains a machine learning model.

### Claim 12 (depends on 9)

The method of claim 9, further comprising an independent adversary vehicle, wherein the independent adversary vehicle, the primary vehicle, and the adversary vehicle are on the same path; and
wherein the custom score function asserts a potential position and a potential acceleration to identify a no collision between adversary and independent adversary vehicles, and a small position and acceleration distance between adversary vehicle and independent adversary vehicle.

### Claim 13 (depends on 12)

The method of claim 12, wherein the sample paths of the independent-adversary and the primary vehicle should not collide.

### Claim 14 (depends on 9)

The method of claim 9, wherein realistic scenarios are simulated from one or more autonomous vehicle reporting databases.

### Claim 15 (depends on 9)

The method of claim 9, wherein one or more random sample paths are added to the high-variance dataset; and
wherein the one or more random sample paths represent one or more naturalistic paths.

### Claim 16 (depends on 9)

The method of claim 9, wherein one or more dissimilar collision and non-collision paths contribute to model robustness.

### Claim 17

A Hybrid-Pair-Cross Entropy method of training, verification, and advanced notice for an autonomous system, the method comprising:
by an adversary vehicle and a primary vehicle, wherein the adversary vehicle, and the primary vehicle are on the same path;
creating a path position probability transition matrix, wherein the path position probability transition matrix comprises an acceleration parameters array of one or more acceleration parameters for a potential position, wherein the path position probability transition matrix comprises equal probabilities for creating a samples array, evaluating a sample path, sorting a sample path, and using Ne elite path;
creating a samples array, wherein the sample array is one or more paths, wherein the one or more paths are a sequence of grid points of a two-dimensional grid, wherein the grid points comprise: a position and an acceleration, wherein the position is a sampled position from a probability distribution of the path probability transition matrix, and wherein the acceleration is a sampled acceleration from an acceleration probability distribution associated with the sampled position;
evaluating a sample path, wherein a custom score function asserts a potential position and a potential acceleration to identify a collision path of the perturbation-adversary and the primary vehicle, a no collision path of adversary vehicle and the primary vehicle, a small position and acceleration distance between adversary and primary vehicle, wherein the custom score function prioritizes a high priority sample path as a slight deviation from a non-colliding path that result in a collision in the Ne elite paths, wherein the custom score function prioritizes successively different colliding or non-colliding paths to generate a high-variance dataset, wherein the high-variance data set trains a machine learning model;
sorting the sample path based on a corresponding score from the custom score function and selecting a subset of the sample paths as Ne elite paths;
using the Ne elite paths to update the path probability transition matrix and the acceleration parameters array;
repeating until the Ne elite paths stabilizes for a predetermined number of iterations;
updating the path probability transition matrix by the number of times the Ne elite paths transitions from location i to location j, divided by N2;
updating Gaussian parameters pairs of the acceleration parameters array as follows:
μ
i
=
1
N
e
⁢
∑
K
=
0
N
e
⁢
accek
[
k
,
i
]
⁢
σ
i
2
=
1
N
e
⁢
∑
k
=
0
N
e
⁢
(
μ
i
-
accel
[
k
,
i
]
)
2
wherein accel[k, i] is the acceleration in cell i of the two-dimensional grid according to a k elite path;
wherein the custom score function prioritizes a high priority sample path as a slight deviation from a non-colliding path that result in a collision in the Ne elite paths;
wherein the custom score function prioritizes successively different colliding or non-colliding paths to generate a high-variance dataset; and
wherein the high-variance data set trains a machine learning model.

### Claim 18 (depends on 17)

The method of claim 17, further comprising an independent adversary vehicle, wherein the independent adversary vehicle, the adversary vehicle, and the primary vehicle are on the same path; and
wherein the custom score function asserts a potential position and a potential acceleration to identify a no collision between the adversary vehicle and independent adversary vehicle, and a small position and acceleration distance between the adversary vehicle and the independent adversary vehicle.

### Claim 19 (depends on 18)

The method of claim 18, wherein the sample paths of the independent adversary vehicle and the primary vehicle should not collide.

### Claim 20 (depends on 19)

The method of claim 19, wherein realistic scenarios are simulated from autonomous vehicle reporting databases; and
wherein one or more dissimilar collision and non-collision paths contribute to model robustness.
