# US 2021/0078735 A1 · Satellite threat mitigation by application of reinforcement machine learning in physics based space simulation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0078735 A1
- Document ID: US20210078735A1
- Status: Published application
- Kind: A1
- Application number: 16/574,200
- Filed: 2019-09-18
- Published: 2021-03-18
- Earliest claimed: 2019-09-18
- Assignees: BAE Systems Information and Electronic Systems Integration Inc.
- Applicants: BAE SYSTEMS Information and Electronic Systems Integration Inc.
- Inventors: Benjamin KAPP; Wyatt K. HARDING; Quoc Toan M. LUONG; Mark J. KOZAR
- Web page: https://patentgrep.com/patent/US20210078735A1

## CPC

- G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30)
- G06N 3/08: Learning methods
- B64G 99/00: Subject matter not provided for in other groups of this subclass
- G06N 3/045: Combinations of networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/0499: Feedforward networks
- G06N 3/092: Reinforcement learning
- G06N 20/00: Machine learning
- B64G 1/244: {Spacecraft control systems}
- G06F 30/25: using particle-based methods
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B64G: COSMONAUTICS; VEHICLES OR EQUIPMENT THEREFOR

## Abstract

The system and method for using a reinforcement machine learning based solution for space applications for automated course of action recommendations for the mitigation of threats to space-based assets. The system can be used to mitigate threats to satellites, and it can be used generally as a multi-domain reinforcement machine learning environment for many different kinds of agents, performing many different kinds of actions, under many different simulated environmental conditions.

## Claims

### Claim 1

A method of threat mitigation for space-based assets, comprising:
providing a reinforcement machine learning agent trained from a physics based space simulation, wherein the reinforcement machine learning agent is configured to:
process environmental information including data about one or more space-based assets and one or more threats;
receive warnings from one or more sensors, wherein the warnings require a course of action; and
providing a suggestion to an analyst for which course of action to follow to mitigate the one or more threats against the one or more space-based assets.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the system provides a plurality of suggested courses of action along with respective confidence intervals.

### Claim 3 (depends on 1)

The method according to claim 1, wherein processing environmental information includes assessing sample courses of action in sample situations.

### Claim 4 (depends on 1)

The method according to claim 1, wherein reinforcement machine learning comprises a reward function to push the system to learn ideal responses to various situations.

### Claim 5 (depends on 1)

The method according to claim 1, wherein reinforcement machine learning comprises a loss function to calculate how well the system estimates a situation and the ideal action to take.

### Claim 6 (depends on 1)

The method according to claim 1, wherein environmental information is input via a text file, graphical user interface, or direct connection to sensors that output the environmental information.

### Claim 7 (depends on 1)

The method according to claim 1, wherein a suggestion is output via a file that displays which course of action should be taken along with what the agent took into consideration.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the file is displayed onto a graphical user interface.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the system is embedded into a space based asset.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the space based asset is a satellite.

### Claim 11

A computer program product including one or more non-transitory machine-readable mediums having instructions encoded thereon that, when executed by one or more processors on board a space based asset, result in operations for mitigating threats to the space based asset, the operations comprising:
training a reinforcement machine learning agent using data on the space based asset and a plurality of threats;
computing a policy and a value of action at a given state on the reinforcement machine learning agent;
processing the policy and the value of action with a simulator, wherein the simulator sends back new state information to the reinforcement machine learning agent which computes new policy and new value of action;
making a decision by the reinforcement machine learning agent and matching to a course of action; and
providing the course of action for execution.

### Claim 12 (depends on 11)

The computer program product according to claim 11, further comprising post-processing the course of action.

### Claim 13 (depends on 11)

The computer program product according to claim 11, wherein providing the course of action is providing the course of action to an operator.

### Claim 14 (depends on 11)

The computer program product according to claim 11, wherein making the decision is done at an end of a simulation.
