# US 2023/0409422 A1 · Systems and Methods for Anomaly Detection in Multi-Modal Data Streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0409422 A1
- Document ID: US20230409422A1
- Status: Published application
- Kind: A1
- Application number: 18/338,222
- Filed: 2023-06-20
- Published: 2023-12-21
- Earliest claimed: 2022-06-17
- Assignees: The Board of Trustees of the Leland Stanford Junior University
- Applicants: The Board of Trustees of the Leland Stanford Junior University
- Inventors: Daniel Ratner; Eric Felix Darve; Ryan Humble
- Web page: https://patentgrep.com/patent/US20230409422A1

## CPC

- G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and method for detecting anomalies in accordance with embodiments of the invention are illustrated. One embodiment includes receiving a first data stream collected from a first sensor, identifying a first set of anomalies in the first data stream using a first model, receiving a second data stream collected from a second sensor, identifying a second set of anomalies in the second data stream using a second model, determining a set of joint anomalies using the first set of anomalies, second set of anomalies, and a threshold, wherein a threshold is some time period, and updating how anomalies are identified in the first and second set of models using the set of joint anomalies.

## Claims

### Claim 1

A method to train models to detect anomalies in multi-modal data streams, comprising:
receiving a first data stream collected from a first sensor;
identifying a first set of anomalies in the first data stream using a first model;
receiving a second data stream collected from a second sensor;
identifying a second set of anomalies in the second data stream using a second model;
determining a set of joint anomalies using the first set of anomalies, second set of anomalies, and a threshold, wherein a threshold is a confidence level; and
updating the first and second models using the set of joint anomalies.

### Claim 2 (depends on 1)

The method of claim 1, wherein updating the first and second models further comprises updating the parameters of the first and second models.

### Claim 3 (depends on 2)

The method of claim 2, further comprising updating the parameters of the first and second models to more accurately identify anomalies.

### Claim 4 (depends on 1)

The method of claim 1, further comprising updating the threshold to more accurately identify joint anomalies using the set of joint anomalies.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first and second models are parameterized functions with modifiable parameters that balances confidence in the anomaly detection predictions and the number of anomalous events predicted.

### Claim 6 (depends on 5)

The method of claim 5, wherein the parameterized functions maximize the covariance of outputs of each of the parameterized functions.

### Claim 7 (depends on 5)

The method of claim 5, wherein the parameterized functions maximize an unsupervised metric.

### Claim 8 (depends on 5)

The method of claim 5, wherein the confidence in the anomaly detection predictions and number of detected anomalies are imputed from other parameters and are not directly exposed.

### Claim 9 (depends on 1)

The method of claim 1, wherein a sensor is a program that collects logs from a device.

### Claim 10 (depends on 1)

The method of claim 1, further comprising generating output.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
displaying the output via a graphical interface; and
sending the output over a network.

### Claim 12 (depends on 10)

The method of claim 10, further comprising performing an event when there is high confidence that a joint anomaly is detected.

### Claim 13 (depends on 12)

The method of claim 12, wherein the event is an automatic shutoff.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
receiving a third data stream collected from the first sensor; and
identifying a third set of anomalies in the third data stream using the updated first model.

### Claim 15 (depends on 1)

The method of claim 1, wherein identifying the first set of anomalies is based on a set of one or more continuous functions.

### Claim 16 (depends on 1)

The method of claim 1, wherein at least one of the first and second models is a neural network.

### Claim 17

A method of detecting manufacturing anomalies on an assembly line, comprising:
receiving a first data stream from a first sensor, wherein the first sensor sends data from a station on an assembly line;
identifying a first set of anomalies in the first data stream using a first model;
receiving a second data stream from a second sensor, wherein the second sensor sends data from another station on the assembly line;
identifying a second set of anomalies in the second data stream using a second model;
determining a set of joint anomalies using the first set of anomalies, second set of anomalies, and a threshold; and
updating how anomalies are identified in the first and second set of models using the set of joint anomalies.

### Claim 18 (depends on 17)

The method of claim 17, wherein the second sensor sends data from the final assembly line product.

### Claim 19 (depends on 1)

The method of claim 1, wherein anomalies are represented by the group consisting of viruses, intrusions, and persistent threats;

### Claim 20 (depends on 1)

The method of claim 1, wherein:
data streams are represented by different medical diagnostics; and
anomalies are represented by disease diagnoses.
