# US 2023/0057100 A1 · Method and system for a continuous discrete recurrent kalman network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0057100 A1
- Document ID: US20230057100A1
- Status: Published application
- Kind: A1
- Application number: 17/407,648
- Filed: 2021-08-20
- Published: 2023-02-23
- Earliest claimed: 2021-08-20
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Mona SCHIRMER; Mazin ELTAYEB; Maja RUDOLPH
- Web page: https://patentgrep.com/patent/US20230057100A1

## CPC

- G06F 18/21375: {involving differential geometry, e.g. embedding of pattern manifold}
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G01S 19/47: the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G01S 19/393: {Trajectory determination or predictive tracking, e.g. Kalman filtering}
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method utilizing a continuous discrete recurrent Kalman network, wherein the method includes receiving, at an encoder, an input from one or more sensors, wherein the input includes one or more time series data associating data at one or more points in time; outputting, to a Kalman filter, a latent observation and uncertainty estimate in response to the input at the encoder; determining a latent state prior and latent state posterior utilizing the Kalman filter; and outputting, via a decoder, a filtered observation utilizing at least the latent state posterior.

## Claims

### Claim 1

A computer-implemented method utilizing a continuous discrete recurrent Kalman network, comprising:
receiving, at an encoder, an input from one or more sensors, wherein the input includes one or more time series data associating data at one or more points in time;
outputting, to a Kalman filter, a latent observation and uncertainty estimate in response to the input at the encoder;
determining a prior latent state and posterior latent state utilizing the Kalman filter; and
outputting, via a decoder, a filtered observation utilizing at least the posterior latent state.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the Kalman filter, the encoder, and the decoder each include model parameters configured to be trained with backpropagation in an end-to-end manner.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the decoder is configured to map the filtered observation to the time series data.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the input includes image sequences and the encoder and decoder include a convolutional neural network.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the Kalman filter is a recurrent continuous-discrete Kalman filter.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the method includes, utilizing the Kalman filter, outputting a discrete observation at an arbitrary time interval.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more sensors includes a camera, global positioning system (GPS) sensor, temperature sensor, oxygen sensor, speed sensor, or a vehicle sensor.

### Claim 8

A system that includes a continuous discrete recurrent Kalman network, comprising:
an encoder configured to receive an input from one or more sensors, wherein the input includes one or more time series data associating data at one or more points in time;
a recurrent continuous-discrete Kalman filter configured to receive as output from the encoder, a latent observation and uncertainty estimate in response to the input at the encoder, and further configured to determine prior latent state and posterior latent state; and
a decoder configured to output a filtered observation utilizing at least the posterior latent state.

### Claim 9 (depends on 8)

The system of claim 8, wherein the Kalman filter, the encoder, and the decoder each include model parameters configured to be trained with backpropagation.

### Claim 10 (depends on 8)

The system of claim 8, wherein the latent observation is output in response to a stochastic differential equation.

### Claim 11 (depends on 8)

The system of claim 8, wherein the decoder is configured to map the filtered observation to the time series data.

### Claim 12 (depends on 8)

The system of claim 8, wherein the input includes image sequences and the encoder and decoder include a convolutional neural network.

### Claim 13 (depends on 8)

The system of claim 8, wherein the Kalman filter is further configured to output a discrete observation at an arbitrary time interval.

### Claim 14 (depends on 8)

The system of claim 8, wherein the one or more sensors includes a camera, global positioning system (GPS) sensor, temperature sensor, oxygen sensor, speed sensor, or a vehicle sensor.

### Claim 15

A system that includes a continuous discrete recurrent Kalman network, comprising:
a processor programmed to:
receive, at an encoder, an input from one or more sensors, wherein the input includes one or more time series data associating data at one or more points in time;
output, to a Kalman filter, a latent observation and an uncertainty estimate in response to the input at the encoder;
determine a prior latent state and posterior latent state utilizing the Kalman filter; and
output, via a decoder, a filtered observation utilizing at least the posterior latent state.

### Claim 16 (depends on 15)

The system of claim 15, wherein the processor is further programmed to output, via a decoder, a filtered observation utilizing at least the posterior latent state.

### Claim 17 (depends on 15)

The system of claim 15, wherein the filtered observation indicates data associated with a future time.

### Claim 18 (depends on 15)

The system of claim 15, wherein the Kalman filter is further configured to utilize the latent observation and the uncertainty estimate to update the prior latent state.

### Claim 19 (depends on 15)

The system of claim 15, wherein the decoder is further configured to map the posterior latent state to an observation space.

### Claim 20 (depends on 15)

The system of claim 15, wherein the Kalman filter is a recurrent continuous-discrete Kalman filter.
