# US 2024/0201669 A1 · System and method with sequence modeling of sensor data for manufacturing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0201669 A1
- Document ID: US20240201669A1
- Status: Published application
- Kind: A1
- Application number: 18/067,432
- Filed: 2022-12-16
- Published: 2024-06-20
- Earliest claimed: 2022-12-16
- Applicants: Robert Bosch GmbH
- Inventors: Filipe Condessa; Devin Willmott; Ivan Batalov; Joao Semedo; Zhichun Huang; Wan-Yi Lin; Bahare Azari
- Web page: https://patentgrep.com/patent/US20240201669A1

## CPC

- G05B 19/4184: {characterised by fault tolerance, reliability of production system}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented system and method includes establishing a station sequence that a given part traverses. A first neural network generates a set of parameter data based on observed measurement data of the given part at each station of a station subsequence. The set of parameter data is associated with a latent variable subsequence corresponding to the station subsequence. A second neural network generates next parameter data based on history measurement data and the set of parameter data. The history measurement data relates to another part processed before the given part and is associated with each station of the station sequence. The next parameter data is associated with a next latent variable that follows the latent variable subsequence. The next latent variable corresponds to a next station that follows the station subsequence in the station sequence. The second neural network generates predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.

## Claims

### Claim 1

A computer-implemented method for predictive measurement monitoring, the method comprising:
establishing a station sequence that includes a plurality of stations that a given part traverses;
receiving, via a first neural network, observed measurement data regarding attributes of the given part as obtained by one or more sensors at each station of a station subsequence of the station sequence;
generating, via the first neural network, a set of parameter data based on the observed measurement data, the set of parameter data being associated with a latent variable subsequence, the latent variable subsequence corresponding to the station subsequence;
receiving, via a second neural network, history measurement data of another part that was processed before the given part, the history measurement data regarding attributes of the another part that are taken with respect to each station of the plurality of stations of the station sequence;
generating, via the second neural network, next parameter data based on the history measurement data while using the set of parameter data, the next parameter data being associated with a next latent variable that follows the latent variable subsequence, the next latent variable corresponding to a next station that follows the station subsequence in the station sequence; and
generating, via the second neural network, predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the first neural network includes a long short-term memory (LSTM) or a temporal convolutional network (TCN) that generates the set of parameter data.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein:
the second neural network includes a first multi-layer perceptron network that generates the next parameter data; and
the second neural network includes a second multi-layer perceptron network that generates the predicted measurement data.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein:
a machine learning model includes an inference model and a generative model;
the inference model includes the first neural network; and
the generative model includes the second neural network.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the first set of parameter data include a posterior mean and a posterior variance associated with each latent variable of the latent variable subsequence.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the next parameter data includes a prior mean and a prior variance associated with the next latent variable.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein:
the first set of observed measurement data is based on multimodal sensor data;
the first set of history measurement data is based on multimodal sensor data; and
the predicted measurement data is based on multimodal sensor data.

### Claim 8

A system comprising:
a processor; and
a memory in data communication with the processor, the memory having computer readable data including instructions stored thereon that, when executed by the processor, cause the processor to perform a method for predictive measurement monitoring, the method including:
establishing a station sequence that includes a plurality of stations that a given part traverses;
receiving, via a first neural network, observed measurement data regarding attributes of the given part as obtained by one or more sensors at each station of a station subsequence of the station sequence;
generating, via the first neural network, a set of parameter data based on the observed measurement data, the set of parameter data being associated with a latent variable subsequence, the latent variable subsequence corresponding to the station subsequence;
receiving, via a second neural network, history measurement data of another part that was processed before the given part, the history measurement data regarding attributes of the another part that are taken with respect to each station of the plurality of stations of the station sequence;
generating, via the second neural network, next parameter data based on the history measurement data while using the set of parameter data, the next parameter data being associated with a next latent variable that follows the latent variable subsequence, the next latent variable corresponding to a next station that follows the station subsequence in the station sequence; and
generating, via the second neural network, predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.

### Claim 9 (depends on 8)

The system of claim 8, wherein the first neural network includes a long short-term memory (LSTM) or a temporal convolutional network (TCN) that generates the set of parameter data.

### Claim 10 (depends on 8)

The system of claim 8, wherein:
the second neural network includes a first multi-layer perceptron network that generates the next parameter data; and
the second neural network includes a second multi-layer perceptron network that generates the predicted measurement data.

### Claim 11 (depends on 8)

The system of claim 8, wherein:
a machine learning model includes an inference model and a generative model;
the inference model includes the first neural network; and
the generative model includes the second neural network.

### Claim 12 (depends on 8)

The system of claim 8, wherein the first set of parameter data include a posterior mean and a posterior variance associated with each latent variable of the latent variable subsequence.

### Claim 13 (depends on 8)

The system of claim 8, wherein the next parameter data includes a prior mean and a prior variance associated with the next latent variable.

### Claim 14 (depends on 8)

The system of claim 8, wherein:
the first set of observed measurement data is based on multimodal sensor data;
the first set of history measurement data is based on multimodal sensor data; and
the predicted measurement data is based on multimodal sensor data.

### Claim 15

A non-transitory computer readable medium having computer readable data including instructions stored thereon that, when executed by a processor, cause the processor to perform a method for predictive measurement monitoring, the method including:
establishing a station sequence that includes a plurality of stations that a given part traverses;
receiving, via a first neural network, observed measurement data regarding attributes of the given part as obtained by one or more sensors at each station of a station subsequence of the station sequence;
generating, via the first neural network, a set of parameter data based on the observed measurement data, the set of parameter data being associated with a latent variable subsequence, the latent variable subsequence corresponding to the station subsequence;
receiving, via a second neural network, history measurement data of another part that was processed before the given part, the history measurement data regarding attributes of the another part that are taken with respect to each station of the plurality of stations of the station sequence;
generating, via the second neural network, next parameter data based on the history measurement data while using the set of parameter data, the next parameter data being associated with a next latent variable that follows the latent variable subsequence, the next latent variable corresponding to a next station that follows the station subsequence in the station sequence; and
generating, via the second neural network, predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the first neural network includes a long short-term memory (LSTM) or a temporal convolutional network (TCN) that generates the set of parameter data.

### Claim 17 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein:
the second neural network includes a first multi-layer perceptron network that generates the next parameter data; and
the second neural network includes a second multi-layer perceptron network that generates the predicted measurement data.

### Claim 18 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein:
a machine learning model includes an inference model and a generative model;
the inference model includes the first neural network; and
the generative model includes the second neural network.

### Claim 19 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the first set of parameter data include a posterior mean and a posterior variance associated with each latent variable of the latent variable subsequence.

### Claim 20 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the next parameter data includes a prior mean and a prior variance associated with the next latent variable.
