# US 12,762,395 B2 · Training a neural network model across multiple domains > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,395 B2 - Document ID: US12762395B2 - Status: Granted - Kind: B2 - Application number: 17/884,205 - Filed: 2022-08-09 - Published: 2023-11-09 - Granted: 2026-10-13 - Earliest claimed: 2022-05-06 - Assignees: The Bank of New York Mellon - Applicants: The Bank of New York Mellon - Inventors: Abhinav Prasad; Beibei Liu; Romil Rathi; Richard Marquis; Rajeev Sambyal - Web page: https://patentgrep.com/patent/US12762395B2 ## CPC - G06N 3/08: Learning methods - G06N 3/045: Combinations of networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The disclosure relates to systems and methods of generating a mixture model for approximating non-normal distributions of time series data. The mixture model may include clusters of normal distributions that together approximate a non-normal distribution. The mixture model may be used to normalize input data for machine learning models. For example, a machine learning model such as an autoencoder may be trained to make predictions on the normalized input data. The predictions may relate to the time series of data. In one example, the time series of data may be market data for a security. The market data my include one or more features that are normalized using the mixture model. The predictions may include a predicted rate at which a lender will charge to borrow a security for short selling, where such rate may depend on the market data for the security. ## Claims ### Claim 1 A system comprising: a memory that stores a plurality of time series of data each relating to a respective domain; a processor programmed to: access first training data relating to a first domain, the first training data having a first time series of data comprising first sequential data values that vary over time; access second training data relating to a second domain, the second training data having a second time series of data comprising second sequential data values that vary independently from the first sequential data values over time; generate a first plurality of sequences from the first training data, the first plurality of sequences being generated to include an ordered subset of sequential data values defined by a sliding temporal window over the first time series of data such that adjacent sequences share overlapping data values; generate a second plurality of sequences from the second training data, the second plurality of sequences being generated to include an ordered subset of sequential data values defined by a second sliding temporal window over the second time series of data such that adjacent sequences share overlapping data values; append the first plurality of sequences and the second plurality of sequences to generate an appended input data relating to the first domain and the second domain, wherein the appended input data comprises a concatenation of the first plurality of sequences and the second plurality of sequences into a unified training dataset that preserves temporal ordering of each sequence and associates each sequence with a domain identifier to train a single machine-learning model to learn feature relationships shared across the first domain and the second domain; and provide the appended input data to a neural network to train the single machine-learning model trained to make predictions in the first domain and/or the second domain. ### Claim 2 (depends on 1) The system of claim 1, wherein the processor is further programmed to: receive input data relating to the first domain or the second domain, the input data comprising an input time series of data; normalize the input data; provide the normalized input data to the single machine-learning model; and generate a prediction based on the input data using the single machine-learning model. ### Claim 3 (depends on 2) The system of claim 2, wherein to normalize the input data, the processor is further programmed to: generate a mixture model comprising a plurality of clusters of normal distributions that together approximate the input data. ### Claim 4 (depends on 3) The system of claim 3, wherein the processor is further programmed to: for each data value in the input data: identify a corresponding cluster from among the plurality of clusters; determine a normalization value based on the corresponding cluster; and normalize the data value based on the normalization value. ### Claim 5 (depends on 2) The system of claim 2, wherein the neural network is part of a parallel neural network architecture comprising a plurality of neural networks, and wherein the processor is further programmed to: provide the appended input data to an input layer of a first neural network of the parallel network architecture. ### Claim 6 (depends on 1) The system of claim 1, wherein each sequence from among the first plurality of sequences comprises a respective subset of the first time series of data. ### Claim 7 (depends on 6) The system of claim 6, wherein each sequence from among the first plurality of sequences have in common at least some of the first time series of data with a next sequence in the first plurality of sequences. ### Claim 8 (depends on 6) The system of claim 6, wherein a number of the plurality of sequences that are generated is based on a size of the first time series of data. ### Claim 9 (depends on 1) The system of claim 1, wherein the single machine-learning model comprises a single Long-term Short-term Memory (LSTM) model. ### Claim 10 A method comprising: accessing, by a processor, first training data relating to a first domain, the first training data having a first time series of data comprising first sequential data values that vary over time; accessing, by the processor, second training data relating to a second domain, the second training data having a second time series of data comprising second sequential data values that vary independently from the first sequential data values over time; generating, by the processor, a first plurality of sequences from the first training data, the first plurality of sequences being generated to include an ordered subset of sequential data values defined by a sliding temporal window over the first time series of data such that adjacent sequences share overlapping data values; generating, by the processor, a second plurality of sequences from the second training data, the second plurality of sequences being generated to include an ordered subset of sequential data values defined by a second sliding temporal window over the second time series of data such that adjacent sequences share overlapping data values; appending, by the processor, the first plurality of sequences and the second plurality of sequences to generate an appended input data relating to the first domain and the second domain, wherein the appended input data comprises a concatenation of the first plurality of sequences and the second plurality of sequences into a unified training dataset that preserves temporal ordering of each sequence and associates each sequence with a domain identifier to train a single machine-learning model to learn feature relationships shared across the first domain and the second domain; and providing, by the processor, the appended input data to a neural network to train the single machine-learning model trained to make predictions in the first domain and/or the second domain. ### Claim 11 (depends on 10) The method of claim 10, the method further comprising: receiving input data relating to the first domain or the second domain, the input data comprising an input time series of data; normalizing the input data; providing the normalized input data to the single machine-learning model; and generating a prediction based on the input data using the single machine-learning model. ### Claim 12 (depends on 11) The method of claim 11, wherein normalizing the input data comprises: generating a mixture model comprising a plurality of clusters of normal distributions that together approximate the input data. ### Claim 13 (depends on 10) The method of claim 10, wherein the neural network is part of a parallel neural network architecture comprising a plurality of neural networks, and wherein the method further comprising: providing the appended input data to an input layer of a first neural network of the parallel network architecture. ### Claim 14 (depends on 10) The method of claim 10, wherein each sequence from among the first plurality of sequences comprises a respective subset of the first time series of data. ### Claim 15 (depends on 14) The method of claim 14, wherein each sequence from among the first plurality of sequences have in common at least some of the first time series of data with a next sequence in the first plurality of sequences. ### Claim 16 (depends on 14) The method of claim 14, wherein a number of the plurality of sequences that are generated is based on a size of the first time series of data. ### Claim 17 (depends on 10) The method of claim 10, wherein training a single machine-learning model comprises training a single Long-term Short-term Memory (LSTM) model. ### Claim 18 A non-transitory computer readable medium storing instructions that, when executed by a processor, causes the processor to: access first training data relating to a first domain, the first training data having a first time series of data comprising first sequential data values that vary over time; access second training data relating to a second domain, the second training data having a second time series of data comprising second sequential data values that vary independently from the first sequential data values over time; generate a first plurality of sequences from the first training data, the first plurality of sequences being generated to include an ordered subset of sequential data values defined by a sliding temporal window over the first time series of data such that adjacent sequences share overlapping data values; generate a second plurality of sequences from the second training data, the second plurality of sequences being generated to include an ordered subset of sequential data values defined by a second sliding temporal window over the second time series of data such that adjacent sequences share overlapping data values; append the first plurality of sequences and the second plurality of sequences to generate an appended input data relating to the first domain and the second domain, wherein the appended input data comprises a concatenation of the first plurality of sequences and the second plurality of sequences into a unified training dataset that preserves temporal ordering of each sequence and associates each sequence with a domain identifier to train a single machine-learning model to learn feature relationships shared across the first domain and the second domain; and provide the appended input data to a neural network to train the single machine-learning model trained to make predictions in the first domain and/or the second domain. ### Claim 19 (depends on 18) The non-transitory computer readable medium of claim 18, wherein the instructions, when executed, further cause the processor to: receive input data relating to the first domain or the second domain, the input data comprising an input time series of data; normalize the input data; provide the normalized input data to the single machine-learning model; and generate a prediction based the input data using the single machine-learning model. ### Claim 20 (depends on 18) The non-transitory computer readable medium of claim 18, wherein the single machine-learning model comprises a single Long-term Short-term Memory (LSTM) model.