# US 2026/0141030 A1 · Method, System, and Computer Program Product for Recurrent Neural Networks for Asynchronous Sequences

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0141030 A1
- Document ID: US20260141030A1
- Status: Published application
- Kind: A1
- Application number: 19/451,080
- Filed: 2026-01-16
- Published: 2026-05-21
- Earliest claimed: 2020-10-29
- Assignees: Visa International Service Association
- Applicants: Visa International Service Association
- Inventors: Javid Ebrahimi; Wei Zhang
- Web page: https://patentgrep.com/patent/US20260141030A1

## CPC

- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06F 18/2163: {Partitioning the feature space}
- G06F 18/24133: {Distances to prototypes}
- G06F 18/25: Fusion techniques
- G06N 3/02: Neural networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06Q 20/4016: {involving fraud or risk level assessment in transaction processing}
- G06Q 30/0631: {Recommending goods or services}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method for recurrent neural networks for asynchronous sequences may include receiving first input data associated with a plurality of first data items ordered in a first sequence and second input data associated with a plurality of second data items ordered in a second sequence. Each first data item may be of a first type, and each second data item may be of a second type. Each respective data item of the first and second data items may be inputted with an indicator associated with a respective type of the respective data item to a recurrent unit of a recurrent neural network (RNN). A respective portion of a hidden state may be determined based on the indicator. The respective portion of the hidden state may be updated based on the respective data item and the indicator. A system and computer program product are also disclosed.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving, with at least one processor, first input data associated with a plurality of first data items ordered in a first time sequence and second input data associated with a plurality of second data items ordered in a second time sequence, each first data item of the plurality of first data items associated with a respective first event of a first type, each second data item of the plurality of second data items associated with a respective second event of a second type different than the first type, wherein the plurality of first data items and the plurality of second data items are received asynchronously;
for each respective data item of the plurality of first data items and the plurality of second data items:
inputting, with the at least one processor, the respective data item and an indicator associated with a respective type of the respective data item to a recurrent unit of a recurrent neural network (RNN), the respective type comprising one of the first type or the second type, wherein the indicator comprises a slice of each flag tensor of a plurality of flag tensors, the plurality of flag tensors comprising a query flag tensor, a key flag tensor, and a value flag tensor;
determining, with the at least one processor, a respective portion of a hidden state of the RNN based on the indicator, wherein determining the respective portion of the hidden state comprises:
splitting the hidden state into a plurality of hidden state segments, at least one of the plurality of hidden state segments comprising the respective portion of the hidden state;
updating, with the at least one processor, each hidden state segment of the plurality of hidden state segments of the RNN using an attention mechanism based on the respective data item, the indicator, and trainable parameters of the RNN, wherein updating each hidden state segment using the attention mechanism comprises:
concatenating each hidden state segment with each of a plurality of flag vectors from each slice of each flag tensor;
determining at least one query vector, at least one key vector, and at least one value vector based on the hidden state segments having the plurality of flag vectors concatenated therewith; and
updating each hidden state segment of the plurality of hidden state segments based on the at least one query vector, the at least one key vector, and the at least one value vector; and
determining, with the at least one processor, at least one prediction based on the hidden state as updated.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of hidden state segments comprises a first hidden state segment associated with the first type and a second hidden state segment associated with the second type.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining the respective portion of the hidden state further comprises selecting the first hidden state segment or the second hidden state segment based on the indicator.

### Claim 4 (depends on 1)

The method of claim 1, wherein each hidden state segment of the plurality of hidden state segments comprises a matrix, at least one of the plurality of hidden state segments comprising the respective portion of the hidden state;
wherein updating each hidden state segment comprises updating the respective portion of the hidden state using the attention mechanism; and
wherein the plurality of hidden state segments as updated combine to provide the hidden state as updated.

### Claim 5 (depends on 1)

The method of claim 1, wherein splitting the hidden state into the plurality of hidden state segments comprises splitting the hidden state into the plurality of hidden state segments based on a number of types.

### Claim 6 (depends on 1)

The method of claim 1, wherein updating each hidden state segment using the attention mechanism further comprises:
appending a previous hidden state of the respective portion of the hidden state with a respective slice of each flag tensor and the respective data item and multiplying with a respective weight to generate each hidden state segment.

### Claim 7 (depends on 1)

The method of claim 1, wherein the at least one prediction comprises a prediction of a next data item in at least one of the first time sequence, the second time sequence, or any combination thereof.

### Claim 8

A system, comprising:
at least one processor configured to:
receive first input data associated with a plurality of first data items ordered in a first time sequence and second input data associated with a plurality of second data items ordered in a second time sequence, each first data item of the plurality of first data items associated with a respective first event of a first type, each second data item of the plurality of second data items associated with a respective second event of a second type different than the first type, wherein the plurality of first data items and the plurality of second data items are received asynchronously;
for each respective data item of the plurality of first data items and the plurality of second data items:
input the respective data item and an indicator associated with a respective type of the respective data item to a recurrent unit of a recurrent neural network (RNN), the respective type comprising one of the first type or the second type, wherein the indicator comprises a slice of each flag tensor of a plurality of flag tensors, the plurality of flag tensors comprising a query flag tensor, a key flag tensor, and a value flag tensor;
determine a respective portion of a hidden state of the RNN based on the indicator, wherein determining the respective portion of the hidden state comprises:
splitting the hidden state into a plurality of hidden state segments, at least one of the plurality of hidden state segments comprising the respective portion of the hidden state;
update each hidden state segment of the plurality of hidden state segments of the RNN using an attention mechanism based on the respective data item, the indicator, and trainable parameters of the RNN, wherein updating each hidden state segment using the attention mechanism comprises:
concatenating each hidden state segment with each of a plurality of flag vectors from each slice of each flag tensor;
determining at least one query vector, at least one key vector, and at least one value vector based on the hidden state segments having the plurality of flag vectors concatenated therewith; and
updating each hidden state segment of the plurality of hidden state segments based on the at least one query vector, the at least one key vector, and the at least one value vector; and
determine at least one prediction based on the hidden state as updated.

### Claim 9 (depends on 8)

The system of claim 8, wherein the plurality of hidden state segments comprises a first hidden state segment associated with the first type and a second hidden state segment associated with the second type.

### Claim 10 (depends on 9)

The system of claim 9, wherein determining the respective portion of the hidden state further comprises selecting the first hidden state segment or the second hidden state segment based on the indicator.

### Claim 11 (depends on 8)

The system of claim 8, wherein each hidden state segment of the plurality of hidden state segments comprises a matrix, at least one of the plurality of hidden state segments comprising the respective portion of the hidden state;
wherein updating each hidden state segment comprises updating the respective portion of the hidden state using the attention mechanism; and
wherein the plurality of hidden state segments as updated combine to provide the hidden state as updated.

### Claim 12 (depends on 8)

The system of claim 8, wherein splitting the hidden state into the plurality of hidden state segments comprises splitting the hidden state into the plurality of hidden state segments based on a number of types.

### Claim 13 (depends on 8)

The system of claim 8, wherein updating each hidden state segment using the attention mechanism further comprises:
appending a previous hidden state of the respective portion of the hidden state with a respective slice of each flag tensor and the respective data item and multiplying with a respective weight to generate each hidden state segment.

### Claim 14 (depends on 8)

The system of claim 8, wherein the at least one prediction comprises a prediction of a next data item in at least one of the first time sequence, the second time sequence, or any combination thereof.

### Claim 15

A computer program product comprising at least one non-transitory computer-readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to:
receive first input data associated with a plurality of first data items ordered in a first time sequence and second input data associated with a plurality of second data items ordered in a second time sequence, each first data item of the plurality of first data items associated with a respective first event of a first type, each second data item of the plurality of second data items associated with a respective second event of a second type different than the first type, wherein the plurality of first data items and the plurality of second data items are received asynchronously;
for each respective data item of the plurality of first data items and the plurality of second data items:
input the respective data item and an indicator associated with a respective type of the respective data item to a recurrent unit of a recurrent neural network (RNN), the respective type comprising one of the first type or the second type, wherein the indicator comprises a slice of each flag tensor of a plurality of flag tensors, the plurality of flag tensors comprising a query flag tensor, a key flag tensor, and a value flag tensor;
determine a respective portion of a hidden state of the RNN based on the indicator, wherein determining the respective portion of the hidden state comprises:
splitting the hidden state into a plurality of hidden state segments, at least one of the plurality of hidden state segments comprising the respective portion of the hidden state;
update each hidden state segment of the plurality of hidden state segments of the RNN using an attention mechanism based on the respective data item, the indicator, and trainable parameters of the RNN, wherein updating each hidden state segment using the attention mechanism comprises:
concatenating each hidden state segment with each of a plurality of flag vectors from each slice of each flag tensor;
determining at least one query vector, at least one key vector, and at least one value vector based on the hidden state segments having the plurality of flag vectors concatenated therewith; and
updating each hidden state segment of the plurality of hidden state segments based on the at least one query vector, the at least one key vector, and the at least one value vector; and
determine at least one prediction based on the hidden state as updated.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the plurality of hidden state segments comprises a first hidden state segment associated with the first type and a second hidden state segment associated with the second type, and wherein determining the respective portion of the hidden state further comprises selecting the first hidden state segment or the second hidden state segment based on the indicator.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein each hidden state segment of the plurality of hidden state segments comprises a matrix, at least one of the plurality of hidden state segments comprising the respective portion of the hidden state;
wherein updating each hidden state segment comprises updating the respective portion of the hidden state using the attention mechanism; and
wherein the plurality of hidden state segments as updated combine to provide the hidden state as updated.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein splitting the hidden state into the plurality of hidden state segments comprises splitting the hidden state into the plurality of hidden state segments based on a number of types.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein updating each hidden state segment using the attention mechanism further comprises:
appending a previous hidden state of the respective portion of the hidden state with a respective slice of each flag tensor and the respective data item and multiplying with a respective weight to generate each hidden state segment.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein the at least one prediction comprises a prediction of a next data item in at least one of the first time sequence, the second time sequence, or any combination thereof.
