# US 2025/0156874 A1 · Generating estimated future transactions using tokens and embeddings of past transactions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156874 A1
- Document ID: US20250156874A1
- Status: Published application
- Kind: A1
- Application number: 18/940,805
- Filed: 2024-11-07
- Published: 2025-05-15
- Earliest claimed: 2023-11-15
- Applicants: Mastercard International Incorporated
- Inventors: Rohit CHAUHAN; Hardik WADHWA; Nitendra RAJPUT; Yatin KATYAL
- Web page: https://patentgrep.com/patent/US20250156874A1

## CPC

- G06Q 20/4016: {involving fraud or risk level assessment in transaction processing}
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/30: Semantic analysis
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06Q 20/382: {insuring higher security of transaction}
- G06Q 30/0231: {Awarding of a frequent usage incentive independent of the monetary value of a good or service purchased, or distance travelled}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computerized method analyzes past transactions using machine learning (ML) models. A group of transactions associated with an account is obtained and the group of transactions is converted into a group of transaction strings using a subset of transaction features. For each transaction string in the group of transaction strings, a transaction token based on the transaction string is generated using a token vocabulary of a tokenizer. The generated transaction tokens associated with the group of transaction strings are provided to a ML model as input and an estimated transaction token is generated by the ML model based on the provided transaction tokens. The generated estimated transaction token is transformed into an estimated transaction using a de-tokenizer associated with the tokenizer. A new transaction associated with the account is approved using the estimated transaction, wherein the estimated transaction is compared to the new transaction.

## Claims

### Claim 1

A system comprising:
a processor; and
a memory comprising computer program code, the memory and the computer program code configured to cause the processor to:
obtain a group of transactions associated with an account;
convert the group of transactions into a group of transaction strings using a subset of transaction features of the group of transactions;
for each transaction string in the group of transaction strings, generate a transaction token based on the transaction string using a token vocabulary of a tokenizer, wherein the token vocabulary associates transaction strings with transaction tokens;
provide the generated transaction tokens associated with the group of transaction strings to a machine learning (ML) model as input;
generate, by the ML model, an estimated transaction token based on the provided transaction tokens;
transform the generated estimated transaction token into an estimated transaction using a de-tokenizer associated with the tokenizer, wherein the estimated transaction is associated with the account with which the obtained group of transactions is associated; and
approve a new transaction associated with the account using the estimated transaction, wherein the estimated transaction is compared to the new transaction.

### Claim 2 (depends on 1)

The system of claim 1, wherein converting the group of transactions into the group of transaction strings using a subset of transaction features of the group of transactions includes:
identifying a numeric transaction feature of the subset of transaction features;
determining a numeric value of the identified numeric transaction feature for a transaction of the group of transactions; and
converting the determined numeric value of the identified numeric transaction feature into a categorical feature value of a categorical feature that is associated with a numeric value range of the numeric transaction feature, wherein the transaction string of the transaction includes the categorical feature value to which the determined numeric value was converted.

### Claim 3 (depends on 1)

The system of claim 1, wherein approving the new transaction associated with the account using the estimated transaction includes:
generating an approval notification associated with the new transaction; and
causing the generated approval notification to be sent to one or more entities associated with the new transaction, wherein the one or more entities associated with the new transaction include a merchant, an acquirer, or an issuer.

### Claim 4 (depends on 1)

The system of claim 1, wherein the memory and the computer program code are configured to further cause the processor to:
estimate future account behavior of the account using the estimated transaction;
determine that the estimated future account behavior indicates a reduction in user loyalty with respect to the account; and
recommend a promotional offer to be offered to a user of the account, whereby the reduction in user loyalty with respect to the account is less likely.

### Claim 5 (depends on 1)

The system of claim 1, wherein the memory and the computer program code are configured to further cause the processor to:
generate, by the ML model, a second estimated transaction token based on the provided transaction tokens;
transform the second generated estimated transaction token into a second estimated transaction using a de-tokenizer associated with the tokenizer, wherein the second estimated transaction is associated with the account with which the obtained group of transactions is associated;
determine a difference between the second estimated transaction and a second new transaction associated with the account exceeds a threshold; and
generate a preemptive fraud indicator based on the determined difference exceeding the threshold.

### Claim 6 (depends on 1)

The system of claim 1, wherein the memory and the computer program code are configured to further cause the processor to:
generate an inventory management indicator associated with the estimated transaction, wherein the inventory management indicator indicates, to a receiving merchant, a need to order an item with which the estimated transaction is associated; and
cause the generated inventory management indicator to be sent to a merchant with which the estimated transaction is associated.

### Claim 7 (depends on 1)

The system of claim 1, wherein the memory and the computer program code are configured to further cause the processor to:
receive a natural language query associated with the account;
generate a natural language query response based on the estimated transaction; and
respond to the received natural language query with the generated natural language response.

### Claim 8

A computerized method comprising:
obtaining a group of transactions associated with an account;
for each transaction in the group of transactions, generating a group of embeddings using subsets of transaction features;
providing the groups of embeddings associated with the group of transactions to a transformer model as input;
generating, by the transformer model, an output group of embeddings based on the provided groups of embeddings;
decoding the output group of embeddings to form an estimated transaction, wherein the estimated transaction is associated with the account with which the obtained group of transactions is associated; and
approving a new transaction associated with the account using the estimated transaction, wherein the estimated transaction is compared to the new transaction.

### Claim 9 (depends on 8)

The computerized method of claim 8, wherein approving the new transaction associated with the account using the estimated transaction includes:
generating an approval notification associated with the new transaction; and
causing the generated approval notification to be sent to one or more entities associated with the new transaction, wherein the one or more entities associated with the new transaction include a merchant, an acquirer, or an issuer.

### Claim 10 (depends on 8)

The computerized method of claim 8, further comprising:
estimating future account behavior of the account using the estimated transaction;
determining that the estimated future account behavior indicates a reduction in user loyalty with respect to the account; and
recommending a promotional offer to be offered to a user of the account, whereby the reduction in user loyalty with respect to the account is less likely.

### Claim 11 (depends on 8)

The computerized method of claim 8, further comprising:
generating, by the transformer model, a second output group of embeddings based on the provided groups of embeddings;
transforming the second output group of embeddings to form a second estimated transaction, wherein the second estimated transaction is associated with the account with which the obtained group of transactions is associated;
determining a difference between the second estimated transaction and a second new transaction associated with the account exceeds a threshold; and
generating a preemptive fraud indicator based on the determined difference exceeding the threshold.

### Claim 12 (depends on 8)

The computerized method of claim 8, further comprising:
generating an inventory management indicator associated with the estimated transaction, wherein the inventory management indicator indicates, to a receiving merchant, a need to order an item with which the estimated transaction is associated; and
causing the generated inventory management indicator to be sent to a merchant with which the estimated transaction is associated.

### Claim 13 (depends on 8)

The computerized method of claim 8, further comprising:
receiving a natural language query associated with the account;
generating a natural language query response based on the estimated transaction; and
responding to the received natural language query with the generated natural language response.

### Claim 14

A computer storage medium has computer-executable instructions that, upon execution by a processor, cause the processor to at least:
obtain a group of transactions associated with an account;
convert the group of transactions into a group of transaction strings using a subset of transaction features of the group of transactions;
for each transaction string in the group of transaction strings, tokenize the transaction string using a tokenizer to obtain an associated transaction token, wherein the tokenizer includes a token vocabulary that associates transaction strings with transaction tokens;
provide transaction tokens associated with the group of transaction strings to an ML model as input;
generate, by the ML model, an estimated transaction token based on the provided transaction tokens;
transform the generated estimated transaction token into an estimated transaction using a de-tokenizer associated with the tokenizer, wherein the estimated transaction is associated with the account with which the obtained group of transactions is associated; and
approve a new transaction associated with the account using the estimated transaction, wherein the estimated transaction is compared to the new transaction.

### Claim 15 (depends on 14)

The computer storage medium of claim 14, wherein converting the group of transactions into the group of transaction strings using a subset of transaction features of the group of transactions includes:
identifying a numeric transaction feature of the subset of transaction features;
determining a numeric value of the identified numeric transaction feature for a transaction of the group of transactions; and
converting the determined numeric value of the identified numeric transaction feature into a categorical feature value of a categorical feature that is associated with a numeric value range of the numeric transaction feature, wherein the transaction string of the transaction includes the categorical feature value to which the determined numeric value was converted.

### Claim 16 (depends on 14)

The computer storage medium of claim 14, wherein approving the new transaction associated with the account using the estimated transaction includes:
generating an approval notification associated with the new transaction; and
causing the generated approval notification to be sent to one or more entities associated with the new transaction, wherein the one or more entities associated with the new transaction include a merchant, an acquirer, or an issuer.

### Claim 17 (depends on 14)

The computer storage medium of claim 14, wherein the computer-executable instructions, upon execution by the processor, further cause the processor to at least:
estimate future account behavior of the account using the estimated transaction;
determine that the estimated future account behavior indicates a reduction in user loyalty with respect to the account; and
recommend a promotional offer to be offered to a user of the account, whereby the reduction in user loyalty with respect to the account is less likely.

### Claim 18 (depends on 14)

The computer storage medium of claim 14, wherein the computer-executable instructions, upon execution by the processor, further cause the processor to at least:
generate, by the ML model, a second estimated transaction token based on the provided transaction tokens;
transform the second generated estimated transaction token into a second estimated transaction using a de-tokenizer associated with the tokenizer, wherein the second estimated transaction is associated with the account with which the obtained group of transactions is associated;
determine a difference between the second estimated transaction and a second new transaction associated with the account exceeds a threshold; and
generate a preemptive fraud indicator based on the determined difference exceeding the threshold.

### Claim 19 (depends on 14)

The computer storage medium of claim 14, wherein the computer-executable instructions, upon execution by the processor, further cause the processor to at least:
generate an inventory management indicator associated with the estimated transaction, wherein the inventory management indicator indicates, to a receiving merchant, a need to order an item with which the estimated transaction is associated; and
cause the generated inventory management indicator to be sent to a merchant with which the estimated transaction is associated.

### Claim 20 (depends on 14)

The computer storage medium of claim 14, wherein the computer-executable instructions, upon execution by the processor, further cause the processor to at least:
receive a natural language query associated with the account;
generate a natural language query response based on the estimated transaction; and
respond to the received natural language query with the generated natural language response.
