# US 2026/0065158 A1 · Methods, apparatuses and computer program products to facilitate streaming processing of large language model inputs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065158 A1
- Document ID: US20260065158A1
- Status: Published application
- Kind: A1
- Application number: 19/311,811
- Filed: 2025-08-27
- Published: 2026-03-05
- Earliest claimed: 2024-08-30
- Applicants: Meta Platforms, Inc.
- Inventors: Frank Torsten Bernd Seide; Yangyang Shi
- Web page: https://patentgrep.com/patent/US20260065158A1

## CPC

- G06N 20/00: Machine learning
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method to train a large language model are provided. The system may access training data including one or more events and including one or more frames of a fixed duration. The system may further generate a label sequence based on the training data, and the system may determine an interleaved embedding sequence from the label sequence. The system may further determine a probability distribution over one or more predicted tokens based at least in part on an embedding of the interleaved embedding sequence. The system may further determine a difference between the probability distribution over the one or more predicted tokens and the label sequence. The system may further modify one or more parameters of the large language model based on the determined difference.

## Claims

### Claim 1

A method, comprising:
accessing, by a machine learning model, training data including one or more events and one or more frames of a fixed duration;
generating a label sequence based on the training data;
determining an interleaved embedding sequence from the label sequence;
determining a probability distribution over one or more predicted tokens based at least in part on an embedding of the interleaved embedding sequence;
determining a difference between the probability distribution over the one or more predicted tokens and the label sequence; and
modifying one or more parameters of the machine learning model based on the determined difference.

### Claim 2 (depends on 1)

The method of claim 1, wherein the generating the label sequence based on the training data comprises:
associating an input condition with the one or more events;
generating a label based on the input condition; and
associating the one or more frames with the label.

### Claim 3 (depends on 2)

The method of claim 2, wherein the generating the label based on the input condition comprises detecting one or more tokens representing at least one of words and sub-words in the training data.

### Claim 4 (depends on 2)

The method of claim 2, wherein the associating the one or more frames with the label comprises:
determining a time corresponding to an appearance of the label in an utterance comprising the training data; and
placing the label in the one or more frames during a period corresponding to at least one of the time of the appearance of the label in the utterance or a time close to the appearance of the label in the utterance.

### Claim 5 (depends on 4)

The method of claim 4, wherein the placing the label in the one or more frames at the time corresponding to the appearance of the label in the utterance comprises:
determining a blank symbol corresponding to a period between the one or more events in the training data; and
placing the label in the one or more frames following the blank symbol.

### Claim 6 (depends on 1)

The method of claim 1, wherein the determining the interleaved embedding sequence from the label sequence comprises transforming the label sequence into a plurality of symbols corresponding to a beginning of the label sequence, an end of the label sequence, and the one or more frames.

### Claim 7 (depends on 1)

The method of claim 1, wherein the determining the probability distribution over the one or more predicted tokens based at least in part on the embedding of the interleaved embedding sequence comprises:
determining a probability distribution of an encoded vector associated with the one or more events, a respective number of blank symbols corresponding to a period between the one or more events in the training data, and a target output in the label sequence; and
selecting the predicted token based on the probability distribution.

### Claim 8 (depends on 1)

The method of claim 1, wherein the determining the difference between the probability distribution over the one or more predicted tokens and the label sequence comprises determining a gradient of a loss function that measures a prediction error of the machine learning model with respect to the one or more parameters.

### Claim 9 (depends on 1)

The method of claim 1, wherein the modifying the one or more parameters of the large language model based on the determined difference comprises adjusting the one or more parameters utilizing at least one gradient that reduces a prediction error of the machine learning model.

### Claim 10

An apparatus comprising:
one or more processors; and
at least one memory storing instructions, that when executed by the one or more processors, cause the apparatus to:
access, by a machine learning model, training data including one or more events and one or more frames of a fixed duration;
generate a label sequence based on the training data;
determine an interleaved embedding sequence from the label sequence;
determine a probability distribution over one or more predicted tokens based at least in part on an embedding of the interleaved embedding sequence;
determine a difference between the probability distribution over the one or more predicted tokens and the label sequence; and
modify one or more parameters of the machine learning model based on the determined difference.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein when the one or more processors execute the instructions to generate the label sequence, the apparatus is further configured to:
associate an input condition with the one or more events;
generate a label based on the input condition; and
associate the one or more frames with the label.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein when the one or more processors execute the instructions to generate the label based on the input condition, the apparatus is further configured to detect one or more tokens representing at least one of words and sub-words in the training data.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein when the one or more processors execute the instructions to associate the one or more frames with the label, the apparatus is further configured to:
detect a time corresponding to an appearance of the label in an utterance comprising the training data; and
place the label in the one or more frames at the time corresponding to an appearance of the label in the utterance.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein when the one or more processors execute the instructions to place the label in the one or more frames at the time corresponding to the appearance of the label in the utterance, the apparatus is further configured to:
detect a blank symbol corresponding to a period between the one or more events in the training data; and
place the label in the one or more frames during a period corresponding to at least one of the time of the appearance of the label in the utterance or a time close to the appearance of the label in the utterance.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein when the one or more processors execute the instructions to derive the interleaved embedding sequence from the label sequence, the apparatus is further configured to transform the label sequence into a plurality of symbols corresponding to a beginning of the label sequence, an end of the label sequence, and the one or more frames.

### Claim 16 (depends on 10)

The apparatus of claim 10, wherein when the one or more processors execute the instructions to determine the probability distribution over the one or more predicted tokens based at least in part on the embedding of the interleaved embedding sequence, the apparatus is further configured to:
determine a probability distribution of an encoded vector associated with the one or more events, a respective number of blank symbols corresponding to a period between the one or more events in the training data, and a target output in the label sequence; and
select the predicted token based on the probability distribution.

### Claim 17 (depends on 10)

The apparatus of claim 10, wherein when the one or more processors execute the instructions to determine a difference between the probability distribution over the one or more predicted tokens and the label sequence, the apparatus is further configured to calculate a gradient of a loss function that measures a prediction error of the machine learning model with respect to the one or more parameters.

### Claim 18 (depends on 10)

The apparatus of claim 10, wherein when the one or more processors execute the instructions to modify one or more parameters of the large language model based on the determined difference, the apparatus if further configured to adjust the one or more parameters utilizing at least one gradient that reduces a prediction error of the machine learning model.

### Claim 19

A non-transitory computer-readable medium storing instructions that, when executed, cause:
accessing, by a machine learning model, training data including one or more events and one or more frames of a fixed duration;
generating a label sequence based on the training data;
determining an interleaved embedding sequence from the label sequence;
determining a probability distribution over one or more predicted tokens based at least in part on an embedding of the interleaved embedding sequence; and
determining a difference between the probability distribution over the one or more predicted tokens and the label sequence; and
modifying one or more parameters of the machine learning model based on the determined difference.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein the instructions generating the label sequence based on the training data, when executed, further cause:
associating an input condition with the one or more events;
generating a label based on the input condition; and
associating the one or more frames with the label.
