# US 2026/0087386 A1 · Using cached experts in mixture of experts (moe)

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087386 A1
- Document ID: US20260087386A1
- Status: Published application
- Kind: A1
- Application number: 19/017,238
- Filed: 2025-01-10
- Published: 2026-03-26
- Earliest claimed: 2024-09-26
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Andrii SKLIAR; Babak EHTESHAMI BEJNORDI; Ties Jehan VAN ROZENDAAL; Marinus Willem VAN BAALEN; Markus NAGEL; Paul Nicholas WHATMOUGH
- Web page: https://patentgrep.com/patent/US20260087386A1

## CPC

- G06N 5/043: Distributed expert systems; Blackboards
- G06N 3/045: Combinations of networks
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and techniques are described herein for processing tokens. For instance, a method for processing tokens is provided. The method may include processing a token at a router model to generate a recommendation a subset of expert models from a plurality of expert models to use for further processing of the token; selecting a number of expert models to use for the further processing of the token based on the recommendation of the subset of expert models and based on cached expert models of the plurality of expert models stored in a cache memory; and processing the token using the selected number of expert models.

## Claims

### Claim 1

An apparatus for processing tokens, the apparatus comprising:
at least one memory, including a cache memory and a memory; and
at least one processor coupled to the at least one memory and configured to:
process a token at a router model to generate a recommendation of a subset of expert models from a plurality of expert models to use for further processing of the token;
select a number of expert models to use for the further processing of the token based on the recommendation of the subset of expert models and based on cached expert models of the plurality of expert models stored in the cache memory; and
process the token using the selected number of expert models.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the cached expert models are loaded into the cache memory to process a previous token.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to:
remove a first expert model of the cached expert models from the cache memory based on the first expert model not being among the selected number of expert models; and
load a second expert model of the plurality of expert models from the memory into the cache memory based on the second expert model being among the selected number of expert models.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to process the token using the selected number of expert models based on the selected number of expert models being among the cached expert models.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the recommendation comprises a respective expert-model rank of each expert model of the plurality of expert models, wherein the at least one processor is configured to select the number of expert models based on the respective expert-model rank of each expert model of the plurality of expert models.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the at least one processor is configured to select a highest-ranked expert model of the plurality of expert models to be among the selected number of expert models.

### Claim 7 (depends on 5)

The apparatus of claim 5, wherein the at least one processor is configured to select an expert model from among the cached expert models to be among the selected number of expert models based on an expert-model rank of the expert model exceeding a ranking threshold.

### Claim 8 (depends on 5)

The apparatus of claim 5, wherein the at least one processor is configured to, based on expert-model ranks of the cached expert models not exceeding a ranking threshold, select the number of expert models from among expert models not loaded into the cache memory.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the recommendation comprises a respective probability of each expert model of the plurality of expert models, wherein the at least one processor is configured to select the number of expert models based on the respective probability of each expert model of the plurality of expert models.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the at least one processor is configured to:
identify probable expert models from among the plurality of expert models based on a sum of probabilities associated with the probable expert models exceeding a probability threshold; and
select an expert model from among the cached expert models to be among the selected number of expert models based on the expert model being among the probable expert models.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein the at least one processor is configured to:
identify probable expert models from among the plurality of expert models based on a sum of probabilities associated with the probable expert models exceeding a probability threshold; and
based on the cached expert models not being among the probable expert models, select the number of expert models from among expert models not loaded into the cache memory.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the recommendation comprises a respective output value of each expert model of the plurality of expert models, wherein the at least one processor is configured to select the number of expert models based on the output values of the plurality of expert models and output values of the cached expert models.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the at least one processor is configured to select a highest-valued expert model of the plurality of expert models to be among the selected number of expert models.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein the at least one processor is configured to increase biasing the output values of the cached expert models for selecting the number of expert models.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the output values of the cached expert models are biased by a predetermined amount.

### Claim 16 (depends on 12)

The apparatus of claim 12, wherein the at least one processor is configured to:
process the output values of the plurality of expert models and a mask indicative of the cached expert models using a machine-learning model to generate biasing values; and
bias the output values of the cached expert models according to the biasing values for selecting the number of expert models.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the at least one processor is configured to process router input values using the machine-learning model to generate the biasing values.

### Claim 18

A method for processing tokens, the method comprising:
processing a token at a router model to generate a recommendation a subset of expert models from a plurality of expert models to use for further processing of the token;
selecting a number of expert models to use for the further processing of the token based on the recommendation of the subset of expert models and based on cached expert models of the plurality of expert models stored in a cache memory; and
processing the token using the selected number of expert models.

### Claim 19 (depends on 18)

The method of claim 18, wherein the cached expert models are loaded into the cache memory to process a previous token.

### Claim 20 (depends on 18)

The method of claim 18, further comprising:
removing a first expert model of the cached expert models from the cache memory based on the first expert model not being among the selected number of expert models; and
loading a second expert model of the plurality of expert models from a memory into the cache memory based on the second expert model being among the selected number of expert models.

### Claim 21 (depends on 18)

The method of claim 18, further comprising processing the token using the selected number of expert models based on the selected number of expert models being among the cached expert models.

### Claim 22 (depends on 18)

The method of claim 18, wherein the recommendation comprises a respective expert-model rank of each expert model of the plurality of expert models, the method further comprising selecting the number of expert models based on the respective expert-model rank of each expert model of the plurality of expert models.

### Claim 23 (depends on 22)

The method of claim 22, further comprising selecting a highest-ranked expert model of the plurality of expert models to be among the selected number of expert models.

### Claim 24 (depends on 22)

The method of claim 22, further comprising selecting an expert model from among the cached expert models to be among the selected number of expert models based on an expert-model rank of the expert model exceeding a ranking threshold.

### Claim 25 (depends on 22)

The method of claim 22, further comprising, based on expert-model ranks of the cached expert models not exceeding a ranking threshold, selecting the number of expert models from among expert models not loaded into the cache memory.

### Claim 26 (depends on 18)

The method of claim 18, wherein the recommendation comprises a respective probability of each expert model of the plurality of expert models, the method further comprising selecting the number of expert models based on the respective probability of each expert model of the plurality of expert models.

### Claim 27 (depends on 26)

The method of claim 26, further comprising:
identifying probable expert models from among the plurality of expert models based on a sum of probabilities associated with the probable expert models exceeding a probability threshold; and
selecting an expert model from among the cached expert models to be among the selected number of expert models based on the expert model being among the probable expert models.

### Claim 28 (depends on 26)

The method of claim 26, further comprising:
identifying probable expert models from among the plurality of expert models based on a sum of probabilities associated with the probable expert models exceeding a probability threshold; and
based on the cached expert models not being among the probable expert models, selecting the number of expert models from among expert models not loaded into the cache memory.

### Claim 29 (depends on 18)

The method of claim 18, wherein the recommendation comprises a respective output value of each expert model of the plurality of expert models, the method further comprising selecting the number of expert models based on the output values of the plurality of expert models and output values of the cached expert models.

### Claim 30 (depends on 29)

The method of claim 29, further comprising selecting a highest-valued expert model of the plurality of expert models to be among the selected number of expert models.

### Claim 31 (depends on 29)

The method of claim 29, further comprising increasing biasing the output values of the cached expert models for selecting the number of expert models.

### Claim 32 (depends on 31)

The method of claim 31, wherein the output values of the cached expert models are biased by a predetermined amount.

### Claim 33 (depends on 29)

The method of claim 29, further comprising:
processing the output values of the plurality of expert models and a mask indicative of the cached expert models using a machine-learning model to generate biasing values; and
biasing the output values of the cached expert models according to the biasing values for selecting the number of expert models.

### Claim 34 (depends on 33)

The method of claim 33, further comprising processing router input values using the machine-learning model to generate the biasing values.
