# US 2026/0099697 A1 · Expert selection from mixture of experts in large language models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099697 A1
- Document ID: US20260099697A1
- Status: Published application
- Kind: A1
- Application number: 19/347,703
- Filed: 2025-10-01
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Usman SAJID; Marie Mai NGUYEN; Shuyi PEI; Younghoon KIM; Rekha PITCHUMANI
- Web page: https://patentgrep.com/patent/US20260099697A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system and a method for a machine learning (ML) model with expert selection are disclosed. The model includes a global selector and a pre-fetcher. The global selector is configured to manage a selection scheme having at least one of a global mode or a local mode. In the global mode, the global selector selects a global expert set from a mixture of experts (MoE) to generate a selected global expert set for each layer prior to an inference phase in the ML model. The pre-fetcher is configured to pre-fetch in the global mode the selected global expert set from a first memory into a second memory. The selected global expert set includes one or more global hot experts.

## Claims

### Claim 1

A device comprising:
a global selector configured to manage a selection scheme having at least one of a global mode or a local mode, wherein in the global mode the global selector selects a global expert set from a mixture of experts (MoE) to generate a selected global expert set for each layer prior to an inference phase in a machine learning (ML) model; and
a pre-fetcher configured to pre-fetch in the global mode the selected global expert set from a first memory into a second memory,
wherein the selected global expert set includes one or more global hot experts.

### Claim 2 (depends on 1)

The device of claim 1, wherein the first memory and the second memory are organized in a tiered memory arrangement.

### Claim 3 (depends on 1)

The device of claim 1,
wherein in the local mode, the each layer selects a local expert set from the MoE to generate a selected local expert set in the inference phase,
wherein the selected local expert set includes one or more local hot experts, and
wherein the pre-fetcher prefetches the selected local expert set for the each layer for the entire layer set from the first memory into the second memory prior to the inference phase.

### Claim 4 (depends on 3)

The device of claim 3,
wherein the selection scheme further includes a mixed mode,
wherein in the mixed mode, each layer selects one of the global expert set or the local expert set for each layer according to a selection flag associated with the each layer and generates the selected one of the global expert set or the local expert set for the each layer for the entire layer set prior to the inference phase, and
wherein in the mixed mode, the pre-fetcher prefetches the selected one of the global expert set or the local expert set for the each layer for the entire layer set from the first memory into the second memory prior to the inference phase.

### Claim 5 (depends on 1)

The device of claim 1, further comprising:
a table configured to store the selected global expert set for the each layer for the entire layer set, and
wherein the pre-fetcher pre-fetches in the global mode the selected global expert set using the table.

### Claim 6 (depends on 1)

The device of claim 1, wherein the MoE in the each layer includes a subnetwork set of a feedforward neural network (FFNN).

### Claim 7 (depends on 1)

The device of claim 1,
wherein the one or more global hot experts have a global performance index exceeding a global performance standard, and
wherein the one or more global hot experts are activated during the inference phase.

### Claim 8 (depends on 3)

The device of claim 3, wherein the one or more local hot experts have a local performance index exceeding a local performance standard.

### Claim 9 (depends on 4)

The device of claim 4, wherein the selection scheme is based on at least one of an operational standard, a performance metric, a utilization metric, or a context metric.

### Claim 10 (depends on 2)

The device of claim 2, wherein the first memory is a slow memory and the second memory is a fast memory.

### Claim 11

A method comprising:
selecting, in a global mode of a selection scheme, a global expert set from a mixture of experts (MoE) to generate a selected global expert set for each layer for an entire layer set prior to an inference phase in a machine learning (ML) model; and
pre-fetching, in the global mode, the selected global expert set from a first memory into a second memory,
wherein the selected global expert set includes one or more global hot experts.

### Claim 12 (depends on 11)

The method of claim 11, wherein the first memory and the second memory are organized in a tiered memory arrangement.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
selecting, by the each layer in a local mode of the selection scheme, a local expert set from the MoE to generate a selected local expert set; and
pre-fetching, in the local mode, the selected local expert set for the each layer for the entire layer set from the first memory into the second memory,
wherein the selected local expert set includes one or more local hot experts.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
selecting, by the each layer in a mixed mode of the selection scheme, one of the global expert set or the local expert set corresponding to the each layer according to a selection flag associated with the each layer;
generating, in the mixed mode, the selected one of the global expert set or the local expert set for the each layer for the entire layer set, and
pre-fetching, in the mixed mode, the selected one of the global expert set or the local expert set for the each layer for the entire layer set from the first memory into the second memory prior to the inference phase.

### Claim 15 (depends on 11)

The method of claim 11, further comprising:
storing the selected global expert set for the each layer for the entire layer set in a table, and
pre-fetching, in the global mode, the selected global expert set using the table.

### Claim 16 (depends on 11)

The method of claim 11, wherein the MoE in the each layer includes a subnetwork set of a feedforward neural network (FFNN).

### Claim 17 (depends on 11)

The method of claim 11,
wherein the one or more global hot experts have a global performance index exceeding a global performance standard, and
wherein the one or more global hot experts are activated during the inference phase.

### Claim 18 (depends on 13)

The method of claim 13, wherein the one or more local hot experts have a local performance index exceeding a local performance standard.

### Claim 19 (depends on 14)

The method of claim 14, wherein the selection scheme is based on at least one of an operational standard, a performance metric, a utilization metric, or a context metric.

### Claim 20

A system comprising:
an input token generator configured to generate an input token set;
a layer set configured to process the input token set;
an expert selector comprising:
a global selector configured to manage a selection scheme having at least one of a global mode or a local mode, wherein in the global mode the global selector selects a global expert set from a mixture of experts (MoE) to generate a selected global expert set for each layer for an entire layer set prior to an inference phase in a machine learning (ML) model, the global selector generating a selected global expert set for the each layer for the entire layer set; and
a pre-fetcher configured to pre-fetch in the global mode the selected global expert set from a first memory into a second memory,
wherein the selected global expert set includes one or more global hot experts.