# US 2026/0093245 A1 · AI-based System and Method for Optimizing Lot Dispatching in Semiconductor Fabrication Using Reinforcement Learning and Fab-wide Digital Twin

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093245 A1
- Document ID: US20260093245A1
- Status: Published application
- Kind: A1
- Application number: 18/900,735
- Filed: 2024-09-28
- Published: 2026-04-02
- Earliest claimed: 2024-09-28
- Assignees: Inspiring Atoms Pte Ltd
- Inventors: Yang Pan
- Web page: https://patentgrep.com/patent/US20260093245A1

## CPC

- G05B 19/41885: {characterised by modeling, simulation of the manufacturing system}
- G05B 19/41865: {characterised by job scheduling, process planning, material flow}
- G05B 19/41875: {characterised by quality surveillance of production}
- G05B 2219/2602: Wafer processing
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Disclosed herein is an AI-based system and method for optimizing lot dispatching in a semiconductor Fab using reinforcement learning (RL) and a Fab-wide digital twin. The system leverages a policy neural network and Monte Carlo Tree Search (MCTS) to enhance cycle time, output, and on-time delivery. The RL agent continuously trains in the background, adapting to changes in Fab operations, ensuring optimized real-time decision-making.

## Claims

### Claim 1

A lot dispatching system in a semiconductor Fab, comprising:
an AI machine including an AI engine;
a Fab-wide digital twin comprising models of various process systems, grouped by types based on manufacturing capacity available for processing a plurality of lots; and
a policy neural network configured to generate outputs representing the probability of selecting a lot for processing in an available process system, wherein the policy neural network is trained using a self-initiated reinforcement learning (RL) process by an RL agent within the AI engine, leveraging data generated by the digital twin.

### Claim 2 (depends on 1)

The system of claim 1, wherein the policy neural network comprises an input layer, a plurality of hidden layers, and an output layer, wherein the output layer generates outputs describing softmax and/or logistic functions for probability distributions of lots awaiting selection for processing by the available process system.

### Claim 3 (depends on 2)

The system of claim 2, wherein the self-initiated RL process further includes a Monte Carlo Tree Search (MCTS) program, which selects the lots to be processed based on the probability distributions.

### Claim 4 (depends on 3)

The system of claim 3, wherein the policy neural network includes states of the process systems and states of the lots as inputs, wherein the states of the process systems further include the available capacities for each type, and the states of the lots include uncompleted process steps and required capacity for each step, wherein the states define a node in a network representing a plurality of state-action pairs.

### Claim 5 (depends on 4)

The system of claim 4, wherein the policy neural network further includes an additional input for predicting availabilities of the process systems for a predefined future duration.

### Claim 6 (depends on 5)

The system of claim 5, wherein the availabilities of the process systems are determined using the digital twins of the process systems.

### Claim 7 (depends on 4)

The system of claim 4, wherein selecting a lot for processing constitutes an action, and the RL agent virtually executes the action using the Fab-wide digital twin, updating the states of the process systems and the lots, generating a new node in the network.

### Claim 8 (depends on 2)

The system of claim 2, wherein the policy neural network further includes a value predictor for assessing the quality of the action.

### Claim 9 (depends on 1)

The system of claim 1, wherein the process systems comprise lithography, etching, deposition, cleaning, implantation, diffusion, metallization, chemical mechanical planarization (CMP), and metrology.

### Claim 10

A method for dispatching lots in a semiconductor Fab, comprising:
initiating, by an RL agent of an AI engine in an AI machine, an episode for training a policy neural network through an RL process, wherein the episode includes a plurality of simulated cases leveraging a Fab-wide digital twin, and wherein the Fab-wide digital twin includes digital twins for various types of process systems;
assigning, by the RL agent, weights to the policy neural network, wherein the policy neural network includes an input layer, a plurality of hidden layers, and an output layer, and wherein the output layer includes outputs describing softmax or logistic functions for generating probability distributions of the lots to be selected for processing by an available process system;
establishing, by the RL agent, a node associated with the states of the process systems and the states of the lots, and expanding the node into a network comprising a plurality of nodes consisting of a plurality of state-action pairs, wherein the RL agent employs the policy neural network and a Monte Carlo Tree Search (MCTS) program to form the state-action pairs, and wherein the states are generated based on the Fab-wide digital twin;
calculating, by the RL agent, a reward for each case, wherein the case includes a chain of state-actions, and wherein the last state is a terminal state meeting criteria for the reward calculation;
determining, by the RL agent, a reward for each state-action pair;
determining a value for each node after the episode is completed;
updating the weights of the policy neural network by leveraging the determined rewards for the state-action pairs and the value for the node, whereby the updated policy neural network becomes more efficient in generating actions with higher value;
finalizing the policy neural network after the RL process has converged; and
applying the trained policy neural network for real-world applications.

### Claim 11 (depends on 10)

The method of claim 10, wherein the policy neural network further includes an additional input for the predicted future availabilities of the process systems, wherein these availabilities are generated based on the digital twins of the process systems.

### Claim 12 (depends on 10)

The method of claim 10, wherein the policy neural network further includes a value predictor as an output, wherein the updated weights further improve the accuracy of the value predictions.

### Claim 13 (depends on 10)

The method of claim 10, wherein multiple episodes are executed to train the policy neural network through reinforcement learning process, with each episode comprising several simulated cases leveraging the Fab-wide digital twin.

### Claim 14 (depends on 10)

The method of claim 10, wherein the RL agent employs strategies to encourage exploration during the training process, including the use of an ϵ-greedy algorithm to balance exploration and exploitation.

### Claim 15 (depends on 10)

The method of claim 10, wherein the process system digital twin comprises models for lithography, etching, deposition, cleaning, implantation, diffusion, metallization, chemical mechanical planarization (CMP), and metrology, each representing the respective process systems in the Fab.

### Claim 16

An AI machine for coordinating operations of a Fab, comprising:
an AI engine comprising a compute engine, wherein the compute engine utilizes a GPU, a high-bandwidth memory (HBM), and a compute unified device architecture (CUDA);
an RL agent, part of the compute engine, designed to conduct a self-initiated reinforcement learning process to train a policy neural network, wherein the trained policy neural network generates probability distributions for selecting lots pending for processing using an available process system; and
a Fab-wide digital twin that generates synthetic data to support the reinforcement learning process, wherein the Fab-wide digital twin comprises digital twins for various process systems in the Fab.

### Claim 17 (depends on 16)

The AI machine of claim 16, wherein the RL agent employs a Monte Carlo Tree Search (MCTS) program to explore possible future states and actions, building a network of state-action pairs to guide decision-making in lot dispatching.

### Claim 18 (depends on 16)

The AI machine of claim 16, wherein the policy neural network includes an input layer with multiple inputs, a plurality of hidden layers, and an output layer with probability distributions, wherein the inputs further comprise states of the process systems, states of the lots, and predicted future availabilities of the process systems for a predefined time window.

### Claim 19 (depends on 16)

The AI machine of claim 16, wherein the process system digital twin further includes models for lithography, etching, deposition, cleaning, implantation, diffusion, metallization, chemical mechanical planarization (CMP), and metrology, each represents a type of process system, and each can be optionally calibrated based on real-time data.

### Claim 20 (depends on 16)

The AI machine of claim 16, wherein the process system digital twins further include mechanisms to predict future states based on measured data from sensors within the process systems, providing real-time feedback to the reinforcement learning process after the policy neural network is trained.
