# US 2025/0086022 A1 · Methods and apparatuses for data processing, storage media, and electronic devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0086022 A1
- Document ID: US20250086022A1
- Status: Published application
- Kind: A1
- Application number: 18/569,295
- Filed: 2023-10-11
- Published: 2025-03-13
- Earliest claimed: 2023-03-10
- Assignees: Zhejiang Lab
- Applicants: Zhejiang Lab
- Inventors: Yong LI; Laiping ZHAO; Jie LI; Wen CHENG; Guang CHEN; Lingfang ZENG
- Web page: https://patentgrep.com/patent/US20250086022A1

## CPC

- G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU]
- G06F 9/5044: {considering hardware capabilities}
- G06F 9/48: Program initiating; Program switching, e.g. by interrupt
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5016: {the resource being the memory}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

A method for data processing is provided, and includes: obtaining each piece of to-be-processed data, determining whether a set amount of the to-be-processed data is capable to be processed under a current processing process by a data processing model, if not, obtaining data processing periods of the data processing model under multiple configuration combinations; for a data processing period of each of the multiple configuration combinations, determining an amount of data that is capable to be processed by the data processing model within the data processing period, as a target data amount; by taking the data processing model to be capable to process the set amount of the to-be-processed data as a target, according to the target data amount for a data processing period of each of the multiple configuration combinations, selecting a target configuration combination from the multiple configuration combinations.

## Claims

### Claim 1

A method for data processing, comprising:
obtaining to-be-processed data;
determining whether a set amount of the to-be-processed data is capable to be processed under a current processing process by a data processing model, in response to determining that the set amount of the to-be-processed data cannot be processed under the current processing process by the data processing model, obtaining data processing periods of the data processing model under multiple configuration combinations;
for a data processing period of each of the multiple configuration combinations, determining an amount of data that is capable to be processed by the data processing model within the data processing period, as a target data amount;
by taking the data processing model to be capable to process the set amount of the to-be-processed data as a target, according to the target data amount for the data processing period of each of the multiple configuration combinations, selecting a target configuration combination from the multiple configuration combinations, and creating a processing process under the target configuration combination; and
according to the processing process under the target configuration combination, performing data processing on the to-be-processed data.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the configuration combination comprises at least one of a quantity of central processing units, a batch processing size, data parallelism, or storage locations of a parameter tensor for an operator, wherein the storage locations of the parameter tensor comprise a local memory node and a remote memory node.

### Claim 3 (depends on 2)

The method according to claim 2, wherein before obtaining the data processing periods of the data processing model under the multiple configuration combinations, the method further comprises:
inputting the multiple configuration combinations into a preset performance estimating model, and for each of the multiple configuration combinations, determining a data processing period of each operator comprised in the data processing model under the configuration combination through the performance estimating model; and
according to the data processing period of each operator under the configuration combination, determining a corresponding data processing period of the data processing model under the configuration combination.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the method further comprises:
in a case where the data processing model comprises at least two parallel linear networks, determining a corresponding data processing period for each of the at least two linear networks under the configuration combination; and
taking a maximum value of the data processing periods for the at least two linear networks under the configuration combination as the data processing period of the data processing model under the configuration combination.

### Claim 5 (depends on 1)

The method according to claim 1, wherein by taking the data processing model to be capable to process the set amount of the to-be-processed data as the target, according to the target data amount for the data processing period of each of the multiple configuration combinations, selecting the target configuration combination from the multiple configuration combinations, and creating the processing process under the target configuration combination comprises:
according to the target data amount for the data processing period of each of the multiple configuration combinations, selecting the target configuration combination from the multiple configuration combinations, and creating the processing process under the target configuration combination;
determining whether the data processing model is capable to process the set amount of the to-be-processed data after creating the processing process under the target configuration combination; and
in response to determining that the data processing model is unable to process the set amount of the to-be-processed data, determining a new target configuration combination from other configuration combinations except the target configuration combination according to the target data amount for the data processing period of each of the multiple configuration combinations, and creating a new processing process under the new target configuration combination, until the data processing model is capable to process the set amount of the to-be-processed data.

### Claim 6 (depends on 2)

The method according to claim 2, wherein by taking the data processing model to be capable to process the set amount of the to-be-processed data as the target, according to the target data amount for the data processing period of each of the multiple configuration combinations, selecting the target configuration combination from the multiple configuration combinations, and creating the processing process under the target configuration combination comprises:
for each target configuration combination of the multiple configuration combinations, setting storage locations of parameter tensors in the target configuration combination as remote memory nodes; and
according to an actual target data amount of the data processing model, deleting an excess processing process.

### Claim 7 (depends on 2)

The method according to claim 2, wherein by taking the data processing model to be capable to process the set amount of the to-be-processed data as the target, according to the target data amount for the data processing period of each of the multiple configuration combinations, selecting the target configuration combination from the multiple configuration combinations comprises:
for each of the multiple configuration combinations, in response to determining that parameter tensors in the configuration combination are stored in the local memory node, determining the data processing period for the data processing model under the configuration combination as a first processing period;
for each of the parameter tensors in the configuration combination, determining the data processing period for the data processing model when the parameter tensor is stored in the remote memory node, as a second processing period;
according to the first processing period and the second processing period, determining an increased data processing period after changing the parameter tensor from being stored in the local memory node to being stored in the remote memory node, as a latency rise time;
according to a memory size occupied by the parameter tensor and the latency rise time for the parameter tensor, determining a greedy coefficient for the parameter tensor; and determining a parameter tensor with a highest greedy coefficient as a target parameter tensor that allows remote access;
determining the data processing period for the data processing model when the target parameter tensor is stored in the remote memory node; and determining whether the data processing period is greater than a preset service quality constraint period; and
in response to determining that the data processing period is less than or equal to the preset service quality constraint period, determining the configuration combination where the target parameter tensor is stored in the remote memory node as the target configuration combination.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the latency rise time for the parameter tensor is negatively correlated with the greedy coefficient for the parameter tensor, and the memory size occupied by the parameter tensor is positively correlated with the greedy coefficient for the parameter tensor.

### Claim 9 (depends on 7)

The method according to claim 7, wherein the method further comprises:
in response to determining that the data processing period is less than the service quality constraint period when the target parameter tensor is stored in the remote memory node, determining a next target parameter tensor and the data processing period for the data processing model when the next target parameter tensor is stored in the remote memory node, until the data processing period is greater than the service quality constraint period.

### Claim 10 (depends on 1)

The method according to claim 1, wherein according to the processing process under the target configuration combination, performing data processing on the to-be-processed data comprises:
according to multiple target configuration combinations respectively corresponding to multiple processing processes, determining a target data amount corresponding to each of the multiple processing processes; and
according to the target data amount corresponding to each of the multiple processing processes, allocating the to-be-processed data required to be processed by the multiple processing processes.

### Claim 11 (depends on 1)

The method according to claim 1, wherein according to the processing process under the target configuration combination, performing data processing on the to-be-processed data comprises:
determining parameter tensors that the data processing model needs to load;
for each of the parameter tensors, determining whether the parameter tensor is stored in a local memory node;
in response to determining that the parameter tensor is stored in the local memory node, mapping a parameter tensor memory in the local memory node through memory mapping and loading the parameter tensor; in response to determining that the parameter tensor is not stored in the local memory node, determining whether the parameter tensor is allowed to be stored in a remote memory node under a current configuration combination;
in a case where the parameter tensor is allowed to be stored in the remote memory node under the current configuration combination, in response to determining that the parameter tensor is stored in the remote memory node, mapping a parameter tensor memory in the remote memory node through memory mapping and loading the parameter tensor; and
according to the processing process under the target configuration combination and the loaded parameter tensor, performing data processing on the to-be-processed data.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the method further comprises:
in response to determining that the parameter tensor is not allowed to be stored in the remote memory node under the current configuration combination, creating a memory region in a local memory and adding the parameter tensor to a local memory node corresponding to the memory region.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the method further comprises:
in response to determining that the parameter tensor is allowed to be stored in the remote memory node under the current configuration combination and the parameter tensor is not stored in the remote memory node, creating a memory region in a local memory and adding the parameter tensor to a local memory node corresponding to the memory region.

### Claim 14 (depends on 11)

The method according to claim 11, wherein for each of the parameter tensors, determining whether the parameter tensor is stored in the local memory node comprises:
for each of the parameter tensors, determining a hash value for the parameter tensor; and
by taking the hash value as authentication information for the parameter tensor, accessing the local memory node and determining whether the parameter tensor is stored in the local memory node.

### Claim 15 (depends on 11)

The method according to claim 11, wherein according to the processing process under the target configuration combination, performing data processing on the to-be-processed data comprises:
according to a processing process comprised in the data processing model, determining mapping times for each of the parameter tensors;
according to the mapping times for each of the parameter tensors, determining a reference count for each of the parameter tensors; and
for each of the parameter tensors, in response to determining that the reference count for the parameter tensor is reset during data processing of the to-be-processed data, reclaiming a tensor memory for the parameter tensor.

### Claim 16 (depends on 2)

The method according to claim 2, wherein creating the processing process under the target configuration combination comprises:
for each processing process, according to a memory size and a CPU quantity for the processing process, filtering out server nodes that do not meet requirements of the configuration combination, and determining remaining server nodes as candidate server nodes;
for each of the candidate server nodes, determining a maximum memory that is allowed to be shared on the candidate server node; and
according to the maximum memory that is allowed to be shared on each of the candidate server nodes, selecting a designated server node as a target server node, and scheduling the processing process to the target server node.

### Claim 17 (depends on 2)

The method according to claim 2, wherein the method is applied to a serverless platform, and the local memory node and the remote memory node are memory nodes on a non-uniform-memory-access memory.

### Claim 18

(canceled)

### Claim 19 (depends on 1)

A computer readable storage medium, wherein the storage medium stores a computer program, and the computer program when executed by a processor achieves a method according to claim 1.

### Claim 20 (depends on 1)

An electronic device comprising a memory, a processor and a computer program stored on the memory and runnable on the processor, wherein the processor, when the program is executed by the processor, achieves a method according to claim 1.