# US 2026/0099303 A1 · Code generation method and apparatus, storage medium and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099303 A1
- Document ID: US20260099303A1
- Status: Published application
- Kind: A1
- Application number: 19/114,774
- Filed: 2024-02-07
- Published: 2026-04-09
- Earliest claimed: 2023-02-23
- Assignees: Douyin Vision Co., Ltd.
- Applicants: Douyin Vision Co., Ltd.
- Inventors: Luping Liu; Qinyun Wu; Ping Yang; Landong Lin; Jie Liu; Huan Song
- Web page: https://patentgrep.com/patent/US20260099303A1

## CPC

- G06F 8/35: model driven
- G06F 8/30: Creation or generation of source code
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/08: Learning methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure relates to a code generation method and apparatus, a storage medium, and an electronic device, to improve the accuracy of automatically generated code. The method includes: acquiring target text, where the target text includes program code text to be supplemented or natural language text for describing a code function; and inputting the target text into a code generation model to obtain a target program code generated based on the target text, where the code generation model is obtained by training a code understanding task and a code generation task, the code understanding task is used for the code generation model to learn a syntax feature and a semantic feature of a sample program code, and the code generation task is used for the code generation model to learn a process of generating a new program code based on the sample program code.

## Claims

### Claim 1

A code generation method, comprising:
acquiring target text, wherein the target text comprises program code text to be supplemented or natural language text for describing a code function; and
inputting the target text into a code generation model to obtain a target program code generated based on the target text, wherein the code generation model is obtained by training with a code understanding task and a code generation task, and the code understanding task is used for the code generation model to learn a syntax feature and a semantic feature of a sample program code, and the code generation task is used for the code generation model to learn a process of generating a new program code based on the sample program code.

### Claim 2 (depends on 1)

The method of claim 1, wherein the code generation model comprises a code understanding layer, a code generation layer and a shared representation layer, a result output by the shared representation layer is used for being input into the code understanding layer or the code generation layer, and a training step of the code generation model comprises:
in response to the code understanding task being executed, performing masking processing on the sample program code to obtain a mask sequence, and inputting the mask sequence into the code generation model to obtain a first prediction code by the shared representation layer and the code understanding layer, and adjusting parameters of the code understanding layer and parameters of the shared representation layer based on the first prediction code; and
in response to the code generation task being executed, inputting the sample program code into the code generation model to obtain a second prediction code by the shared representation layer and the code generation layer, and adjusting parameters of the code generation layer and parameters of the shared representation layer based on a result output by the code generation model.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing the masking processing on the sample program code to obtain the mask sequence comprises:
performing tokenization on the sample program code to obtain a token sequence;
selecting a first preset proportion of first target tokens in the token sequence randomly, and replacing the first target tokens with a first preset mask symbol to obtain the mask sequence; and/or
selecting a second preset proportion of second target tokens in the token sequence randomly, and selecting, for each second target token, a target remaining token from remaining tokens randomly, and replacing the second target token with the target remaining token to obtain the mask sequence, wherein the remaining tokens are tokens in the token sequence except the second target tokens.

### Claim 4 (depends on 2)

The method of claim 2, wherein performing the masking processing on the sample program code to obtain the mask sequence comprises:
performing tokenization on the sample program code to obtain a token sequence;
determining a function token for characterizing a function name in the token sequence, randomly selecting target function tokens in the function tokens by a third preset proportion, and replacing the target function tokens with a second preset mask symbol to obtain the mask sequence; and/or
determining an interface token for characterizing an interface name in the token sequence, randomly selecting a fourth preset proportion of target interface tokens in the interface tokens, and replacing the target interface tokens with a third preset mask symbol to obtain the mask sequence.

### Claim 5 (depends on 2)

The method of claim 2, wherein inputting the mask sequence into the code generation model to obtain the first prediction code by the shared representation layer and the code understanding layer comprises:
determining a feature vector sequence corresponding to the mask sequence;
for each feature vector in the feature vector sequence, obtaining an intermediate feature vector by the shared representation layer performing calculation according to all feature vectors located before and after the feature vector in the feature vector sequence and an attention mechanism;
for each intermediate feature vector, obtaining a target feature vector by the code understanding layer performing attention calculation according to intermediate feature vectors excluding the intermediate feature vector and the attention mechanism; and
obtaining a first prediction program code according to the target feature vector.

### Claim 6 (depends on 2)

The method of claim 2, wherein inputting the sample program code into the code generation model to obtain the second prediction code by the shared representation layer and the code generation layer comprises:
determining a feature vector sequence corresponding to the sample program code, wherein the feature vector sequence comprises a number N of feature vectors, and N is a positive integer;
for an ith feature vector in the feature vector sequence, obtaining an i-th intermediate feature vector by the shared representation layer performing calculation according to all feature vectors located before and after the i-th feature vector in the feature vector sequence and an attention mechanism, wherein i is a positive integer;
for the i-th intermediate feature vector, obtaining a target feature vector by the code generation layer performing according to preceding feature vectors and the attention mechanism, wherein the preceding feature vectors are all intermediate feature vectors obtained before the i-th intermediate feature vector; and
obtaining a second prediction program code according to the target feature vector.

### Claim 7 (depends on 1)

The method of claim 1, wherein the sample program code is obtained by:
acquiring a code dataset comprising a plurality of code files;
performing, for each of the code files, at least one of the following preprocessing steps to obtain a training dataset: in response to a proportion of a number of characters to a total number of symbols in a code file being greater than or equal to a preset character proportion, adding the code file to the training dataset; in response to an average number of characters per line of code in a code file being less than or equal to a first preset threshold, adding the code file to the training dataset; or in response to a number of characters of comment information in a code file being less than or equal to a second preset threshold, adding the code file to the training dataset; and
determining a program code corresponding to each of the code files in the training dataset as the sample program code.

### Claim 8 (depends on 1)

The method of claim 1, wherein the code generation model is used for obtaining the target program code generated based on the target text by:
determining a prediction word based on the target text, and using the prediction word as an initial target word to cyclically perform the following process:
concatenating the target token to a token sequence corresponding to the target text to obtain a target token sequence, and determining a plurality of candidate prediction words and a probability of each of the candidate prediction words in a preset vocabulary based on the target token sequence, and selecting a word from a candidate set corresponding to the candidate prediction words as a new target word randomly until a length of the target token sequence reaches a preset length or the target word is a preset end symbol;
wherein a sum of probabilities of the candidate prediction words in the candidate set is greater than a preset probability.

### Claim 9

(canceled)

### Claim 10

A non-transitory computer readable medium, storing a computer program thereon, wherein the program, when executed by a processing apparatus, cause the processing apparatus to:
acquire target text, wherein the target text comprises program code text to be supplemented or natural language text for describing a code function; and
input the target text into a code generation model to obtain a target program code generated based on the target text, wherein the code generation model is obtained by training with a code understanding task and a code generation task, and the code understanding task is used for the code generation model to learn a syntax feature and a semantic feature of a sample program code, and the code generation task is used for the code generation model to learn a process of generating a new program code based on the sample program code.

### Claim 11

An electronic device, comprising:
a storage apparatus, storing a computer program thereon; and
a processing apparatus, configured to execute the computer program in the storage apparatus to:
acquire target text, wherein the target text comprises program code text to be supplemented or natural language text for describing a code function; and
input the target text into a code generation model to obtain a target program code generated based on the target text, wherein the code generation model is obtained by training with a code understanding task and a code generation task, and the code understanding task is used for the code generation model to learn a syntax feature and a semantic feature of a sample program code, and the code generation task is used for the code generation model to learn a process of generating a new program code based on the sample program code.

### Claim 12 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the code generation model comprises a code understanding layer, a code generation layer and a shared representation layer, a result output by the shared representation layer is used for being input into the code understanding layer or the code generation layer, and in a training step of the code generation model, the processing apparatus is caused to:
in response to the code understanding task being executed, perform masking processing on the sample program code to obtain a mask sequence, and input the mask sequence into the code generation model to obtain a first prediction code by the shared representation layer and the code understanding layer, and adjust parameters of the code understanding layer and parameters of the shared representation layer based on the first prediction code; and
in response to the code generation task being executed, input the sample program code into the code generation model to obtain a second prediction code by the shared representation layer and the code generation layer, and adjust parameters of the code generation layer and parameters of the shared representation layer based on a result output by the code generation model.

### Claim 13 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the program that causes the processing apparatus to perform the masking processing on the sample program code to obtain the mask sequence further causes the processing apparatus to:
perform tokenization on the sample program code to obtain a token sequence;
select a first preset proportion of first target tokens in the token sequence randomly, and replace the first target tokens with a first preset mask symbol to obtain the mask sequence; and/or
select a second preset proportion of second target tokens in the token sequence randomly, and select, for each second target token, a target remaining token from remaining tokens randomly, and replace the second target token with the target remaining token to obtain the mask sequence, wherein the remaining tokens are tokens in the token sequence except the second target tokens.

### Claim 14 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the program that causes the processing apparatus to perform the masking processing on the sample program code to obtain the mask sequence further causes the processing apparatus to:
perform tokenization on the sample program code to obtain a token sequence;
determine a function token for characterizing a function name in the token sequence, randomly select target function tokens in the function tokens by a third preset proportion, and replace the target function tokens with a second preset mask symbol to obtain the mask sequence; and/or
determine an interface token for characterizing an interface name in the token sequence, randomly select a fourth preset proportion of target interface tokens in the interface tokens, and replace the target interface tokens with a third preset mask symbol to obtain the mask sequence.

### Claim 15 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the program that causes the processing apparatus to input the mask sequence into the code generation model to obtain the first prediction code by the shared representation layer and the code understanding layer comprises:
determine a feature vector sequence corresponding to the mask sequence;
for each feature vector in the feature vector sequence, obtain an intermediate feature vector by the shared representation layer performing calculation according to all feature vectors located before and after the feature vector in the feature vector sequence and an attention mechanism;
for each intermediate feature vector, obtain a target feature vector by the code understanding layer performing attention calculation according to intermediate feature vectors excluding the intermediate feature vector and the attention mechanism; and
obtain a first prediction program code according to the target feature vector.

### Claim 16 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the program that causes the processing apparatus to input the sample program code into the code generation model to obtain the second prediction code by the shared representation layer and the code generation layer further causes the processing apparatus to:
determine a feature vector sequence corresponding to the sample program code, wherein the feature vector sequence comprises a number N of feature vectors, and N is a positive integer;
for an ith feature vector in the feature vector sequence, obtain an i-th intermediate feature vector by the shared representation layer performing calculation according to all feature vectors located before and after the i-th feature vector in the feature vector sequence and an attention mechanism, wherein i is a positive integer;
for the i-th intermediate feature vector, obtain a target feature vector by the code generation layer performing according to preceding feature vectors and the attention mechanism, wherein the preceding feature vectors are all intermediate feature vectors obtained before the i-th intermediate feature vector; and
obtain a second prediction program code according to the target feature vector.

### Claim 17 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the sample program code is obtained by the processing apparatus being caused to:
acquire a code dataset comprising a plurality of code files;
perform, for each of the code files, at least one of the following preprocessing steps to obtain a training dataset: in response to a proportion of a number of characters to a total number of symbols in a code file being greater than or equal to a preset character proportion, add the code file to the training dataset; in response to an average number of characters per line of code in a code file being less than or equal to a first preset threshold, adding the code file to the training dataset; or in response to a number of characters of comment information in a code file being less than or equal to a second preset threshold, add the code file to the training dataset; and
determine a program code corresponding to each of the code files in the training dataset as the sample program code.

### Claim 18 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the code generation model is used for obtaining the target program code generated based on the target text by:
determining a prediction word based on the target text, and using the prediction word as an initial target word to cyclically perform the following process:
concatenating the target token to a token sequence corresponding to the target text to obtain a target token sequence, and determining a plurality of candidate prediction words and a probability of each of the candidate prediction words in a preset vocabulary based on the target token sequence, and selecting a word from a candidate set corresponding to the candidate prediction words as a new target word randomly until a length of the target token sequence reaches a preset length or the target word is a preset end symbol;
wherein a sum of probabilities of the candidate prediction words in the candidate set is greater than a preset probability.

### Claim 19 (depends on 11)

The electronic device of claim 11, wherein the code generation model comprises a code understanding layer, a code generation layer and a shared representation layer, a result output by the shared representation layer is used for being input into the code understanding layer or the code generation layer, and in a training step of the code generation model, the processing apparatus is caused to:
in response to the code understanding task being executed, perform masking processing on the sample program code to obtain a mask sequence, and input the mask sequence into the code generation model to obtain a first prediction code by the shared representation layer and the code understanding layer, and adjust parameters of the code understanding layer and parameters of the shared representation layer based on the first prediction code; and
in response to the code generation task being executed, input the sample program code into the code generation model to obtain a second prediction code by the shared representation layer and the code generation layer, and adjust parameters of the code generation layer and parameters of the shared representation layer based on a result output by the code generation model.

### Claim 20 (depends on 19)

The electronic device of claim 19, wherein the computer program that causes the processing apparatus to perform the masking processing on the sample program code to obtain the mask sequence further causes the computer processing apparatus to:
perform tokenization on the sample program code to obtain a token sequence;
select a first preset proportion of first target tokens in the token sequence randomly, and replace the first target tokens with a first preset mask symbol to obtain the mask sequence; and/or
select a second preset proportion of second target tokens in the token sequence randomly, and select, for each second target token, a target remaining token from remaining tokens randomly, and replace the second target token with the target remaining token to obtain the mask sequence, wherein the remaining tokens are tokens in the token sequence except the second target tokens.

### Claim 21 (depends on 19)

The electronic device of claim 19, wherein the computer program that causes the processing apparatus to perform the masking processing on the sample program code to obtain the mask sequence further causes the computer processing apparatus to:
perform tokenization on the sample program code to obtain a token sequence;
determine a function token for characterizing a function name in the token sequence, randomly select target function tokens in the function tokens by a third preset proportion, and replace the target function tokens with a second preset mask symbol to obtain the mask sequence; and/or
determine an interface token for characterizing an interface name in the token sequence, randomly select a fourth preset proportion of target interface tokens in the interface tokens, and replace the target interface tokens with a third preset mask symbol to obtain the mask sequence.
