# US 2024/0004638 A1 · Systems and methods for detection of software vulnerability fix

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004638 A1
- Document ID: US20240004638A1
- Status: Published application
- Kind: A1
- Application number: 18/468,126
- Filed: 2023-09-15
- Published: 2024-01-04
- Earliest claimed: 2021-07-06
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Jiayuan ZHOU; Michael PACHECO; Xin XIA; Yuan WANG
- Web page: https://patentgrep.com/patent/US20240004638A1

## CPC

- G06F 8/65: Updates (security arrangements therefor G06F21/57)
- G06F 8/71: Version control (security arrangements therefor G06F21/57); Configuration management
- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 8/427: {Parsing}
- G06F 2221/033: Test or assess software
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods and systems are described for detecting and reporting a vulnerability fix in a code repository. A commit obtained from the code repository is preprocessed to generate file-level token sequences each representing a file-level code change for respective files. Respective file-level code change embedding vectors are generated by inputting each file-level token sequence into a transformer model, each file-level code change embedding vector being a vector representation of the file-level code change for the respective file. The file-level code change embedding vectors are combined into a commit-level code change embedding vector that represents all code changes contained in the commit. A predicted commit-level vulnerability fix score is generated by inputting the commit-level code change embedding vector into a classifier. A vulnerability fix report is outputted, containing an identification of the commit and the predicted commit-level vulnerability fix score.

## Claims

### Claim 1

A method comprising:
obtaining a commit from a code repository;
preprocessing the commit to generate one or more file-level token sequences each representing at least one file-level code change for respective one or more files;
generating respective one or more file-level code change embedding vectors by inputting each of the one or more file level token sequences into a transformer model, each respective file-level code change embedding vector being a vector representation of the at least one file-level code change for the respective file;
combining the one or more file-level code change embedding vectors into a commit-level code change embedding vector that is a vector representation of all code changes contained in the commit;
generating a predicted commit-level vulnerability fix score by inputting the commit level code change embedding vector into a classifier; and
outputting a vulnerability fix report containing an identification of the commit and the predicted commit-level vulnerability fix score associated with the commit.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
identifying, from an embedding vector database, one or more embedding vectors that are most similar to the commit-level code change embedding vector, the one or more identified embedding vectors being associated with respective one or more unique identifiers;
identifying, from a known vulnerability fix database, one or more known vulnerability fixes corresponding to the respective one or more unique identifiers; and
extracting vulnerability fix information for each of the identified one or more known vulnerability fixes from the known vulnerability fix database;
wherein the extracted vulnerability fix information is associated with the commit and included in the vulnerability fix report.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
each of the one or more file-level token sequences includes a key token; and
the transformer model processes each given file-level token sequence by encoding each token in the given file-level token sequence into a respective embedding vector, the embedding vector encoded from the key token being the file-level code change embedding vector for the given file-level token sequence.

### Claim 4 (depends on 3)

The method of claim 3, wherein preprocessing the commit comprises:
parsing the commit to extract any lines of added code for a given file, and tokenizing any lines of added code into a sequence of added code tokens;
parsing the commit to extract any lines of removed code for the given file, and tokenizing any lines of removed code into a sequence of removed code tokens;
parsing the commit to extract any code description, and tokenizing any code description into a sequence of description tokens; and
concatenating the sequence of description tokens, the sequence of added code tokens and the sequence of removed code tokens, and appending the key token, to generate the file-level token sequence representing code change for the given file.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
monitoring the code repository to detect any commit that is new to the code repository; and
obtaining the commit from the code repository in response to detection that the commit is new.

### Claim 6 (depends on 1)

The method of claim 1, wherein the transformer model is fine-tuned from a pre-trained language model that has been pre-trained to generate embedding vectors representing human language, the transformer model being fine-tuned to generate the one or more file-level code change embedding vectors representing software code changes.

### Claim 7 (depends on 6)

The method of claim 6, further comprising, prior to obtaining the commit from the code repository:
fine-tuning the transformer model from the pre-trained language model to generate file-level code change embedding vectors using a training dataset; and
training the classifier to generate commit-level vulnerability fix scores using the training dataset.

### Claim 8 (depends on 1)

The method of claim 1, wherein combining the one or more file-level code change embedding vectors into the commit-level code change embedding vector comprises averaging the one or more file-level code change embedding vectors.

### Claim 9 (depends on 1)

The method of claim 1, wherein combining the one or more file-level code change embedding vectors into the commit-level code change embedding vector comprises, for each vector element of the commit-level code change embedding vector, taking a maximum value of a corresponding vector element across all of the one or more file-level code change embedding vectors.

### Claim 10

A system comprising:
a processing unit configured to execute instructions to cause the system to:
obtain a commit from a code repository;
preprocess the commit to generate one or more file-level token sequences each representing at least one file-level code change for respective one or more files;
generate respective one or more file-level code change embedding vectors by inputting each of the one or more file level token sequences into a transformer model, each respective file-level code change embedding vector being a vector representation of the at least one file-level code change for the respective file;
combine the one or more file-level code change embedding vectors into a commit-level code change embedding vector that is a vector representation of all code changes contained in the commit;
generate a predicted commit-level vulnerability fix score by inputting the commit level code change embedding vector into a classifier; and
output a vulnerability fix report containing an identification of the commit and the predicted commit-level vulnerability fix score associated with the commit.

### Claim 11 (depends on 10)

The system of claim 10, wherein the processing unit is further configured to execute instructions to cause the system to:
identify, from an embedding vector database, one or more embedding vectors that are most similar to the commit-level code change embedding vector, the one or more identified embedding vectors being associated with respective one or more unique identifiers;
identify, from a known vulnerability fix database, one or more known vulnerability fixes corresponding to the respective one or more unique identifiers; and
extract vulnerability fix information for each of the identified one or more known vulnerability fixes from the known vulnerability fix database;
wherein the extracted vulnerability fix information is associated with the commit and included in the vulnerability fix report.

### Claim 12 (depends on 10)

The system of claim 10, wherein:
each of the one or more file-level token sequences includes a key token; and
the transformer model processes each given file-level token sequence by encoding each token in the given file-level token sequence into a respective embedding vector, the embedding vector encoded from the key token being the file-level code change embedding vector for the given file-level token sequence.

### Claim 13 (depends on 12)

The system of claim 12, wherein the processing unit is further configured to execute instructions to cause the system to preprocess the commit by:
parsing the commit to extract any lines of added code for a given file, and tokenizing any lines of added code into a sequence of added code tokens;
parsing the commit to extract any lines of removed code for the given file, and tokenizing any lines of removed code into a sequence of removed code tokens;
parsing the commit to extract any code description, and tokenizing any code description into a sequence of description tokens; and
concatenating the sequence of description tokens, the sequence of added code tokens and the sequence of removed code tokens, and appending the key token, to generate the file-level token sequence representing code change for the given file.

### Claim 14 (depends on 10)

The system of claim 10, wherein the processing unit is further configured to execute instructions to cause the system to:
monitor the code repository to detect any commit that is new to the code repository; and
obtain the commit from the code repository in response to detection that the commit is new.

### Claim 15 (depends on 14)

The system of claim 14, wherein the system is part of a monitoring system for monitoring the code repository.

### Claim 16 (depends on 10)

The system of claim 10, wherein the transformer model is fine-tuned from a pre-trained language model that has been pre-trained to generate embedding vectors representing human language, the transformer model being fine-tuned to generate the one or more file-level code change embedding vectors representing software code changes.

### Claim 17 (depends on 16)

The system of claim 16, wherein the processing unit is further configured to execute instructions to cause the system to, prior to obtaining the commit from the code repository:
fine-tune the transformer model from the pre-trained language model to generate file-level code change embedding vectors using a training dataset; and
train the classifier to generate commit-level vulnerability fix scores using the training dataset.

### Claim 18 (depends on 10)

The system of claim 10, wherein combining the one or more file-level code change embedding vectors into the commit-level code change embedding vector comprises averaging the one or more file-level code change embedding vectors.

### Claim 19 (depends on 10)

The system of claim 10, wherein combining the one or more file-level code change embedding vectors into the commit-level code change embedding vector comprises, for each vector element of the commit-level code change embedding vector, taking a maximum value of a corresponding vector element across all of the one or more file-level code change embedding vectors.

### Claim 20

A computer readable medium having instructions encoded thereon, wherein the instructions, when executed by a processing unit of a system, cause the system to:
obtain a commit from a code repository;
preprocess the commit to generate one or more file-level token sequences each representing at least one file-level code change for respective one or more files;
generate respective one or more file-level code change embedding vectors by inputting each of the one or more file level token sequences into a transformer model, each respective file-level code change embedding vector being a vector representation of the at least one file-level code change for the respective file;
combine the one or more file-level code change embedding vectors into a commit-level code change embedding vector that is a vector representation of all code changes contained in the commit;
generate a predicted commit-level vulnerability fix score by inputting the commit level code change embedding vector into a classifier; and
output a vulnerability fix report containing an identification of the commit and the predicted commit-level vulnerability fix score associated with the commit.
