US 2025/0086095 A1 · MACHINE LEARNING-BASED TEST SELECTION FOR AUTOMATED CODE VALIDATION

Abstract

A machine learning model is trained from characteristics of code changes and characteristics of tests to generate an output indicative of a likely test result of running a corresponding test on a code change. One or more machine learning models may be trained for a specific code repository and based on developer feedback. When a code change is generated by a developer to code in a code repository, a machine learning model is selected based on the repository and characteristics or features of the code change are extracted and input to the machine learning model. The machine learning model generates a model output indicative of the likely test results of running each of a plurality of different tests on the code change. The model output indicates how likely it is that each of the plurality of different tests will fail. Based on the model output, a test selection system selects a subset of the plurality of different tests that should be run against the code changes.

Claims20

Dependent claims

A computer implemented method of testing a code change to a code base in a code repository, comprising:

identifying a test identification model based on the code repository;
extracting a set of features from the code change and from a plurality of available tests;
running the identified test identification model based on the set of features;
generating, with the identified test identification model, a test identification output corresponding to each of the plurality of available tests, the test identification output being indicative of whether a corresponding test, if run against the code change, will fail;
and
selecting a set of tests to run against the code change based on the test identification output.

3 more elements

A computer implemented method, comprising:

obtaining test training data based on tests run against code changes to a code base in a code repository;
obtaining code training data based on the code changes;
generating a set of test training features from the test training data;
generating a set of code training features from the code training data;
and
training a plurality of different types of test identification models to receive an input code change and identify a set of tests, from a plurality of available tests, that are likely to fail when executed against the input code change, based on the test training features and the code training features.

3 more elements

A computing system, comprising:

a machine learned test predictor configured to receive a set of features corresponding to code changes to a code base and generate an output indicative of a predicted test result for a test run against the code change;
a test selector configured to automatically select the test for running against the code change based on predicted test result;
a test execution system configured to run the selected test against the code change;
and
a predictor training system configured to detect developer feedback relative to the selected test and include a feedback feature, generated based on the detected developer feedback, in training the machine learned test predictor.

2 more elements

Dependencies are inferred from the claim text.

Machine learning-based test selection for automated code validation

Life of this patent

Machine learning-based test selection for automated code validation

Life of this patent

Abstract

Claims20