Unit test case generation with transformers
AssigneeMicrosoft Technology Licensing, LLC
InventorsDawn Drain, Neelakantan Sundaresan, Alexey Svyatkovskiy, Michele Tufano
Life of this patent
Estimating the term…
Abstract
A unit test generation system employs a neural transformer model with attention to generate candidate unit test sequences given a focal method of a programming language. The neural transformer model is pre-trained with source code programs and natural language text and fine-tuned with mapped test case pairs. A mapped test case pair includes a focal method and a unit test case for the focal method. In this manner, the neural transformer model is trained to learn the semantics and statistical properties of a natural language, the syntax of a programming language and the relationships between the code elements of the programming language and the syntax of a unit test case.
Claims14
A system comprising:
- one or more processors;
- and
- a memory that stores one or more programs that are configured to be executed by the one or more processors, the one or more programs including instructions to perform actions that:
- pre-train a first neural transformer model with attention with a first unsupervised training dataset to learn semantic relationships of a natural language, the first unsupervised training dataset comprising a plurality of sequences of the natural language;
- pre-train the first neural transformer model with attention with a second unsupervised training dataset to learn syntax of a programming language and se mantic relationships of code elements of the programming language, the second unsupervised training dataset comprising a plurality of sequences of source code from source code methods of the programming language;
- fine-tune the first neural transformer model with attention with a supervised training dataset to learn to generate a unit test case fora given focal method written in the programming language, wherein the supervised dataset comprises mapped unit test case pairs, wherein a mapped unit test case pair of the mapped unit test case pairs includes a focal method and an associated unit test case written in the programming language, wherein fine-tuning the first neural transformer model with attention with the supervised training dataset is a translation task with a training objective that learns a mapping of a focal method to a unit test case, fmi→tci, as a conditional probability P(tci, fmi);
- and
- deploy the first neural transformer model with attention in a unit test case generation system to predict a unit test case fora target method of a source code program.
A method performed by one or more computing devices, comprising:
- pre-training a neural transformer model with attention with a first unsupervised training dataset comprising a plurality of sequences of natural language text to learn semantic relationships of a natural language;
- pre-training the neural transformer model with attention with a second unsupervised training dataset comprising a plurality of sequences of source code to learn syntax of a programming language and semantic relationships of code elements of the programming language;
- fine-tuning the neural transformer model with attention with a supervised training dataset to learn to generate a unit test case fora given focal method written in the programming language, wherein the supervised dataset comprises mapped unit test case pairs, wherein a mapped unit test case pair of the mapped unit test case pairs comprises a first focal method and an associated unit test case written in the programming language, wherein fine-tuning the neural transformer model with attention with the supervised training dataset is a translation task with a training objective that learns a mapping of a focal method to a unit test case, fmi→tci, as a conditional probability P(tci, fmi);
- and
- deploying the neural transformer model with attention to automatically predict a unit test case fora given target method of a source code program.
Dependencies are inferred from the claim text.