System and method for code smell detection using transformer-based code representations with self-supervision by predicting reserved words

A system for source code smell detection, comprising:

• processing circuitry configured to
• input a source code,
• create pseudo labels by a proxy task based on a vector of tokens for the source code,
• train a transformer model on the pseudo labels, as a pre-trained model that outputs a prediction of a value of tokens in the vector of tokens,
• apply the pre-trained model to a plurality of fine-tuning models for respective downstream tasks, wherein each fine-tuning model is created by training the pre-trained model, and
• output from each fine-tuning model, an indication of whether a code smell has been detected in the source code.

A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by processing circuitry, cause the processing circuitry to perform a method of analyzing computer source code to detect code smells, the method comprising:

• inputting, via the processing circuitry, the source code;
• creating, via the processing circuitry, pseudo labels by a proxy task based on a vector of tokens for the source code;
• training, via the processing circuitry, a transformer model on the pseudo labels, as a pre-trained model that outputs a prediction of a value of tokens in the vector of tokens;
• applying, via the processing circuitry, the pre-trained model to a plurality of fine-tuning models for respective downstream tasks, wherein each fine-tuning model is created by training the pre-trained model;
• and
• outputting, via the processing circuitry, from each fine-tuning model, an indication of whether a code smell has been detected in the source code.