US 2023/0141348 A1Published application

Method and apparatus for determining code generation quality and efficiency evaluation values based on multiple indicators

AssigneeKunming University of Science and Technology

InventorsYing JIANG, Wenjun YAO, Yang YANG, Shouguo TANG, Lingyu LI, Runxin LI

Life of this patent

Estimating the term…

Loading patent…

Abstract

The disclosure discloses a method and an apparatus for determining code generation quality and efficiency evaluation values based on multiple indicators. The method includes that: an object code is acquired; an interactive data set is acquired; a multidimensional evaluation indicators is determined and characteristic values corresponding to each piece of data in the interactive data set are extracted; characteristic values corresponding to each piece of data in the interactive data set are determined as a characteristic data set; a multidimensional syntax tree is constructed; feature learning is performed on the multidimensional syntax tree to obtain feature parameters; the evaluation parameter model is controlled to output an evaluation value. According to the disclosure, the evaluation method in a related art is to evaluate a static automatic code generation result, ignoring the influence of code changes on the evaluation result, resulting in low reliability of the evaluation result is solved.

Claims14

Dependent claims

A method for determining code generation quality and efficiency evaluation values based on multiple indicators, comprising:

acquiring an object code, the object code comprising a first code input by a target object and a second code recommended by a code generation tool and determined by the target object;
acquiring an interactive data set, the interactive data set comprising behavioral interaction data between the target object and the code generation tool;
determining multidimensional evaluation indicators and extracting a plurality of characteristic values corresponding to each piece of data in the interactive data set according to the multidimensional evaluation indicators, wherein each characteristic value corresponds to an evaluation indicator of the each dimension one by one, the multidimensional evaluation indicators being configured to evaluate quality of generating the second code and efficiency of generating the second code;
determining the plurality of the characteristic values corresponding to each piece of the data in the interactive data set as a characteristic data set;
constructing a multidimensional syntax tree according to the object code and the characteristic data set;
performing feature learning on the multidimensional syntax tree by a convolutional neural network to obtain feature parameters;
and
inputting the feature parameters to a value evaluation parameter model, and controlling the evaluation parameter model to output evaluation value, wherein the evaluation value comprises a first evaluation value and a second evaluation value, the first evaluation value being configured to evaluate the quality of generating the second code, and the second evaluation value being configured to evaluate the efficiency of generating the second code.

5 more elements

An apparatus for determining code generation quality and efficiency evaluation values based on multiple indicators, comprising:

a first acquisition unit, configured to acquire an object code, the object code comprising a first code input by a target object and a second code recommended by a code generation tool and determined by the target object;
a second acquisition unit, configured to acquire an interactive data set, the interactive data set comprising behavioral interaction data between the target object and the code generation tool;
a first determination unit, configured to determine a multidimensional evaluation indicators and extract a plurality of characteristic values corresponding to each piece of data in the interactive data set according to multidimensional evaluation indicators, wherein each characteristic value corresponds to evaluation indicator of each dimension one by one, the multidimensional evaluation indicators being configured to evaluate quality of generating the second code and efficiency of generating the second code;
a second determination unit, configured to determine the plurality of the characteristic values corresponding to each piece of the data in the interactive data set as a characteristic data set;
a construction unit, configured to construct a multidimensional syntax tree according to the object code and the characteristic data set;
a learning unit, configured to perform feature learning on the multidimensional syntax tree by a convolutional neural network to obtain feature parameters;
and
a first control unit, configured to input the feature parameters to a value evaluation parameter model, and control the evaluation parameter model to output an evaluation value, wherein the evaluation value comprises a first evaluation value and a second evaluation value, the first evaluation value being configured to evaluate the quality of generating the second code, and the second evaluation value being configured to evaluate the efficiency of generating the second code.

5 more elements

Dependencies are inferred from the claim text.