Automatic performance testing and performance regression analysis in a continuous integration environment

A method performed by a processor of a computing device, comprising:

• recording performance information during performance tests performed on programming code;
• identifying a performance regression that occurred multiple of the performance tests, the identifying comprising:
• analyzing the recorded performance information to identify a performance change reflected in one or more performance metrics that occurred in the multiple performance tests, and
• determining whether the performance regression is observable in that the performance change in at least one of the one or more metrics for the multiple performance tests surpasses a tolerance level;
• analyzing the identified observable performance regression to generate a call graph for the callable units of functionality of the programming code executed during the performance tests, the call graph containing call relationships among the callable units of functionality of the programming code and performance information of the callable units of functionality of the program code;
• processing the call graph by applying at least one anchor point to the call graph, for each of at least two callable units of functionality, analyzing the call graph to identify a cause of the identified performance regression;
• and
• outputting information about the identified cause on an output device.

A non-transitory computer-readable storage medium storing instructions that cause a processor of a computing device to perform the following:

• recording performance information during performance tests performed in programming code;
• identifying a performance regression that occurred at multiple of the performance tests, the identifying comprising:
• analyzing recorded performance information to identify a performance change reflected in one or more performance metrics that occurred in the multiple performance tests, and
• determining whether the performance regression is observable in the performance change in at least one of the one or more performance metrics for the multiple performance tests surpasses a tolerance level;
• analyzing the identified observable performance regression to generate a data structure for callable units of functionality of the programming code executed during performing the performance tests, the data structure containing call relationships among the callable units of functionality of the program code and performance information of the callable units of functionality of the programming code;
• for reach of at least two callable units of functionality, analyzing the data structure to identify a cause of the identified performance regression;
• and
• outputting information about the identified cause on an output device.

A non-transitory computer-readable storage medium storing instructions that cause a processor of a computing device to perform the following:

• recording performance information during performance tests performed on programming code;
• identifying a performance regression that occurred multiple of the performance tests, the identifying comprising:
• analyzing the recorded performance information to identify a performance change reflected in one or more performance metrics that occurred in the multiple performance tests, and
• determining whether the performance regression is observable in that the performance change in at least one of the one or more metrics for the multiple performance tests surpasses a tolerance level;
• analyzing the identified observable performance regression to generate a call graph for the callable units of functionality of the programming code executed during the performance tests, the call graph containing call relationships among the callable units of functionality of the programming code and performance information of the callable units of functionality of the program code;
• processing the call graph by applying at least one anchor point to the call graph;
• for each of at least two callable units of functionality, analyzing the call graph to identify a cause of the identified performance regression;
• and
• outputting information about the identified cause on an output device.

A computing device, comprising:

• a memory storing processor-executable instructions;
• a processor for executing the processor-executable instructions to perform the following:
• recording performance information during performance tests performed on programming code;
• identifying a performance regression that occurred in multiple of the performance tests, the identifying comprising:
• analyzing the recorded performance information to identify a performance change reflected in one or more performance metric that occurred in the multiple performance tests, and
• determining whether the performance regression is observable in that the performance change in at least one of the one or more performance metrics for the multiple performance tests surpasses a tolerance level;
• analyzing the identified observable performance regression to generate a data structure for callable units of functionality of the programming code executed during performing the performance tests, the data structure containing call relationships among the callable units of functionality of the programming code and performance information of the callable units of functionality of the program code;
• for each of at least two callable units of functionality, analyzing the data structure to identify a cause of the identified performance regression;
• and
• outputting information about the identified cause on an output device.