Back to results
US 10,592,392 B2Granted
System and method for predictive performance analysis via dynamic benchmark generation
AssigneeInternational Business Machines Corporation
InventorsDavid Breitgand, Alex Glikson, Doron Podoleanu
Life of this patent
Estimating the term…
Abstract
This invention relates to generating benchmarks without the need for executing the application on a specific benchmarking platform or by testing the application against known inputs and outputs. Instead, the system, in an embodiment, allows for source code comparisons to known systems to predict performance. The benchmarking can be done for varying deployment platforms and various usage patterns. The source code comparison process allows for fast performance prediction by using a distance vector.
Claims17
A system for generating performance benchmark results of applications:
- a benchmarks repository comprising a plurality of results of running a plurality of benchmarks of application classes on a plurality of computing infrastructures;
- and
- a server comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor and configured to:
- i. receive an application source code, an infrastructure alternative, and a usage description of the application, the usage description describing at least one of an expected number of connections, expected transaction types, expectations for a business cycle, expectations for trends in demand, and maximal and minimal number of requests of a given type;
- ii. perform static analysis on the application source code by comparing a plurality of points in the application source code with a plurality of points in at least one benchmarked application class to determine a similarity of the application to a benchmarked application class on the received infrastructure alternative in the benchmarks repository comprising a plurality of results of running a plurality of benchmarks of application classes on a plurality of computing infrastructures, wherein each comparison of each of the plurality of points generates a value and the plurality of values form a similarity vector, wherein the comparison of each of the plurality of points is determined based on at least one of a minimal, maximal, and average number of connections, a workload type encoded, a number of transaction types, minimal, maximal, and average size of messages sent, and wherein each of the plurality of values of the formed similarity vector is weighted and the weights are adjusted based on an observed quality of prediction over time;
- iii. perform analysis on the usage description to determine a similarity to known usage, wherein the similarity is determined using the generated similarity vector;
- and
- iv. when the similarity of the application to known application classes is below a first threshold or the similarity to known usage is below a second threshold, generate a new custom benchmark from the application, execute it on the infrastructure alternative, and add the new benchmark to the benchmarks repository;
- and
- v. when the similarity of the application to known application classes is above the first threshold and the similarity to known usage is above the second threshold, generate a benchmark result for of the application based on results from the benchmark repository.
7 more elements
A method for system for generating performance benchmark results of applications, the method comprising:
- in a server comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, receiving a source code for an application that is executable on a computing infrastructure;
- receiving a usage description of the application, the usage description describing at least one of an expected number of connections, expected transaction types, expectations for a business cycle, expectations for trends in demand, and maximal and minimal number of requests of a given type;
- performing static analysis on the source code by comparing a plurality of points in the application source code with a plurality of points in at least one benchmarked application class to determine a similarity of the application to a benchmarked application class on the computing infrastructure in a benchmarks repository comprising a plurality of results of running a plurality of benchmarks of application classes on a plurality of computing infrastructures, wherein each comparison of each of the plurality of points generates a value and the plurality of values form a similarity vector, wherein the comparison of each of the plurality of points is determined based on at least one of a minimal, maximal, and average number of connections, a workload type encoded, a number of transaction types, minimal, maximal, and average size of messages sent, and wherein each of the plurality of values of the formed similarity vector is weighted and the weights are adjusted based on an observed quality of prediction over time;
- performing analysis on the usage description to determine a similarity to known usage, wherein the similarity is determined using the generated similarity vector;
- and
- when the similarity of the application to known application classes is below a first threshold or the similarity to known usage is below a second threshold, generate a new custom benchmark from the application, execute it on the infrastructure alternative, and add the new benchmark to the benchmarks repository;
- and
- when the similarity of the application to known application classes is above the first threshold and the similarity to known usage is above the second threshold, generate a benchmark result for of the application based on results from the benchmark repository.
5 more elements
Dependencies are inferred from the claim text.