US 12,112,287 B1Granted

Automated estimation of resources related to testing within a service provider network

AssigneeAmazon Technologies, Inc.

InventorsYuk Lun Patrick Kwan, Huang Li, Michael Ho, Gary Rittinger, Kavitha Thiyaghu

Life of this patent

Estimating the term…

Loading patent…

Abstract

This disclosure describes a test prediction service for predicting needed resources and associated costs for testing projects related to testing of new programs, services, etc., within a service provider network. The test prediction service uses a first machine learning model for predicting “hard costs” using data from a first data source that includes end-to-end test details such as associated resource and infrastructure use (costs) of the service provider network during previous end-to-end testing projects. A second machine learning model is used for predicting “soft costs” using data from a second data source that includes data, e.g., engineering headcount/hours spent developing the end-to-end testing and/or during previous end-to-end testing. The data is used to train the machine learning models. The test prediction service uses the trained machine learning models to generate estimated testing costs of new testing projects when a user enters project attributes for a new testing project.

Claims12

Dependent claims

A computer-implemented method comprising:

receiving, at a test prediction service within a service provider network, information related to end-to-end testing of a new commerce platform within the service provider network prior to launching of the new commerce platform within the service provider network;
determining, by the test prediction service and utilizing a first machine learning algorithm, needed first resources for the end-to-end testing of the new commerce platform prior to launching of the new commerce platform, wherein the needed first resources comprise services of the service provider network that are consumed during the end-to-end testing;
determining, by the test prediction service and using a second machine learning algorithm, needed second resources for the end-to-end testing of the new commerce platform prior to launching of the new commerce platform, wherein the needed second resources comprise a number of individuals and a corresponding amount of time of the number of individuals to be active in developing the testing of the new commerce platform and executing the testing of the new commerce platform;
and
based on (i) the information related to the new commerce platform within the service provider network and (ii) the first and second needed resources for testing of the new commerce platform prior to launching of the new commerce platform, predicting, by an estimation application programming interface (API) of the test prediction service, an estimated cost associated with (i) the needed first resources to perform the testing of the new commerce platform and (ii) the needed second resources to perform the testing of the new commerce platform,
wherein a first set of data obtained from a first service of the service provider network is utilized for training the first machine learning algorithm utilized by the test prediction service for predicting the services of the service provider network to be consumed during the testing,
wherein a second set of data obtained from a second service of the service provider network is utilized for training the second machine learning algorithm utilized by the test prediction service for predicting the number of individuals and the corresponding amount of time of the number of individuals to be active in developing the testing of the new commerce platform and executing the testing of the new commerce platform, and
wherein the predicting comprises:
consuming, by the estimation API, a first endpoint produced using the first machine learning algorithm;
and
consuming, by the estimation API, a second endpoint produced using the second machine learning algorithm.

8 more elements

A method comprising:

receiving, at a prediction service within a service provider network, information related to a new commerce platform within the service provider network;
prior to launching of the new commerce platform, determining, by the prediction service, (i) needed first resources for a test project for testing of the new commerce platform and (ii) needed second resources for the test project for testing of the new commerce platform, wherein the needed first resources comprise services of the service provider network to be consumed during the testing, and wherein the needed second resources comprise a number of individuals and a corresponding amount of time of the number of individuals to be active in developing the testing of the new commerce platform and executing the testing of the new commerce platform;
based at least in part on (i) the information related to the new commerce platform within the service provider network and (ii) the needed first and second resources for the test project for testing of the new commerce platform, predicting, by an estimation application programming interface (API) of the prediction service, an estimated cost associated with the needed first and second resources to perform the test project for testing of the new commerce platform;
and
based at least in part on the estimated cost, at least one of (i) prioritizing lower cost tests, (ii) scheduling usage of programs and services of the service provider network, (iii) adding a first test to the test project, (iv) removing a second test from the test project, (v) cancelling the test project, or (vi) cancelling the launching of the new commerce platform,
wherein a first set of data obtained from a first service of the service provider network is utilized for training a first machine learning algorithm utilized by the test prediction service for determining the services of the service provider network to be consumed during the testing,
wherein a second set of data obtained from a second service of the service provider network is utilized for training a second machine learning algorithm utilized by the prediction service for predicting the number of individuals and the corresponding amount of time of the number of individuals to be active in developing the testing of the new commerce platform and executing the testing of the new commerce platform, and
wherein the predicting comprises:
consuming, by the estimation API, a first endpoint produced by a first machine learning algorithm;
and
consuming, by the estimation API, a second endpoint produced by a second machine learning algorithm.

8 more elements

One or more computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform operations comprising:

receiving, at a prediction service within a service provider network, information related to a new commerce platform within the service provider network;
prior to launching of the new commerce platform, determining, by the prediction service, (i) needed first resources for a test project for testing of the new commerce platform and (ii) needed second resources for the test project for testing of the new commerce platform, wherein the needed first resources comprise services of the service provider network to be consumed during the testing, and wherein the needed second resources comprise a number of individuals and a corresponding amount of time of the number of individuals to be active in developing the testing of the new commerce platform and executing the testing of the new commerce platform;
based at least in part on (i) the information related to the new commerce platform within the service provider network and (ii) the needed first and second resources for the test project for testing of the new commerce platform, predicting, by an estimation application programming interface (API) of the prediction service, an estimated cost associated with the needed first and second resources to perform the test project for testing of the new commerce platform;
and
based at least in part on the estimated cost, at least one of (i) prioritizing lower cost tests, (ii) scheduling usage of programs and services of the service provider network, (iii) adding a first test to the test project, (iv) removing a second test from the test project, (v) cancelling the test project, or (vi) cancelling the launching of the new commerce platform,
wherein a first set of data obtained from a first service of the service provider network is utilized for training a first machine learning algorithm utilized by the test prediction service for determining the services of the service provider network to be consumed during the testing,
wherein a second set of data obtained from a second service of the service provider network is utilized for training a second machine learning algorithm utilized by the prediction service for predicting the number of individuals and the corresponding amount of time of the number of individuals to be active in developing the testing of the new commerce platform and executing the testing of the new commerce platform, and
wherein the predicting comprises:
consuming, by the estimation API, a first endpoint produced by a first machine learning algorithm;
and
consuming, by the estimation API, a second endpoint produced by a second machine learning algorithm.

8 more elements

Dependencies are inferred from the claim text.