US 2020/0089651 A1 · USING MACHINE-LEARNING METHODS TO FACILITATE EXPERIMENTAL EVALUATION OF MODIFICATIONS TO A COMPUTATIONAL ENVIRONMENT WITHIN A DISTRIBUTED SYSTEM

Abstract

The present disclosure provides an experimentation framework for a computational environment in a distributed system. A machine-learning model may be created that predicts at least one output produced by the computational environment based on at least one input provided to the computational environment. During an evaluation time period that is subsequent to at least one modification being made to the computational environment, at least one modified output produced by the computational environment may be determined. The machine-learning model may be used to calculate at least one predicted output that would have been produced by the computational environment during the evaluation time period if the at least one modification had not been made. A determination may also be made about how the at least one modification affected the computational environment based on a comparison of the at least one modified output and the at least one predicted output.

Claims20

Dependent claims

A method for experimenting with modifications to a computational environment within a distributed system, comprising:

creating a machine-learning model for the computational environment, wherein the machine-learning model predicts at least one output produced by the computational environment based on at least one input provided to the computational environment;
determining at least one modified output produced by the computational environment within the distributed system, the at least one modified output being produced during an evaluation time period that is subsequent to at least one modification being made to the computational environment;
calculating at least one predicted output that would have been produced by the computational environment during the evaluation time period if the at least one modification had not been made to the computational environment, wherein the at least one predicted output is calculated using the machine-learning model;
comparing the at least one modified output and the at least one predicted output to produce comparison information that shows one or more differences between the at least one modified output and the at least one predicted output;
and
outputting an indication of how the at least one modification affected the computational environment based on the comparison information.

3 more elements

A system for experimenting with modifications to a computational environment within a distributed system, comprising:

one or more processors;
and
memory comprising instructions that are executable by the one or more processors to perform operations comprising:
creating a machine-learning model for the computational environment, wherein the machine-learning model predicts at least one output produced by the computational environment based on at least one input provided to the computational environment;
determining at least one modified output produced by the computational environment within the distributed system, the at least one modified output being produced during an evaluation time period that is subsequent to at least one modification being made to the computational environment;
calculating at least one predicted output that would have been produced by the computational environment during the evaluation time period if the at least one modification had not been made to the computational environment, wherein the at least one predicted output is calculated using the machine-learning model;
comparing the at least one modified output and the at least one predicted output to produce comparison information that shows one or more differences between the at least one modified output and the at least one predicted output;
and
outputting an indication of how the at least one modification affected the computational environment based on the comparison information.

6 more elements

A non-transitory computer-readable medium having computer-executable instructions stored thereon that, when executed, cause one or more processors to perform operations comprising:

creating a machine-learning model for a computational environment within a distributed system, wherein the machine-learning model predicts at least one output produced by the computational environment based on at least one input provided to the computational environment;
determining at least one modified output produced by the computational environment within the distributed system, the at least one modified output being produced during an evaluation time period that is subsequent to at least one modification being made to the computational environment;
calculating at least one predicted output that would have been produced by the computational environment during the evaluation time period if the at least one modification had not been made to the computational environment, wherein the at least one predicted output is calculated using the machine-learning model;
comparing the at least one modified output and the at least one predicted output to produce comparison information that shows one or more differences between the at least one modified output and the at least one predicted output;
and
outputting an indication of how the at least one modification affected the computational environment based on the comparison information.

3 more elements

Dependencies are inferred from the claim text.

Using machine-learning methods to facilitate experimental evaluation of modifications to a computational environment within a distributed system

Life of this patent

Using machine-learning methods to facilitate experimental evaluation of modifications to a computational environment within a distributed system

Life of this patent

Abstract

Claims20