Systems and methods for selecting test combinations of hardware and software features for feature validation

A system comprising:

• a processing system;
• and
• memory coupled to the processing system, the memory comprising computer executable instructions that, when executed by the processing system, perform operations comprising:
• receiving first metrics associated with a first batch of test combinations for multiple computing entities, the first metrics comprising:
• data associated with a first fleet prevalence and a first risk determined for the multiple computing entities;
• and
• a parameter indicating a degree of exploration versus a degree of exploitation for selecting a next test combination, wherein increasing the degree of exploration corresponds to increasing a probability of selecting the next test combination at random and increasing the degree of exploitation corresponds to increasing a probability the next test combination is selected based on at least one of a higher fleet prevalence or a higher risk than a previous test combination;
• providing the first metrics to a machine learning algorithm trained to determine an impact of features of the multiple computing entities on a performance of the multiple computing entities;
• determining a second batch of test combinations associated with a subset of the multiple computing entities based on an output of the machine learning algorithm, the second batch of test combinations including a subset of the first batch of test combinations;
• and
• executing the second batch of test combinations for the subset of the multiple computing entities.

A computer storage medium implemented on a device, the computer storage medium having computer code that, when executed on a processor, implements a method comprising:

• receiving first metrics associated with a first batch of test combinations for multiple computing entities, the first metrics comprising:
• data associated with a first fleet prevalence and a first risk determined for the multiple computing entities;
• and
• a parameter indicating a degree of exploration versus a degree of exploitation for selecting a next test combination, wherein increasing the degree of exploration corresponds to increasing a probability of selecting the next test combination at random and increasing the degree of exploitation corresponds to increasing a probability the next test combination is selected based on at least one of a higher fleet prevalence or a higher risk than a previous test combination;
• providing the first metrics to a machine learning algorithm trained to determine an impact of features of the multiple computing entities on a performance of the multiple computing entities;
• determining a second batch of test combinations associated with a subset of the multiple computing entities based on an output of the machine learning algorithm, the second batch of test combinations including a subset of the first batch of test combinations;
• and
• executing the second batch of test combinations for the subset of the multiple computing entities.