# US 9,436,725 B1 · Live data center test framework > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,725 B1 - Document ID: US9436725B1 - Status: Granted - Kind: B1 - Application number: 13/781,407 - Filed: 2013-02-28 - Granted: 2016-09-06 - Earliest claimed: 2013-02-28 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Anirudh Balachandra Aithal; Michael David Marr - Web page: https://patentgrep.com/patent/US9436725B1 ## CPC - G06F 16/24: Querying - G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) - G06F 11/2205: {using arrangements specific to the hardware being tested} - G06F 11/263: Generation of test inputs, e.g. test vectors, patterns or sequences {; with adaptation of the tested hardware for testability with external testers} - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 11/3414: {Workload generation, e.g. scripts, playback} - G06F 11/3452: {Performance evaluation by statistical analysis} - G06F 2201/815: Virtual - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods are described for testing computing resources. In one embodiment, a search space of computing settings is analyzed in accordance with weighted data that maps computing performance parameters with the computing settings. A subset of the computing settings is selected to generate a test population to optimize at least one computing performance parameter. One or more computing devices in a computing environment are configured in accordance with the test population, and the test conditions are iteratively updated based on test results in accordance with the test population and a fitness function. ## Claims ### Claim 1 One or more non-transitory computer-readable storage media having collectively stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to: select a test population based on one or more performance metrics; and cause execution of tests in accordance with the test population and bias the test population to iteratively update the test population based on results of the tests, wherein the tests are executed on computing devices selected from a computing environment based on availability criteria in accordance with a computing configuration management function, wherein the computing configuration management function is configured to determine a cost for removal of the selected computing devices from the computing environment during the execution of the tests, wherein the tests are executed when the cost for removal of the selected computing devices is less than a threshold. ### Claim 2 (depends on 1) The non-transitory computer-readable storage media of claim 1, wherein the threshold is indicative of a market price for providing computing resources on the selected computing devices. ### Claim 3 (depends on 1) The non-transitory computer-readable storage media of claim 1, wherein the computing configuration management function is further configured to indicate when one or more computing devices have computing capacity available for a test in accordance with an auto scaling function. ### Claim 4 A computer-implemented method for computing device testing comprising: selecting a test population based on one or more performance metrics; and causing execution of tests in accordance with the test population and bias the test population to iteratively update the test population based on results of the tests, wherein the tests are executed on computing devices selected from a computing environment based on availability criteria in accordance with a computing configuration management function, wherein the computing configuration management function is configured to determine a cost for removal of the selected computing devices from the computing environment during the execution of the tests, wherein the tests are executed when the cost for removal of the selected computing devices is less than a threshold. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the threshold is indicative of a market price for providing computing resources on the selected computing devices. ### Claim 6 (depends on 4) The computer-implemented method of claim 4, wherein the computing configuration management function is further configured to indicate when one or more computing devices have computing capacity available for a test in accordance with an auto scaling function. ### Claim 7 A computing system for computing device testing comprising: at least one computing processor; and at least one memory in communication with the at least one processor, the at least one memory having stored therein computer instructions that, upon execution by the at least one processor, cause at least cause the computing system to: select a test population based on one or more performance metrics; and cause execution of tests in accordance with the test population and bias the test population to iteratively update the test population based on results of the tests, wherein the tests are executed on computing devices selected from a computing environment based on availability criteria in accordance with a computing configuration management function, wherein the computing configuration management function is configured to determine a cost for removal of the selected computing devices from the computing environment during the execution of the tests, wherein the tests are executed when the cost for removal of the selected computing devices is less than a threshold. ### Claim 8 (depends on 7) The computing system of claim 7, wherein the threshold is indicative of a market price for providing computing resources on the selected computing devices. ### Claim 9 (depends on 7) The computing system of claim 7, wherein the computing configuration management function is further configured to indicate when one or more computing devices have computing capacity available for a test in accordance with an auto scaling function.