# US 11,907,096 B2 · Database simulation modeling framework > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,096 B2 - Document ID: US11907096B2 - Status: Granted - Kind: B2 - Application number: 17/565,815 - Filed: 2021-12-30 - Published: 2023-07-06 - Granted: 2024-02-20 - Earliest claimed: 2021-12-30 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Zi Ye; Justin Grant Moeller; Ya Lin; Willis Lang - Web page: https://patentgrep.com/patent/US11907096B2 ## CPC - G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU] - G06F 11/3433: {for load management (allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029)} - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 11/3409: {for performance assessment} - G06F 11/3414: {Workload generation, e.g. scripts, playback} - G06F 11/3428: {Benchmarking} - G06F 11/3457: {Performance evaluation by simulation} - G06F 11/3684: {for test design, e.g. generating new test cases} - G06F 11/3698: Environments for analysis, debugging or testing of software - G06F 16/217: {Database tuning (G06F16/2282 takes precedence; database performance monitoring G06F11/3409)} - G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor - G06F 9/5072: {Grid computing} - G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)} - G06F 9/5083: {Techniques for rebalancing the load in a distributed system} - G06F 2201/80: Database-specific techniques - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, systems, and computer program products are provided for creating a resource management testing environment. An initial population of databases is established in a database ring, having an in initial count of databases and different types of databases that are determined based on an initial database population model. The initial population model receives ring classification information for the database ring from a ring grouping model. A sequence of database population-change events is generated based on a model, to change the population of the databases over time in the ring. An orchestration framework performs testing of resource manager operations based on the model-defined initial population of databases and the model-defined populations of databases changed over time. Model-defined resource usage metrics for each database are utilized to test the resource manager operations. Resource usage metrics and database add/drop events of a production system are used to train the models. ## Claims ### Claim 1 A system for creating a resource management testing environment, the system comprising: one or more processors; and one or more memory devices that store program code to be executed by the one or more processors, the program code comprising: an initial database (DB) population generator configured to: establish a model-defined initial population of databases in a database ring of a database system, the model-defined initial population of databases comprising a model-defined count of databases and proportions of different model-defined types of databases based on output of an initial database population model; a DB population-change generator configured to: generate a model-defined sequence of database population-change events that are used to change the population of the databases in the database ring over time based on output of a database population-change events model, wherein the database population-change events model is trained based on production telemetry data to mimic behavior of a production system; and an orchestration framework configured to: perform a test of resource management in the database system based on the model-defined initial population of databases and the model defined populations of databases changed over time based on the output of the database population-change events model. ### Claim 2 (depends on 1) The system of claim 1, wherein at least one of the following occurs according to the model-defined sequence of database population-change events: the model-defined count of databases within the database ring changes over time; or the proportions of the different model-defined types of databases within the database ring changes over time. ### Claim 3 (depends on 1) The system of claim 1, wherein a type of database of the different model-defined types of databases is determined based on one or more of the following: a type of software utilized for the database, a type of hardware utilized for the database, a generation of software utilized for the database, a generation of hardware utilized for the database, a configuration of software components of the database, a configuration of hardware components of the database, available disk capacity, available memory capacity, a number of compute cores, and remote or local database storage type. ### Claim 4 (depends on 1) The system of claim 1, wherein the model-defined sequence of database population-change events comprises at least one of a create database event or a drop database event. ### Claim 5 (depends on 1) The system of claim 1, wherein the program code further comprises a database ring grouping generator configured to: generate a plurality of model-defined ring groups comprising at least the database ring based on output of a ring grouping model trained based on resource utilization metrics of a production database system, wherein the database ring belongs to a classification that is based on at least one resource utilization metric dimension and at least one resource utilization metric level type defined based on the output of the ring grouping model. ### Claim 6 (depends on 1) The system of claim 1, wherein the program code further comprises a database resource usage generator configured to: generate a model-defined set of database resource usage metrics for a database in a current population of databases; wherein: the current population of databases comprises one of the initial population of databases or a subsequent population of databases, and the database in the current population of databases reports the model-defined set of database resource usage metrics to the orchestration framework for testing at least one of database placement, load balancing, or scaling operations in the database ring by the orchestration framework in response to the reported model-defined set of database resource usage metrics. ### Claim 7 (depends on 6) The system of claim 6, wherein the model-defined set of database resource usage metrics comprises at least one of model-defined compute resource usage, model-defined memory usage, or model-defined disk usage and models utilized for defining the set of database resource usage metrics are trained based on resource usage metrics of a production database system. ### Claim 8 A system, for creating a resource management testing environment, the system comprising: one or more processors; and one or more memory devices that store program code to be executed by the one or more processors, the program code comprising: an initial database (DB) population generator configured to: establish a model-defined initial population of databases in a database ring where each database in the initial population has an artificial initial resource utilization metric value determined based on output of an initial database population model, wherein the initial database population model is trained based on resource usage metrics of a production database system; wherein the initial population of databases in the database ring comprises a count of databases having different types of databases that are determined based on the initial database population model; and an orchestration framework configured to: perform a test of a resource manager for the database ring utilizing the artificial initial resource utilization metric value, wherein the resource manager receives the artificial initial resource utilization metric value from at least one database of the model defined initial population of databases. ### Claim 9 (depends on 8) The system of claim 8, wherein types of the different types of databases in the initial population comprise local-storage databases or remote-storage databases determined based on the initial database population model. ### Claim 10 (depends on 8) The system of claim 8, wherein types of the different types of databases in the initial population comprise different resource capacities or different resource counts in the initial population of databases determined based on the initial database population model. ### Claim 11 (depends on 8) The system of claim 8, wherein the initial database population model receives input comprising one or more test scenario parameters for: target resource utilization levels, target range of number of databases, and target ring group classification, for defining the initial population of databases and each artificial initial resource utilization metric value for each database of the initial population of databases. ### Claim 12 (depends on 8) The system of claim 8, wherein the database ring is classified according to at least one type of resource usage metric level determined based on output of a ring grouping model, wherein the ring grouping model is trained based on database resource usage metrics of a production database system and outputs a definition for at least one ring group comprising the database ring. ### Claim 13 (depends on 8) The system of claim 8, wherein the orchestration framework is further configured to: perform a test of the resource manager for the database ring utilizing a subsequent artificial resource utilization metric value of at least one database of a subsequent model defined population of databases; and perform a test of the resource manager in the database system based on a sequence of model defined database population-change events. ### Claim 14 A system for managing a resource management testing environment, the system comprising: one or more processors; and one or more memory devices that store program code to be executed by the one or more processors, the program code comprising: a database (DB) population-change generator configured to: generate a model-defined sequence of database population-change events that change a population of databases in a database ring over time based on output of a database population-change events model, wherein the database population-change events model is trained based on production telemetry data to mimic behavior of a production system; and an orchestration framework configured to: perform a test of resource management operations in the database ring over time during the model-defined sequence of database population-change events. ### Claim 15 (depends on 14) The system of claim 14, wherein a count of databases within the database ring changes over time according to the model-defined sequence of population-change events. ### Claim 16 (depends on 14) The system of claim 14, wherein types of different types of databases within the database ring change over time according to the model-defined sequence of database population-change events. ### Claim 17 (depends on 14) The system of claim 14, wherein the program code further comprises a database resource usage generator configured to: generate a model-defined set of database resource usage metrics for at least one database in the database ring for each changed population of databases in the database ring over time; wherein at least one database reports the model-defined set of database resource usage metrics to the orchestration framework for controlling at least one of database placement, load balancing, or scaling operations in the database ring by the orchestration framework in response to the reported model-defined set of database resource usage metrics. ### Claim 18 A system for creating a resource management testing environment, the system comprising: a processor; and a memory device that stores program code to be executed by the processor, the program code comprising: an initial database (DB) population generator configured to: establish a model-defined initial population of databases in a database ring of a database system, the model-defined initial population of databases comprising a model-defined count of databases and proportions of different model-defined types of databases based on output of an initial database population model; a DB population-change generator configured to: generate a model-defined sequence of database population-change events that are used to change the population of the databases in the database ring over time based on output of a database population-change events model; and an orchestration framework configured to: perform a test of resource management in the database system based on the model-defined initial population of databases and the model defined populations of databases changed over time based on the output of the database population-change events model; and a database ring grouping generator configured to: generate a plurality of model-defined ring groups comprising at least the database ring based on output of a ring grouping model trained based on resource utilization metrics of a production database system, wherein the database ring belongs to a classification that is based on at least one resource utilization metric dimension and at least one resource utilization metric level type defined based on the output of the ring grouping model. ### Claim 19 A system for creating a resource management testing environment, the system comprising: a processor; and a memory device that stores program code to be executed by the processor, the program code comprising: an initial database (DB) population generator configured to: establish a model-defined initial population of databases in a database ring of a database system, the model-defined initial population of databases comprising a model-defined count of databases and proportions of different model-defined types of databases based on output of an initial database population model; a DB population-change generator configured to: generate a model-defined sequence of database population-change events that are used to change the population of the databases in the database ring over time based on output of a database population-change events model; and an orchestration framework configured to: perform a test of resource management in the database system based on the model-defined initial population of databases and the model defined populations of databases changed over time based on the output of the database population-change events model; and a database resource usage generator configured to: generate a model-defined set of database resource usage metrics for a database in a current population of databases; wherein: the current population of databases comprises one of the initial population of databases or a subsequent population of databases, and the database in the current population of databases reports the model-defined set of database resource usage metrics to the orchestration framework for testing at least one of database placement, load balancing, or scaling operations in the database ring by the orchestration framework in response to the reported model-defined set of database resource usage metrics. ### Claim 20 (depends on 19) The system of claim 19, wherein the database resource usage generator is further configured to: iteratively generate a new model-defined set of database resource usage metrics for a database of a subsequent current population of databases as the current population of databases iteratively changes over time, wherein each current population of databases comprises at least one of the model-defined initial population of databases or a subsequent model-defined population of databases. ### Claim 21 (depends on 19) The system of claim 19, wherein the model-defined set of database resource usage metrics is determined based on individual database resource usage metrics models for each different type of resource usage metric including one or more of the following individual database resource usage metrics models: a disk space model based on disk space changes between timestamps, the disk space model comprising one or more disk space sub-models, the one or more disk space sub-models including one or more of: a steady-state growth model for databases with local database storage and for databases with remote database storage; a predictable rapid growth model for databases with local database storage; or an initial creation growth model for created or restored databases; and a memory usage model comprising one or more memory usage sub-models, the one or more memory usage sub-models including one or more of: a steady-state growth model to model steady-state growth of database level memory usage; and an initial creation model to model memory usage in a time period at database creation time. ### Claim 22 (depends on 19) The system of claim 19, wherein: the current population of databases comprises a local storage database; the local storage database is configurable as stateful; the database resource usage generator comprises a disk usage model that generates disk usage metrics of the model-defined set of database resource usage metrics for the local storage database; and the database resource usage generator is further configured to: in response to the local storage database being configured as stateful, store the disk usage metrics to permanent memory for the local storage database to persist the state of the disk usage model; and read the disk usage metrics from permanent memory after a failover of the local storage database for reporting disk usage metrics to the orchestration framework after the failover. ### Claim 23 A system, for creating a resource management testing environment, the system comprising: a processor; and a memory device that stores program code to be executed by the processor, the program code comprising: an initial database (DB) population generator configured to: establish a model-defined initial population of databases in a database ring where each database in the initial population has an artificial initial resource utilization metric value determined based on output of an initial database population model, wherein the initial database population model is trained based on resource usage metrics of a production database system; wherein the initial population of databases in the database ring comprises a count of databases having different types of databases that are determined based on the initial database population model; and an orchestration framework configured to: perform a test of a resource manager for the database ring utilizing the artificial initial resource utilization metric value of at least one database of the model defined initial population of databases; perform a test of the resource manager for the database ring utilizing a subsequent artificial resource utilization metric value of at least one database of a subsequent model defined population of databases; and perform a test of the resource manager in the database system based on a sequence of model defined database population-change events.