Managed tuning for data clouds

A computer-implemented method to configure a data warehouse system, comprising:

• obtaining, by a configuration management system, historical query workload metadata associated with a data warehouse from the data warehouse system;
• determining, based on the historical query workload metadata, a predictive schedule that includes a first configuration setting associated with a configurable parameter for a first time period, wherein the first time period is a future time period, wherein the first configuration setting is associated with a lower computing resource utilization at the data warehouse system when compared to a previous configuration setting, wherein the configurable parameter is a warehouse size for a data warehouse, and wherein determining the first configuration setting for the warehouse size for the first time period comprises:
• determining a plurality of query latencies for queries included in the historical query workload metadata;
• determining a plurality of time window specific threshold percentile latencies that correspond to a predetermined threshold percentile of query latencies of the plurality of query latencies for queries that were processed by the data warehouse system during a corresponding time window of a plurality of time windows in the historical query workload metadata;
• determining a reference threshold percentile latency that is based on a predetermined threshold percentile of the plurality of time window specific threshold percentile latencies;
• and
• setting the first configuration setting to a specified setting of a plurality of specified settings if the time window specific threshold percentile latency for the first time period is less than the reference threshold percentile latency, wherein the specified setting is lower than the previous configuration setting;
• transmitting, to the data warehouse system, the first configuration setting for the configurable parameter;
• receiving, from the data warehouse system, during the first time period, query workload metadata;
• determining, based on the query workload metadata, whether the query workload metadata meets a threshold performance;
• and
• based on a determination that the query workload metadata does not meet the threshold performance, transmitting a backoff configuration setting for the configurable parameter to the data warehouse system.

A computer-implemented method to configure a data warehouse system, comprising:

• obtaining, by a configuration management system, historical query workload metadata associated with a data warehouse from the data warehouse system;
• determining, based on the historical query workload metadata, a predictive schedule that includes a first configuration setting associated with a configurable parameter for a first time period, wherein the first time period is a future time period, wherein the first configuration setting is associated with a lower computing resource utilization at the data warehouse system when compared to a previous configuration setting, wherein the configurable parameter is a maximum number of warehouse clusters per data warehouse, and wherein determining the first configuration setting for the first time period comprises:
• determining a plurality of time window specific threshold percentile maximum cluster counts that correspond to a predetermined threshold percentile of maximum cluster counts for queries that were processed by the data warehouse system during a corresponding time window in the historical query workload metadata;
• determining a plurality of query latencies for queries included in the historical query workload metadata;
• determining a plurality of time window specific threshold percentile latencies that correspond to a predetermined threshold percentile of query latencies of the plurality of query latencies for queries that were processed by the data warehouse system during a corresponding time window in the historical query workload metadata;
• determining a reference threshold percentile latency that is based on a predetermined threshold percentile of the plurality of time window specific threshold percentile latencies;
• and
• determining the first configuration setting by:
• decrementing the time window specific threshold percentile maximum cluster count if the time window specific threshold percentile latency for the first time period is less than the reference threshold percentile latency;
• and
• setting the first configuration setting to the time window specific threshold percentile maximum cluster count if the time window specific threshold percentile latency for the first time period is greater than or equal to the reference threshold percentile latency;
• transmitting, to the data warehouse system, the first configuration setting for the configurable parameter;
• receiving, from the data warehouse system, during the first time period, query workload metadata;
• determining, based on the query workload metadata, whether the query workload metadata meets a threshold performance;
• and
• based on a determination that the query workload metadata does not meet the threshold performance, transmitting a backoff configuration setting for the configurable parameter to the data warehouse system.

A non-transitory computer-readable medium comprising instructions that, responsive to execution by a processing device, causes the processing device to perform operations comprising:

• obtaining historical query workload metadata associated with a data warehouse from a data warehouse system;
• determining, based on the historical query workload metadata, a predictive schedule that includes a first configuration setting associated with a configurable parameter for a first time period, wherein the first time period is a future time period, wherein the first configuration setting is associated with a different computing resource utilization at the data warehouse system when compared to a previous configuration setting, wherein the configurable parameter is an auto-suspend interval, and wherein determining the first configuration setting for the auto-suspend interval for the first time period comprises:
• determining a plurality of time intervals between query arrival times for queries included in the historical query workload metadata;
• and
• setting the first configuration setting to a minimum auto-suspend interval of a plurality of specified auto-suspend intervals whereby a number of affected queries meets an affected queries threshold, wherein an affected query is a query with an associated time interval that exceeds a corresponding auto-suspend interval of the plurality of specified auto-suspend intervals;
• transmitting, to the data warehouse system, the first configuration setting for the configurable parameter;
• receiving, from the data warehouse system, during the first time period, query workload metadata;
• determining, whether the query workload metadata meets a threshold performance;
• and
• based on a determination that the query workload metadata does not meet the threshold performance, transmitting a backoff configuration setting for the configurable parameter to the data warehouse system.

A system comprising:

• a memory with instructions stored thereon;
• and
• a processing device, coupled to the memory, the processing device configured to access the memory and execute the instructions, wherein the instructions cause the processing device to perform operations including:
• obtaining historical query workload metadata associated with a data warehouse from a data warehouse system;
• determining, based on the historical query workload metadata, a predictive schedule that includes a first configuration setting associated with a configurable parameter for a first time period, wherein the first time period is a future time period, wherein the first configuration setting is associated with a different computing resource utilization at the data warehouse system when compared to a previous configuration setting, wherein the configurable parameter is a number of warehouse clusters for the data warehouse, and wherein determining the first configuration setting for the first time period comprises:
• determining a plurality of time window specific threshold percentile maximum cluster counts that correspond to a predetermined threshold percentile of maximum cluster counts for queries that were processed by the data warehouse system during a corresponding time window in the historical query workload metadata;
• determining a plurality of query latencies for queries included in the historical query workload metadata;
• determining a plurality of time window specific threshold percentile latencies that correspond to a predetermined threshold percentile of query latencies of the plurality of query latencies for queries that were processed by the data warehouse system during a corresponding time window in the historical query workload metadata;
• determining a reference threshold percentile latency that is based on a predetermined threshold percentile of the plurality of time window specific threshold percentile latencies;
• and
• determining the first configuration setting by:
• decrementing the time window specific threshold percentile maximum cluster count if the time window specific threshold percentile latency for the first time period is less than the reference threshold percentile latency;
• and
• setting the first configuration setting to the time window specific threshold percentile maximum cluster count if the time window specific threshold percentile latency for the first time period is greater than or equal to the reference threshold percentile latency;
• transmitting, to the data warehouse system, the first configuration setting for the configurable parameter;
• receiving, from the data warehouse system, during the first time period, query workload metadata;
• determining, whether the query workload metadata meets a threshold performance;
• and
• based on a determination that the query workload metadata does not meet the threshold performance, transmitting a backoff configuration setting for the configurable parameter to the data warehouse system.