US 2022/0092067 A1Published application

Data-driven checkpoint selector

AssigneeMicrosoft Technology Licensing, LLC

InventorsYiwen Zhu, Alekh Jindal, Malay Kumar Bag, Hiren Shantilal Patel

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques are described herein that are capable of selecting checkpoints of a database job. For instance, at compile time, temporal indicators associated with the query plans of the database job are determined. Each temporal indicator indicates first and second subsets of stages of the respective query plan. Values of attributes of each stage in at least each first subset are predicted using a machine learning technique. At the compile time, candidate stage(s) for each query plan are identified based on the respective candidate stage being a child of stage(s) in the corresponding second subset or not being a child of another stage in the respective query plan. The candidate stage(s) for each query plan are selectively chosen as respective checkpoint(s) based on whether the values of the attributes of each stage in at least the first subset of the stages of the respective query plan satisfy one or more criteria.

Claims20

Dependent claims

A system to select checkpoints of a recurrent database job, the system comprising:

memory;
and
one or more processors coupled to the memory, the one or more processors configured to:
determine, at compile time of database queries that define the recurrent database job, temporal indicators associated with query plans that result from compilation of the respective database queries such that one or more temporal indicators are associated with each query plan, each of the one or more temporal indicators associated with each query plan indicating a first subset of a plurality of stages of the respective query plan that are to have executed by a respective reference time indicated by the respective temporal indicator and a second subset of the plurality of stages of the respective query plan that are not to have executed by the respective reference time, each stage including a plurality of threads;
predict a plurality of values of a plurality of respective attributes of each stage in at least the first subset of the plurality of stages of each query plan indicated by a first temporal indicator associated with the respective query plan by analyzing a plurality of historical values of the plurality of attributes of the respective stage for each of a plurality of historical executions of the respective stage using a machine learning technique;
identify, at the compile time of the database queries, one or more candidate stages for each query plan from the first subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan based on each candidate stage being a child of one or more stages in the second subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan or not being a child of another stage in the respective query plan;
and
selectively choose the one or more candidate stages for each query plan that correspond to the first temporal indicator associated with the respective query plan to be one or more respective checkpoints of the recurrent database job based on whether the plurality of values of the plurality of respective attributes of each stage in at least the first subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan satisfy one or more criteria.

5 more elements

A method of selecting checkpoints of a recurrent database job, the method comprising:

determining, at compile time of database queries that define the recurrent database job, temporal indicators associated with query plans that result from compilation of the respective database queries such that one or more temporal indicators are associated with each query plan, each of the one or more temporal indicators associated with each query plan indicating a first subset of a plurality of stages of the respective query plan that are to have executed by a respective reference time indicated by the respective temporal indicator and a second subset of the plurality of stages of the respective query plan that are not to have executed by the respective reference time, each stage including a plurality of threads;
predicting a plurality of values of a plurality of respective attributes of each stage in at least the first subset of the plurality of stages of each query plan indicated by a first temporal indicator associated with the respective query plan by analyzing a plurality of historical values of the plurality of attributes of the respective stage for each of a plurality of historical executions of the respective stage using a machine learning technique;
identifying, at the compile time of the database queries, one or more candidate stages for each query plan from the first subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan based on each candidate stage being a child of one or more stages in the second subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan or not being a child of another stage in the respective query plan;
and
selectively choosing the one or more candidate stages for each query plan that correspond to the first temporal indicator associated with the respective query plan to be one or more respective checkpoints of the recurrent database job based on whether the plurality of values of the plurality of respective attributes of each stage in at least the first subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan satisfy one or more criteria.

2 more elements

A computer program product comprising a computer-readable storage medium having instructions recorded thereon for enabling a processor-based system to select checkpoints of a recurrent database job by performing operations, the operations comprising:

determine, at compile time of database queries that define the recurrent database job, temporal indicators associated with query plans that result from compilation of the respective database queries such that one or more temporal indicators are associated with each query plan, each of the one or more temporal indicators associated with each query plan indicating a first subset of a plurality of stages of the respective query plan that are to have executed by a respective reference time indicated by the respective temporal indicator and a second subset of the plurality of stages of the respective query plan that are not to have executed by the respective reference time, each stage including a plurality of threads;
predict a plurality of values of a plurality of respective attributes of each stage in at least the first subset of the plurality of stages of each query plan indicated by a first temporal indicator associated with the respective query plan by analyzing a plurality of historical values of the plurality of attributes of the respective stage for each of a plurality of historical executions of the respective stage using a machine learning technique;
identify, at the compile time of the database queries, one or more candidate stages for each query plan from the first subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan based on each candidate stage being a child of one or more stages in the second subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan or not being a child of another stage in the respective query plan;
and
selectively choose the one or more candidate stages for each query plan that correspond to the first temporal indicator associated with the respective query plan to be one or more respective checkpoints of the recurrent database job based on whether the plurality of values of the plurality of respective attributes of each stage in at least the first subset of the plurality of stages of the respective query plan indicated by the first temporal indicator associated with the respective query plan satisfy one or more criteria.

2 more elements

Dependencies are inferred from the claim text.