Query watchdog
AssigneeDatabricks, Inc.
InventorsAlicja Luszczak, Srinath Shankar, Shi Xin
Life of this patent
Estimating the term…
Abstract
A system for monitoring job execution includes an interface and a processor. The interface is configured to receive an indication to start a cluster processing job. The processor is configured to determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion; and in the event that processing the data instance satisfies the watchdog criterion, cause the processing of the data instance to be killed.
Claims20
A system for monitoring job execution, comprising:
- an interface configured to:
- receive an indication to start a cluster processing job;
- and
- a processor configured to:
- determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion;
- and
- in the event that processing the data instance satisfies the watchdog criterion:
- determine whether the cluster processing job should be killed, comprising to:
- determine whether the cluster processing job be killed after a single data instance that satisfies the watchdog criterion;
- and
- in response to a determination that the cluster processing job be killed after the single data instance that satisfies the watchdog criterion, indicate that the cluster processing job should be killed.
A method for monitoring job execution, comprising:
- receiving an indication to start a cluster processing job;
- and
- determining, using a processor, whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion;
- and
- in the event that processing the data instance satisfies the watchdog criterion:
- determining whether the cluster processing job should be killed, comprising:
- determining whether the cluster processing job be killed after a single data instance that satisfies the watchdog criterion;
- and
- in response to a determination that the cluster processing job be killed after the single data instance that satisfies the watchdog criterion, indicating that the cluster processing job should be killed.
A computer program product for monitoring job execution, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
- receiving an indication to start a cluster processing job;
- and
- determining whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion;
- and
- in the event that processing the data instance satisfies the watchdog criterion:
- determining whether the cluster processing job should be killed, comprising:
- determining whether the cluster processing job be killed after a single data instance that satisfies the watchdog criterion;
- and
- in response to a determination that the cluster processing job be killed after the single data instance that satisfies the watchdog criterion, indicating that the cluster processing job should be killed.
Dependencies are inferred from the claim text.