Query Watchdog
AssigneeDatabricks, Inc.
InventorsAlicja Luszczak, Srinath Shankar, Shi Xin
Life of this patent
Estimating the term…
Abstract
A system for monitoring job execution includes an interface and a processor. The interface is configured to receive an indication to start a cluster processing job. The processor is configured to determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion; and in the event that processing the data instance satisfies the watchdog criterion, cause the processing of the data instance to be killed.
Claims20
A method comprising:
- receiving an indication that processing of a task has been terminated, wherein the task is one of one of more tasks of a cluster processing job;
- incrementing a counter value, the counter value indicative of a total number of tasks associated with the cluster processing job that have been terminated;
- and
- automatically terminating execution of the cluster processing job responsive to the counter value meeting a threshold value.
A system comprising:
- one or more computer processors;
- and
- one or more computer readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to:
- receive an indication that processing of a task has been terminated, wherein the task is one of one of more tasks of a cluster processing job;
- increment a counter value, the counter value indicative of a total number of tasks associated with the cluster processing job that have been terminated;
- and
- automatically terminate execution of the cluster processing job responsive to the counter value meeting a threshold value.
A non-transitory computer readable medium comprising stored instructions encoded thereon that, when executed by one or more computer processors of a computing system, cause the computing system to:
- receive an indication that processing of a task has been terminated, wherein the task is one of one of more tasks of a cluster processing job;
- increment a counter value, the counter value indicative of a total number of tasks associated with the cluster processing job that have been terminated;
- and
- automatically terminate execution of the cluster processing job a responsive to the counter value meeting a threshold value.
Dependencies are inferred from the claim text.