Back to results
US 11,238,036 B2Granted
System performance logging of complex remote query processor query operations
AssigneeDeephaven Data Labs LLC
InventorsDavid R. Kent, IV, Ryan Caudy, Charles Wright, Radu Teodorescu
Life of this patent
Estimating the term…
Abstract
Described are methods, systems and computer readable media for performance logging of complex query operations.
Claims13
A system for improving performance of a remote query computer system by using dynamic query performance logging to identify and remediate efficiency impediments in the remote query computer system, the system comprising:
- one or more processors;
- computer readable storage coupled to the one or more processors, the computer readable storage having stored thereon instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:
- receiving, at a query server computer, a digital request from a client computer to execute a query task;
- executing the query task;
- generating query task data using an update propagation graph, the update propagation graph being a structure implemented with connected nodes through which data is propagated to determine an output of the query task, each of the connected nodes being a static node or a dynamic node;
- storing the query task data for subsequent retrieval and analysis;
- executing a query update, the query task comprising a recurring query operation, the executing the query update comprising updating one or more dynamic components of the recurring query operation;
- configuring a query update interval;
- generating query update data for the update interval, the generating the query update data including:
- starting a refresh cycle during which one or more changes are propagated through the update propagation graph to determine an updated output of the query task based on the one or more changes;
- listening, for each node of one or more dynamic nodes of the update propagation graph, at a listener associated with the node for change messages from one or more parent nodes of the node;
- after starting the refresh cycle, receiving, at each of one or more listeners associated with nodes of the update propagation graph, one or more change messages at the listener from a parent node of the listener's associated node, updates being propagated node-to-node through the update propagation graph via the change messages,
- determining an occurrence of an update to one or more nodes of the update propagation graph,
- after the receiving the one or more change messages, ending the refresh cycle to indicate completion of propagation of the one or more changes through the update propagation graph and determine the updated output of the query task, and
- creating at least a portion of the query update data for the occurrence of the update to one or more nodes of the update propagation graph;
- determining that the query update may not complete within the query update interval;
- extending the query update interval to reach the end of the refresh cycle and allow full execution of the query update during the query update interval;
- storing the query update data for subsequent retrieval and analysis;
- retrieving the query task data and the query update data;
- using a performance-enhancing processor to analyze the retrieved query task data and the retrieved query update data to obtain an analysis result;
- with the performance-enhancing processor, identifying based on the analysis result an efficiency impediment in the query server computer;
- and
- responsive to the identifying, making changes to stored instructions and thereby alleviating the identified efficiency impediment and improving efficiency of the query server computer,
- wherein the efficiency impediment is one or more of:
- a dataflow bottleneck, excessive use of processor resources, or excessive use of RAM.
22 more elements
A method comprising:
- receiving, at a query server computer, a digital request from a client computer to execute a query task;
- executing the query task;
- generating query task data using an update propagation graph, the update propagation graph being a structure implemented with connected nodes through which data is propagated to determine an output of the query task, each of the connected nodes being a static node or a dynamic node;
- storing the query task data for subsequent retrieval and analysis;
- executing a query update, the query task comprising a recurring query operation, the executing the query update comprising updating one or more dynamic components of the recurring query operation;
- configuring a query update interval;
- generating query task update data for the update interval, the generating the query task update data including:
- starting a refresh cycle during which one or more changes are propagated through the update propagation graph to determine an updated output of the query task based on the one or more changes,
- listening, for each node of one or more dynamic nodes of the update propagation graph, at a listener associated with the node for change messages from one or more parent nodes of the node,
- after starting the refresh cycle, receiving, at each of one or more listeners associated with nodes of the update propagation graph, one or more change messages at the listener from a parent node of the listener's associated node, updates being propagated node-to-node through the update propagation graph via the change messages,
- determining an occurrence of an update to one or more nodes of the update propagation graph,
- after the receiving the one or more change messages, ending the refresh cycle to indicate completion of propagation of the one or more changes through the update propagation graph and determine the updated output of the query task, and
- creating at least a portion of the query task update data for the occurrence of the update to one or more nodes of the update propagation graph;
- determining that the query update may not complete within the query update interval;
- extending the query update interval to reach the end of the refresh cycle and allow full execution of the query update during the query update interval;
- storing the query task update data for subsequent retrieval and analysis;
- retrieving the query task data and the query task update data;
- analyzing the retrieved query task data and the retrieved query task update data to obtain an analysis result;
- identifying based on the analysis result an efficiency impediment in the query server computer;
- and
- making changes to stored instructions and thereby alleviating the identified efficiency impediment and improving efficiency of the query server computer,
- wherein the efficiency impediment is one or more of:
- a dataflow bottleneck, excessive use of processor resources, or excessive use of RAM.
20 more elements
A nontransitory computer readable medium having stored thereon software instructions that, when executed by one or more processors, cause the one or more processors to perform operations including:
- receiving, at a query server computer, a digital request from a client computer to execute a query task;
- executing the query task;
- generating query task data using an update propagation graph, the update propagation graph being a structure implemented with connected nodes through which data is propagated to determine an output of the query task, each of the connected nodes being a static node or a dynamic node;
- storing the query task data for subsequent retrieval and analysis;
- executing a query update, the query task comprising a recurring query operation, the executing the query update comprising updating one or more dynamic components of the recurring query operation;
- configuring a query update interval;
- generating query task update data for the update interval, the generating the query task update data including:
- starting a refresh cycle during which one or more changes are propagated through the update propagation graph to determine an updated output of the query task based on the one or more changes,
- listening, for each node of one or more dynamic nodes of the update propagation graph, at a listener associated with the node for change messages from one or more parent nodes of the node,
- after starting the refresh cycle, receiving, at each of one or more listeners associated with nodes of the update propagation graph, one or more change messages at the listener from a parent node of the listener's associated node, updates being propagated node-to-node through the update propagation graph via the change messages,
- determining an occurrence of an update to one or more nodes of the update propagation graph,
- after the receiving the one or more change messages, ending the refresh cycle to indicate completion of propagation of the one or more changes through the update propagation graph and determine the updated output of the query task, and
- creating at least a portion of the query task update data for the occurrence of the update to one or more nodes of the update propagation graph;
- determining that the query update may not complete within the query update interval;
- extending the query update interval to reach the end of the refresh cycle and allow full execution of the query update during the query update interval;
- storing the query task update data for subsequent retrieval and analysis;
- retrieving the query task data and the query task update data;
- analyzing the retrieved query task data and the retrieved query task update data to obtain an analysis result;
- identifying based on the analysis result an efficiency impediment in the query server computer;
- and
- making changes to stored instructions and thereby alleviating the identified efficiency impediment and improving efficiency of the query server computer,
- wherein the efficiency impediment is one or more of:
- a dataflow bottleneck, excessive use of processor resources, or excessive use of RAM.
20 more elements
A system for improving performance of a remote query computer system by using dynamic query performance logging to identify and remediate efficiency impediments in the remote query computer system, the system comprising:
- one or more processors;
- computer readable storage coupled to the one or more processors, the computer readable storage having stored thereon instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:
- receiving, at a query server computer, a digital request from a client computer to execute a query task;
- executing the query task;
- generating query task data using an update propagation graph, the update propagation graph being a structure implemented with connected nodes through which data is propagated to determine an output of the query task, each of the connected nodes being a static node or a dynamic node;
- storing the query task data for subsequent retrieval and analysis;
- executing a query update, the query task comprising a recurring query operation, the executing the query update comprising updating one or more dynamic components of the recurring query operation;
- configuring a query update interval;
- generating query update data for the update interval, the generating the query update data including:
- starting a refresh cycle during which one or more changes are propagated through the update propagation graph to determine an updated output of the query task based on the one or more changes,
- listening, for each node of one or more dynamic nodes of the update propagation graph, at a listener associated with the node for change messages from one or more parent nodes of the node,
- after starting the refresh cycle, receiving, at each of one or more listeners associated with nodes of the update propagation graph, one or more change messages at the listener from a parent node of the listener's associated node, updates being propagated node-to-node through the update propagation graph via the change messages,
- determining an occurrence of an update to one or more nodes of the update propagation graph,
- after the receiving the one or more change messages, ending the refresh cycle to indicate completion of propagation of the one or more changes through the update propagation graph and determine the updated output of the query task, and
- creating at least a portion of the query update data for the occurrence of the update to one or more nodes of the update propagation graph;
- determining that the query update may not complete within the query update interval;
- extending the query update interval to reach the end of the refresh cycle and allow full execution of the query update during the query update interval;
- storing the query update data for subsequent retrieval and analysis;
- retrieving the query task data and the query update data;
- analyzing the retrieved query task data and the retrieved query update data to obtain an analysis result;
- identifying based on the analysis result an efficiency impediment in the query server computer;
- and
- making changes to stored software instructions and thereby alleviating the identified efficiency impediment and improving efficiency of the query server computer,
- wherein the efficiency impediment is one or more of:
- a dataflow bottleneck, excessive use of processor resources, or excessive use of RAM.
22 more elements
Dependencies are inferred from the claim text.