# US 12,222,946 B2 · Data system configured to transparently cache data of data sources and access the cached data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,222,946 B2 - Document ID: US12222946B2 - Status: Granted - Kind: B2 - Application number: 18/146,948 - Filed: 2022-12-27 - Published: 2023-06-15 - Granted: 2025-02-11 - Earliest claimed: 2019-04-30 - Assignees: Dremio Corporation - Applicants: Dremio Corporation - Inventors: Jacques Nadeau; Tomer Shiran; Arvind Arun Pande; Thomas W. Fry - Web page: https://patentgrep.com/patent/US12222946B2 ## CPC - G06F 16/24552: {Database cache management} - G06F 16/137: {Hash-based (content-based indexing of textual data G06F16/31)} - G06F 16/182: Distributed file systems - G06F 16/248: Presentation of query results - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The disclosed embodiments include a method for caching by a data system. The method includes automatically caching a portion of a data object from an external data source to a local cluster of nodes in accordance with a unit of caching. The portion of the data object can be selected for caching based on a frequency of accessing the portion of the data object. The portion of the data object in the cache is mapped to the external data source in accordance with a unit of hashing. The method further includes, responsive to the data system receiving a query for data stored in the external data source, obtaining query results that satisfy the received query by reading the portion of the cached data object instead of reading the data object from the external data source. ## Claims ### Claim 1 A method for query planning, the method comprising: generating a query plan in response to receiving a query for a data object including multiple logical blocks, wherein generating the query plan includes creating multiple phases that are configured to read the multiple logical blocks of the data object, and wherein each of the multiple phases includes query fragments; calculating one or more possible nodes to map each query fragment in a phase of the multiple phases which includes configuring the query fragment to read at least one logical block from an external data source; for each query fragment, utilizing a hash function to return a set of possible nodes to map to the query fragment; analyzing a load on each node in the set of possible nodes to map to the query fragments, wherein each node includes a local database in which each node caches the logical blocks as cached logical blocks; selecting a particular node to map to each query fragment based on the analysis of the load on each node in the set of possible nodes, wherein the selecting is based on determining whether the cached logical blocks are stored in the local database, and wherein the selecting includes configuring the particular node to read the cached logical block instead of reading the logical block; sending the query fragments to the selected nodes for processing; and returning, in response to another query for the data object, a query result that includes the cached logical blocks obtained from the selected nodes. ### Claim 2 (depends on 1) The method of claim 1 further comprising: determining a configured ring associated with a phase; wherein the configured ring defines a configuration in which the data object is mapped to the one or more possible nodes; determining that the configured ring is a single configured ring; and in response to determining the single configured ring: for each query fragment, returning a single target node; mapping each query fragment to the target node; and sending each query fragment to the target node for processing. ### Claim 3 (depends on 1) The method of claim 1, wherein selecting the node to map to each query fragment further comprises: performing load balancing of the set of possible nodes. ### Claim 4 (depends on 1) The method of claim 1, wherein a plurality of load balancing strategies are used in selecting the node to map to each query fragment. ### Claim 5 (depends on 1) The method of claim 1 further comprising: enabling multiple configured rings for the data object so that multiple copies of the data object are cached locally; wherein each of the multiple configured rings defines a configuration in which the data object is mapped to the one or more possible nodes; and mapping requests for the data object to the one or more possible nodes. ### Claim 6 (depends on 5) The method of claim 5, wherein the multiple configured rings create an evenly distributed mapping of the data object to the possible nodes. ### Claim 7 (depends on 1) The method of claim 1, wherein mapping the selected node to each query fragment is calculated based on inputs to a hash algorithms, the inputs being related to the data object. ### Claim 8 (depends on 7) The method of claim 7, wherein the inputs to the hash algorithms include a cluster configuration, file type, file path, file name, file split, or partition details. ### Claim 9 A non-transitory computer-readable storage medium storing instructions, which, when executed by one or more data processors of a system, cause the system to: generate a query plan in response to receiving a query for a data object including multiple logical blocks, wherein generating the query plan includes creating multiple phases that are configured to read the multiple logical blocks query fragments of the data object; calculate one or more possible nodes to map each query fragment in a phase of the multiple phases which includes configuring the query fragment to read at least one logical block from an external data source; for each query fragment, utilize a hash function to return a set of one or more possible nodes to map to the query fragment; analyze a load on each node in the set of one or more possible nodes to map to the query fragments, wherein each node includes a local database in which each node caches the logical blocks as cached logical blocks; select a node to map to each query fragment based on the analysis of the load on each node in the set of one or more possible nodes, wherein the selecting is based on determining whether the cached logical blocks are stored in the local database, and wherein the selecting includes configuring the particular nodes to read the cached logical block instead of reading the logical block; send the query fragments to the selected nodes for processing the query; and return a query result that includes the data object obtained from the external data source. ### Claim 10 (depends on 9) The non-transitory computer-readable storage medium of claim 9 including further instructions which, when executed by the one or more processors of the system, cause the system to: determine a configured ring associated with a phase; wherein the configured ring defines a configuration in which the data object is mapped to the one or more possible nodes; determine that the configured ring is a single configured ring; and in response to determining the single configured ring: for each query fragment, return a single target node; map each query fragment to the target node; and send each query fragment to the target node for processing. ### Claim 11 (depends on 9) The non-transitory computer-readable storage medium of claim 9, wherein selecting the node to map to each query fragment further comprises: performing load balancing of the set of one or more possible nodes. ### Claim 12 (depends on 9) The non-transitory computer-readable storage medium of claim 9, wherein a plurality of load balancing strategies are used in selecting the node to map to each query fragment. ### Claim 13 (depends on 9) The non-transitory computer-readable storage medium of claim 9, including further instructions which, when executed by the one or more processors of the system, cause the system to: enable multiple configured rings for the data object so that multiple copies of the data object are cached locally; wherein each of the multiple configured rings defines a configuration in which the data object is mapped to the one or more possible nodes; and map requests for the data object to the one or more possible nodes. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the multiple configured rings create an evenly distributed mapping of the data object to the one or more possible nodes. ### Claim 15 (depends on 9) The non-transitory computer-readable storage medium of claim 9, wherein mapping the selected node to each query fragment is calculated based on inputs to a hash algorithms, the inputs being related to the data object. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the inputs to the hash algorithms include one of a cluster configuration, file type, file path, file name, file split, or partition details. ### Claim 17 A data system for query planning comprising: at least one hardware processor; and at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the data system to: generate a query plan in response to receiving a query for a data object including multiple logical blocks, wherein generating the query plan includes creating multiple phases that are configured to read the multiple logical blocks query fragments of the data object; calculate one or more possible nodes to map each query fragment in a phase of the multiple phases which includes configuring the query fragment to read at least one logical block from an external data source; for each query fragment, utilizing a hash function to return a set of one or more possible nodes to map to the query fragment; analyze a load on each node in the set of one or more possible nodes to map to the query fragments, wherein each node includes a local database in which each node caches the logical blocks as cached logical blocks; select a node to map to each query fragment based on the analysis of the load on each node in the set of one or more possible nodes, wherein the selecting is based on determining whether the cached logical blocks are stored in the local database, and wherein the selecting includes configuring the particular node to read the cached logical block instead of reading the logical block; send the query fragments to the selected nodes for processing the query; and return a query result that includes the cached logical blocks obtained from the external data source. ### Claim 18 (depends on 17) The data system of claim 17, where the data system is further instructed to: determine a configured ring associated with a phase; wherein the configured ring defines a configuration in which the data object is mapped to the one or more possible nodes; determine that the configured ring is a single configured ring; and in response to determining the single configured ring: for each query fragment, return a single target node; map each query fragment to the target node; and send each query fragment to the target node for processing. ### Claim 19 (depends on 17) The data system of claim 17, wherein selecting the node to map to each query fragment further comprises: performing load balancing of the set of one or more possible nodes. ### Claim 20 (depends on 17) The data system of claim 17, wherein mapping the selected node to each query fragment is calculated based on inputs to a hash algorithms, the inputs being related to the data object.