# US 12,216,656 B2 · Scalable query processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,216,656 B2
- Document ID: US12216656B2
- Status: Granted
- Kind: B2
- Application number: 18/477,808
- Filed: 2023-09-29
- Published: 2024-01-25
- Granted: 2025-02-04
- Earliest claimed: 2020-06-01
- Assignees: Snowflake Inc.
- Applicants: Snowflake Inc.
- Inventors: Thierry Cruanes; Igor Demura; Varun Ganesh; Prasanna Rajaperumal; Libo Wang; Jiaqi Yan
- Web page: https://patentgrep.com/patent/US12216656B2

## CPC

- G06F 16/24532: {of parallel queries}
- G06F 16/24542: {Plan optimisation}
- G06F 16/24537: {of operators}
- G06F 16/24539: {using cached or materialised query results}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments of the present disclosure may provide a dynamic query execution model. This query execution model may provide acceleration by scaling out parallel parts of a query (also referred to as a fragment) to additional computing resources, for example computing resources leased from a pool of computing resources. Execution of the parts of the query may be coordinated by a parent query coordinator, where the query originated, and a fragment query coordinator.

## Claims

### Claim 1

A method comprising:
receiving, by one or more processors, a query directed at a data set stored in a network-based database system;
generating a query plan to execute the query;
identifying a portion of the query plan that is eligible for fragment processing based on an output-to-input ratio where input represents size of one or more input files for fragment processing and output represents size of an output file based on the one or more input file;
executing, by a parent query coordinator using one or more computing resources of a first set of computing resources assigned to the parent query coordinator, the identified portion of the query plan on a first batch of files of the data set to generate a first batch result;
transmitting instructions to a fragment query coordinator for the fragment query coordinator to execute the identified portion of the query on a second batch of files of the data set to generate the output file using a second set of computing resources assigned to the fragment query coordinator, the output file being stored in a storage location;
scanning, by the parent query coordinator, the output file from the storage location to generate a second batch result;
combining the first batch result and the second batch result to generate combined results; and
executing remaining portion of the query plan on the combined results to generate a response to the query.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
loading files of the data set into a first shared file queue as a continuous scanset;
grouping a first set of files as the first batch and providing the first batch to the parent query coordinator; and
grouping a second set of files as the second batch and providing the second batch to the fragment query coordinator.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
providing additional batches serially until all files in the continuous scanset have been provided.

### Claim 4 (depends on 1)

The method of claim 1, wherein identifying the portion of the query plan that is eligible for fragment processing is further based on a set of criteria.

### Claim 5 (depends on 4)

The method of claim 4, wherein the set of criteria includes a type of link connecting at least two operators of a plurality of operators in the query plan.

### Claim 6 (depends on 5)

The method of claim 5, wherein each link connects a first and second operator of the plurality of operators and the type of link indicates whether the first operator is executable by a computing resource without communicating with another computing resource.

### Claim 7 (depends on 4)

The method of claim 4, wherein the set of criteria includes whether the identified portion is executable by a computing resource without communicating with another computing resource.

### Claim 8

A system comprising:
one or more processors of a machine; and
a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
receiving a query directed at a data set stored in a network-based database system;
generating a query plan to execute the query;
identifying a portion of the query plan that is eligible for fragment processing based on an output-to-input ratio where input represents size of one or more input files for fragment processing and output represents size of an output file based on the one or more input file;
executing, by a parent query coordinator using one or more computing resources of a first set of computing resources assigned to the parent query coordinator, the identified portion of the query plan on a first batch of files of the data set to generate a first batch result;
transmitting instructions to a fragment query coordinator for the fragment query coordinator to execute the identified portion of the query on a second batch of files of the data set to generate the output file using a second set of computing resources assigned to the fragment query coordinator, the output file being stored in a storage location;
scanning, by the parent query coordinator, the output file from the storage location to generate a second batch result;
combining the first batch result and the second batch result to generate combined results; and
executing remaining portion of the query plan on the combined results to generate a response to the query.

### Claim 9 (depends on 8)

The system of claim 8, the operations further comprising:
loading files of the data set into a first shared file queue as a continuous scanset;
grouping a first set of files as the first batch and providing the first batch to the parent query coordinator; and
grouping a second set of files as the second batch and providing the second batch to the fragment query coordinator.

### Claim 10 (depends on 9)

The system of claim 9, the operations further comprising:
providing additional batches serially until all files in the continuous scanset have been provided.

### Claim 11 (depends on 8)

The system of claim 8, wherein identifying the portion of the query plan that is eligible for fragment processing is further based on a set of criteria.

### Claim 12 (depends on 11)

The system of claim 11, wherein the set of criteria includes a type of link connecting at least two operators of a plurality of operators in the query plan.

### Claim 13 (depends on 12)

The system of claim 12, wherein each link connects a first and second operator of the plurality of operators and the type of link indicates whether the first operator is executable by a computing resource without communicating with another computing resource.

### Claim 14 (depends on 11)

The system of claim 11, wherein the set of criteria includes whether the identified portion is executable by a computing resource without communicating with another computing resource.

### Claim 15

A non-transitory machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
receiving a query directed at a data set stored in a network-based database system;
generating a query plan to execute the query;
identifying a portion of the query plan that is eligible for fragment processing based on an output-to-input ratio where input represents size of one or more input files for fragment processing and output represents size of an output file based on the one or more input file;
executing, by a parent query coordinator using one or more computing resources of a first set of computing resources assigned to the parent query coordinator, the identified portion of the query plan on a first batch of files of the data set to generate a first batch result;
transmitting instructions to a fragment query coordinator for the fragment query coordinator to execute the identified portion of the query on a second batch of files of the data set to generate the output file using a second set of computing resources assigned to the fragment query coordinator, the output file being stored in a storage location;
scanning, by the parent query coordinator, the output file from the storage location to generate a second batch result;
combining the first batch result and the second batch result to generate combined results; and
executing remaining portion of the query plan on the combined results to generate a response to the query.

### Claim 16 (depends on 15)

The non-transitory machine-storage medium of claim 15, further comprising:
loading files of the data set into a first shared file queue as a continuous scanset;
grouping a first set of files as the first batch and providing the first batch to the parent query coordinator; and
grouping a second set of files as the second batch and providing the second batch to the fragment query coordinator.

### Claim 17 (depends on 16)

The non-transitory machine-storage medium of claim 16, further comprising:
providing additional batches serially until all files in the continuous scanset have been provided.

### Claim 18 (depends on 15)

The non-transitory machine-storage medium of claim 15, wherein identifying the portion of the query plan that is eligible for fragment processing is further based on a set of criteria.

### Claim 19 (depends on 18)

The non-transitory machine-storage medium of claim 18, wherein the set of criteria includes a type of link connecting at least two operators of a plurality of operators in the query plan.

### Claim 20 (depends on 19)

The non-transitory machine-storage medium of claim 19, wherein each link connects a first and second operator of the plurality of operators and the type of link indicates whether the first operator is executable by a computing resource without communicating with another computing resource.

### Claim 21 (depends on 18)

The non-transitory machine-storage medium of claim 18, wherein the set of criteria includes whether the identified portion is executable by a computing resource without communicating with another computing resource.