# US 12,222,960 B2 · Scalable compaction for a distributed database

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,222,960 B2
- Document ID: US12222960B2
- Status: Granted
- Kind: B2
- Application number: 18/326,683
- Filed: 2023-05-31
- Published: 2023-10-12
- Granted: 2025-02-11
- Earliest claimed: 2021-08-13
- Assignees: Snowflake Inc.
- Applicants: Snowflake Inc.
- Inventors: Chen Luo; Alexander Miller; William Waddington; Zhaohui Zhang
- Web page: https://patentgrep.com/patent/US12222960B2

## CPC

- G06F 16/24568: {Data stream processing; Continuous queries}
- G06F 16/273: {Asynchronous replication or reconciliation}
- G06F 16/2343: {Locking methods, e.g. distributed locking or locking implementation details}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The subject technology performs, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format. The subject technology processes, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data. The subject technology stores the results data.

## Claims

### Claim 1

A method comprising:
performing, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format;
processing, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data; and
storing the results data.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a plurality of additional transactional queries against the distributed database;
processing, by the transactional threads, the plurality of additional transactional queries using the key-value pairs in the commit format, the transactional threads using commit times in the key-value pairs to generate additional results data; and storing the additional results data.

### Claim 3 (depends on 1)

The method of claim 1, wherein the distributed database comprises a transaction status table that indicates whether data of transactions has committed to the distributed database.

### Claim 4 (depends on 3)

The method of claim 3, wherein queries for data having key-value pairs in the initial format are executed by identifying transaction identifiers in the key-value pairs and access the transaction status table to determine commit statuses.

### Claim 5 (depends on 3)

The method of claim 3, further comprising: performing, using a dedicated compactor thread in one of the execution nodes, transformation and compaction of the key-value pairs from the initial format to the commit format.

### Claim 6 (depends on 5)

The method of claim 5, further comprising: performing, using the dedicated compactor thread, compaction of the transaction status table by deleting key-value pairs in the transaction status table that have corresponding key-value pairs in the commit format.

### Claim 7 (depends on 1)

The method of claim 1, wherein each key-value pair in the initial format includes a transaction identifier value.

### Claim 8 (depends on 7)

The method of claim 7, wherein the asynchronous threads perform transformation by rewriting each key-value pair such that the commit time replaces the transaction identifier value.

### Claim 9 (depends on 1)

The method of claim 1, wherein the key-value pairs having transaction identifiers are deleted by the asynchronous threads.

### Claim 10

A system comprising:
one or more processors of a machine; and
a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
performing, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format;
processing, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data; and
storing the results data.

### Claim 11 (depends on 10)

The system of claim 10, wherein the operations further comprise:
receiving a plurality of additional transactional queries against the distributed database;
processing, by the transactional threads, the plurality of additional transactional queries using the key-value pairs in the commit format, the transactional threads using commit times in the key-value pairs to generate additional results data; and storing the additional results data.

### Claim 12 (depends on 10)

The system of claim 10, wherein the distributed database comprises a transaction status table that indicates whether data of transactions has committed to the distributed database.

### Claim 13 (depends on 12)

The system of claim 12, wherein queries for data having key-value pairs in the initial format are executed by identifying transaction identifiers in the key-value pairs and access the transaction status table to determine commit statuses.

### Claim 14 (depends on 12)

The system of claim 12, wherein the operations further comprise:
performing, using a dedicated compactor thread in one of the execution nodes, transformation and compaction of the key-value pairs from the initial format to the commit format.

### Claim 15 (depends on 14)

The system of claim 14, wherein the operations further comprise:
performing, using the dedicated compactor thread, compaction of the transaction status table by deleting key-value pairs in the transaction status table that have corresponding key-value pairs in the commit format.

### Claim 16 (depends on 10)

The system of claim 10, wherein each key-value pair in the initial format includes a transaction identifier value.

### Claim 17 (depends on 16)

The system of claim 16, wherein the asynchronous threads perform transformation by rewriting each key-value pair such that the commit time replaces the transaction identifier value.

### Claim 18 (depends on 10)

The system of claim 10, wherein the key-value pairs having transaction identifiers are deleted by the asynchronous threads.

### Claim 19

A non-transitory machine storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:
performing, using asynchronous threads of execution nodes of a distributed database, asynchronous transformation and compaction of key-value pairs of data that correspond to a plurality of transactional queries, the asynchronous threads transforming the key-value pairs from an initial format to a commit format that includes a commit time of data committed to the distributed database, the asynchronous threads compacting the key-value pairs by deleting the key-value pairs in the initial format;
processing, using transactional threads in the execution nodes, the plurality of transactional queries to generate results data, the asynchronous threads performing the asynchronous transformation and compaction while the transactional threads generate the results data; and
storing the results data.

### Claim 20 (depends on 19)

The non-transitory machine storage medium of claim 19, wherein the operations further comprise:
receiving a plurality of additional transactional queries against the distributed database;
processing, by the transactional threads, the plurality of additional transactional queries using the key-value pairs in the commit format, the transactional threads using commit times in the key-value pairs to generate additional results data; and
storing the additional results data.

### Claim 21 (depends on 19)

The non-transitory machine storage medium of claim 19, wherein the distributed database comprises a transaction status table that indicates whether data of transactions has committed to the distributed database.

### Claim 22 (depends on 21)

The non-transitory machine storage medium of claim 21, wherein queries for data having key-value pairs in the initial format are executed by identifying transaction identifiers in the key-value pairs and access the transaction status table to determine commit statuses.

### Claim 23 (depends on 21)

The non-transitory machine storage medium of claim 21, wherein the operations further comprise: performing, using a dedicated compactor thread in one of the execution nodes, transformation and compaction of the key-value pairs from the initial format to the commit format.

### Claim 24 (depends on 23)

The non-transitory machine storage medium of claim 23, wherein the operations further comprise: performing, using the dedicated compactor thread, compaction of the transaction status table by deleting key-value pairs in the transaction status table that have corresponding key-value pairs in the commit format.

### Claim 25 (depends on 19)

The non-transitory machine storage medium of claim 19, wherein each key-value pair in the initial format includes a transaction identifier value.

### Claim 26 (depends on 25)

The non-transitory machine storage medium of claim 25, wherein the asynchronous threads perform transformation by rewriting each key-value pair such that the commit time replaces the transaction identifier value.

### Claim 27 (depends on 19)

The non-transitory machine storage medium of claim 19, wherein the key-value pairs having transaction identifiers are deleted by the asynchronous threads.