# US 12,007,961 B2 · Schema evolution

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,961 B2
- Document ID: US12007961B2
- Status: Granted
- Kind: B2
- Application number: 18/345,987
- Filed: 2023-06-30
- Published: 2023-12-14
- Granted: 2024-06-11
- Earliest claimed: 2022-06-08
- Assignees: Snowflake Inc.
- Applicants: Snowflake Inc.
- Inventors: Istvan Cseri; Benoit Dageville; Ganeshan Ramachandran Iyer; Yucan Liu; Jiaqi Yan
- Web page: https://patentgrep.com/patent/US12007961B2

## CPC

- G06F 16/211: {Schema design and management}
- G06F 16/258: {Data format conversion from or to a database}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques for schema mismatch detection and evolution are described. When data is being uploaded into a source table, schema of the data to be uploaded can be compared with the schema for the source table. If a schema mismatch is detected, the schema of the source table can be modified, and the upload can be continued without data loss.

## Claims

### Claim 1

A method comprising:
receiving, by at least one hardware processor, a notification of a schema mismatch between a file to be ingested and a source table;
modifying a schema for the source table based on the notification;
quarantining at least one ingest task associated with the file to be ingested based on detecting the schema mismatch;
compiling a modified query plan for the at least one quarantined ingest task based on the modified schema of the source table;
executing the at least one quarantined ingest task based on the modified query plan; and
ingesting the file into the source table using the modified schema.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a notification regarding the file to be ingested into the source table;
generating, a query plan for ingesting the file; and
assigning the at least one ingest task based on the query plan to one or more execution platforms.

### Claim 3 (depends on 1)

The method of claim 1, wherein the schema mismatch is detected based on the file including one or more columns not in the source table.

### Claim 4 (depends on 1)

The method of claim 1, wherein the schema mismatch is detected based on the file including null values in a designated non-nullable column in the source table.

### Claim 5 (depends on 1)

The method of claim 1, wherein the notification includes information regarding a type of schema mismatch.

### Claim 6 (depends on 1)

The method of claim 1, wherein modifying the schema for the source table includes constructing a data definition language statement to add or alter a column based on information in the notification.

### Claim 7 (depends on 1)

The method of claim 1, wherein detecting the schema mismatch includes scanning schema information saved in metadata for the file.

### Claim 8 (depends on 1)

The method of claim 1, wherein detecting the schema mismatch includes:
scanning a set of rows from the file, and
determining a schema for the file based on the scanned set of rows.

### Claim 9

A machine-storage medium embodying instructions that, when executed by one or more machines, cause the one or more machines to perform operations comprising:
receiving a notification of a schema mismatch between a file to be ingested and a source table;
modifying a schema for the source table based on the notification;
quarantining at least one ingest task associated with the file to be ingested based on detecting the schema mismatch;
compiling a modified query plan for the at least one quarantined ingest task based on the modified schema of the source table;
executing the at least one quarantined ingest task based on the modified query plan; and
ingesting the file into the source table using the modified schema.

### Claim 10 (depends on 9)

The machine-storage medium of claim 9, further comprising:
receiving a notification regarding the file to be ingested into the source table;
generating, a query plan for ingesting the file; and
assigning the at least one ingest task based on the query plan to one or more execution platforms.

### Claim 11 (depends on 9)

The machine-storage medium of claim 9, wherein the schema mismatch is detected based on the file including one or more columns not in the source table.

### Claim 12 (depends on 9)

The machine-storage medium of claim 9, wherein the schema mismatch is detected based on the file including null values in a designated non-nullable column in the source table.

### Claim 13 (depends on 9)

The machine-storage medium of claim 9, wherein the notification includes information regarding a type of schema mismatch.

### Claim 14 (depends on 9)

The machine-storage medium of claim 9, wherein modifying the schema for the source table includes constructing a data definition language statement to add or alter a column based on information in the notification.

### Claim 15 (depends on 9)

The machine-storage medium of claim 9, wherein detecting the schema mismatch includes scanning schema information saved in metadata for the file.

### Claim 16 (depends on 9)

The machine-storage medium of claim 9, wherein detecting the schema mismatch includes:
scanning a set of rows from the file, and
determining a schema for the file based on the scanned set of rows.

### Claim 17

A system comprising:
at least one hardware processor; and
at least one memory storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
receiving a notification of a schema mismatch between a file to be ingested and a source table;
modifying a schema for the source table based on the notification;
quarantining at least one ingest task associated with the file to be ingested based on detecting the schema mismatch;
compiling a modified query plan for the at least one quarantined ingest task based on the modified schema of the source table;
executing the at least one quarantined ingest task based on the modified query plan; and
ingesting the file into the source table using the modified schema.

### Claim 18 (depends on 17)

The system of claim 17, the operations further comprising:
receiving a notification regarding the file to be ingested into the source table;
generating, a query plan for ingesting the file; and
assigning the at least one ingest task based on the query plan to one or more execution platforms.

### Claim 19 (depends on 17)

The system of claim 17, wherein the schema mismatch is detected based on the file including one or more columns not in the source table.

### Claim 20 (depends on 17)

The system of claim 17, wherein the schema mismatch is detected based on the file including null values in a designated non-nullable column in the source table.

### Claim 21 (depends on 17)

The system of claim 17, wherein the notification includes information regarding a type of schema mismatch.

### Claim 22 (depends on 17)

The system of claim 17, wherein modifying the schema for the source table includes constructing a data definition language statement to add or alter a column based on information in the notification.

### Claim 23 (depends on 17)

The system of claim 17, wherein detecting the schema mismatch includes scanning schema information saved in metadata for the file.

### Claim 24 (depends on 17)

The system of claim 17, wherein detecting the schema mismatch includes:
scanning a set of rows from the file, and
determining a schema for the file based on the scanned set of rows.