Batch data ingestion in database systems

A database system comprising:

• at least one processor;
• and
• one or more non-transitory computer readable storage media containing instructions executable by the at least one processor for causing the at least one processor to perform operations comprising:
• obtaining, at the database system, an ingest request to ingest one or more files into a table of a database;
• after obtaining the ingest request and prior to the ingesting of the one or more files, persisting the one or more files in a first file queue that corresponds to the table, the first file queue further corresponds to a client account, and the database system further comprising a second file queue that corresponds to both a second client account and a second table;
• assigning the one or more files to one or more execution nodes to be ingested into the table;
• ingesting, by the one or more execution nodes, the one or more files into one or more micro-partitions of the table, each of the one or more micro-partitions comprising contiguous units of storage of a storage device;
• and
• registering metadata after the one or more files are ingested into the one or more micro-partitions of the table, the metadata identifying the one or more files and the one or more micro-partitions.

One or more non-transitory computer readable storage media containing instructions executable by at least one processor for causing the at least one processor to perform operations comprising:

• obtaining, at a database system, an ingest request to ingest one or more files into a table of a database;
• after obtaining the ingest request and prior to the ingesting of the one or more files, persisting the one or more files in a first file queue that corresponds to the table, the first file queue further corresponds to a client account, and the database system further comprising a second file queue that corresponds to both a second client account and a second table;
• assigning the one or more files to one or more execution nodes to be ingested into the table;
• ingesting, by the one or more execution nodes, the one or more files into one or more micro-partitions of the table, each of the one or more micro-partitions comprising contiguous units of storage of a storage device;
• and
• registering metadata after the one or more files are ingested into the one or more micro-partitions of the table, the metadata identifying the one or more files and the one or more micro-partitions.