Unifying runtime catalog and metastore for a cloud storage system

A system comprising:

• one or more processors;
• and
• one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for processing queries, the operations comprising:
• receiving a request from a query engine to access data from a cloud storage table residing in a cloud storage system;
• identifying a format of the query engine and a format of the cloud storage table from a cloud storage table inventory;
• determining that the format of the query engine and the format of the cloud storage table are incompatible based on a data storage type associated with the format of the query engine and a data storage type associated with the format of the cloud storage table;
• based on determining that the format of the query engine and the format of the cloud storage table are incompatible, representing table metadata of the cloud storage table in the format of the query engine using bi-directional translation layer comprising a mapping that directly converts, during runtime, the format of the cloud storage table to the format of the query engine;
• and
• providing the table metadata represented in the format of the query engine to the query engine for the query engine to access the data.

A non-transitory computer readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for processing queries, the operations comprising:

• receiving a request from a query engine to access data from a cloud storage table residing in a cloud storage system;
• identifying a format of the query engine and a format of the cloud storage table from a cloud storage table;
• determining that the format of the query engine and the format of the cloud storage table are incompatible based on a data storage type associated with the format of the query engine and a data storage type associated with the format of the cloud storage table;
• based on determining that the format of the query engine and the format of the cloud storage table are incompatible, representing table metadata of the cloud storage table in the format of the query engine using a bi-directional translation layer comprising a mapping that directly converts, during runtime, the format of the cloud storage table to the format of the query engine;
• and
• providing the table metadata represented in the format of the query engine to the query engine for the query engine to access the data.