Metadata on delta sharing tables
AssigneeSap SE
InventorsJohannes Alberti, Rubens Luiz Rech, JR.
Life of this patent
Estimating the term…
Abstract
In an example embodiment, additional metadata is attached to a delta tables, via delta shares, in the form of a common schema notation (CSN) entity. When a request is received from a client for a share of a range of delta tables where the range comprises the delta table to which the additional metadata is attached, the most recent CSN entity for each of the delta tables in the range are aggregated and returned to the client.
Claims20
A system comprising:
- at least one hardware processor;
- and
- a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the system to perform operations comprising:
- storing a first common schema notation file for a delta table in a data lake, the data lake being a storage containing structured, unstructured, and semi-structured data, the delta table describing differences among multiple versions of a data lake file stored in the data lake, the first common schema notation file being associated with a first starting delta table version;
- storing a second common schema notation file for the delta table, the second common schema notation file being associated with a second starting delta table version that is higher than the first starting delta table version;
- receiving, from a client, a request identifying the delta table and a requested delta table version;
- selecting, based on the requested delta table version, one of the first common schema notation file and the second common schema notation file as a selected common schema notation file, wherein the selecting comprises identifying a common schema notation file, among the first and second common schema notation file, having a highest associated starting delta table version that is less than or equal to the requested delta table version;
- and
- providing the selected common schema notation file to the client.
A method comprising:
- storing a first common schema notation file for a delta table in a data lake, the data lake being a storage containing structured, unstructured, and semi-structured data, the delta table describing differences among multiple versions of a data lake file stored in the data lake, the first common schema notation file being associated with a first starting delta table version;
- storing a second common schema notation file for the delta table, the second common schema notation file being associated with a second starting delta table version that is higher than the first starting delta table version;
- receiving, from a client, a request identifying the delta table and a requested delta table version;
- selecting, based on the requested delta table version, one of the first common schema notation file and the second common schema notation file as a selected common schema notation file, wherein the selecting comprises identifying a common schema notation file, among the first and second common schema notation file, having a highest associated starting delta table version that is less than or equal to the requested delta table version;
- and
- providing the selected common schema notation file to the client.
A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- storing a first common schema notation file for a delta table in a data lake, the data lake being a storage containing structured, unstructured, and semi-structured data, the delta table describing differences among multiple versions of a data lake file stored in the data lake, the first common schema notation file being associated with a first starting delta table version;
- storing a second common schema notation file for the delta table, the second common schema notation file being associated with a second starting delta table version that is higher than the first starting delta table version;
- receiving, from a client, a request identifying the delta table and a requested delta table version;
- selecting, based on the requested delta table version, one of the first common schema notation file and the second common schema notation file as a selected common schema notation file, wherein the selecting comprises identifying a common schema notation file, among the first and second common schema notation file, having a highest associated starting delta table version that is less than or equal to the requested delta table version;
- and
- providing the selected common schema notation file to the client.
Dependencies are inferred from the claim text.