Metadata on delta sharing tables
AssigneeSap SE
InventorsJohannes Alberti, Rubens Luiz Rech, JR.
Life of this patent
Estimating the term…
Abstract
In an example embodiment, additional metadata is attached to a delta tables, via delta shares, in the form of a common schema notation (CSN) entity. When a request is received from a client for a share of a range of delta tables where the range comprises the delta table to which the additional metadata is attached, the most recent CSN entity for each of the delta tables in the range are aggregated and returned to the client.
Claims20
A system comprising:
- at least one hardware processor;
- and
- a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
- generating a first piece of metadata for a delta table in a data lake, the data lake being a storage containing structured, unstructured, and semi-structured data, the delta table describing differences among multiple versions of a data like file stored in the data lake, the first piece of metadata being of a first metadata type that is not supported by a format of the delta table;
- storing the first piece of metadata in a common schema notation file, the common schema notation file having a format that is used for defining and documenting structure of data models;
- binding the common schema notation file to the delta table, wherein the binding comprises attaching the common schema notation file to the delta table;
- receiving a request from a client for a share of a range of delta tables, the range comprising the delta table;
- aggregating a most recent common schema notation file for each delta table in the range into an aggregated common schema notation file separate and distinct from a delta table;
- and
- sharing the aggregated common schema notation file with the client.
A method comprising:
- generating a first piece of metadata for a delta table in a data lake, the data lake being a storage containing structured, unstructured, and semi-structured data, the delta table describing differences among multiple versions of a data like file stored in the data lake, the first piece of metadata being of a first metadata type that is not supported by a format of the delta table;
- storing the first piece of metadata in a common schema notation file, the common schema notation file having a format that is used for defining and documenting structure of data models;
- binding the common schema notation file to the delta table, wherein the binding comprises attaching the common schema notation file to the delta table;
- receiving a request from a client for a share of a range of delta tables, the range comprising the delta table;
- aggregating a most recent common schema notation file for each delta table in the range into an aggregated common schema notation file separate and distinct from a delta table;
- and
- sharing the aggregated common schema notation file with the client.
A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- generating a first piece of metadata for a delta table in a data lake, the data lake being a storage containing structured, unstructured, and semi-structured data, the delta table describing differences among multiple versions of a data like file stored in the data lake, the first piece of metadata being of a first metadata type that is not supported by a format of the delta table;
- storing the first piece of metadata in a common schema notation file, the common schema notation file having a format that is used for defining and documenting structure of data models;
- binding the common schema notation file to the delta table, wherein the binding comprises attaching the common schema notation file to the delta table;
- receiving a request from a client for a share of a range of delta tables, the range comprising the delta table;
- aggregating a most recent common schema notation file for each delta table in the range into an aggregated common schema notation file separate and distinct from a delta table;
- and
- sharing the aggregated common schema notation file with the client.
Dependencies are inferred from the claim text.