Version control system using content-based datasets and dataset snapshots

A computer-implemented method of managing different versions of data objects for a version control system (VCS) during a lifecycle of the data objects, comprising:

• identifying data objects that evolve through the different versions during the lifecycle;
• producing a dataset for the data objects data as a group by scanning the data objects to identify metadata of the grouped data to be re-versioned together throughout the lifecycle, and storing the identified metadata in the dataset;
• executing a user entered query against a catalog comprising a plurality of data catalogs to produce the dataset, wherein the dataset comprises both dynamic dataset data and static dataset data, wherein the static dataset data comprises a fixed amount of data set at a time of creation, and the dynamic dataset data comprises an amount of data that changes over time;
• storing the dynamic dataset data in a first data catalog of the catalog and the static dataset data in a second data catalog of the catalog, wherein the first data catalog is used when new data is ingested by the VCS;
• converting the dynamic dataset data into a static dataset by copying results of a query of the dynamic dataset data into the second data catalog by copying data from a first storage system to a second storage system and adding an entry to the second data catalog to record an identifier of the copied data, and integrating a data mover to the static dataset to move the data as part of an workflow exposed in the catalog to process the user entered query;
• generating dataset snapshots as read-only dataset components for the dataset as it progresses along the lifecycle;
• copying the dataset to a remote storage location using a dataset backup;
• assigning a snapshot-ID to each dataset snapshot for tracking a corresponding snapshot through the commit map and commit-tree;
• and
• performing one or more VCS operations on an identified dataset snapshot including at least one of a commit, checkout, merge, branch, or merge-branch operation.

A computer-implemented method of managing different versions of data objects for a version control system (VCS) during a lifecycle of the data objects, comprising:

• implementing the VCS to manage changes to software programs, documents, web sites, and other content data embodying the data objects, and wherein the first version and revised version are each denoted by successive alphanumeric version character, and wherein each identifier of the snapshot-ID and commit-ID reference the version character;
• producing a dataset for the data objects data as a group by scanning the data objects to identify metadata of the grouped data to be re-versioned together, and storing the identified metadata in the dataset;
• executing a user entered query against a catalog comprising a plurality of data catalogs to produce the dataset, wherein the dataset comprises both dynamic dataset data and static dataset data, wherein the static dataset data comprises a fixed amount of data set at a time of creation, and the dynamic dataset data comprises an amount of data that changes over time;
• storing the dynamic dataset data in a first data catalog of the catalog and the static dataset data in a second data catalog of the catalog, wherein the first data catalog is used when new data is ingested by the VCS;
• converting the dynamic dataset data into a static dataset by copying results of a query of the dynamic dataset data into the second data catalog by copying data from a first storage system to a second storage system and adding an entry to the second data catalog to record an identifier of the copied data, and integrating a data mover to the static dataset to move the data as part of an workflow exposed in the catalog to process the user entered query;
• and
• generating dataset snapshots as read-only dataset components for the dataset as it progresses along the lifecycle, wherein the dataset is generated by gathering the identified metadata for storage in a data catalog, and executing a user entered query comprising metadata selectors as dataset tags for matching against the cataloged metadata to generate the dataset.