Data processing independent of storage, format or schema
AssigneeHitachi Vantara, Ltd.
InventorJacob Russell GMINDER
Life of this patent
Estimating the term…
Abstract
In some examples, a computing device is able to communicate with a plurality of data clusters. The computing device may determine a location, format and/or schema of data resource(s) stored in the data clusters upon execution of a process. Metadata may be searched and read to determine the location, format and/or schema of data resource(s) and those data resources may be written to other destinations within the data cluster(s). The locations of the destinations may also be determined at runtime. In some examples, parameters may be configured or specified to drive the execution of one or more operations or processes.
Claims15
A system comprising:
- a first computing device configured to communicate over a network with a plurality of data cluster computing devices associated with a plurality of data clusters storing data resources, and configured to manage metadata associated with the stored data resources, the first computing device executing a single instance of an application to perform operations comprising:
- obtaining parameters indicating first metadata, and second metadata in a respective data cluster;
- searching the metadata using the first metadata as a search query and returning metadata as a first search result;
- obtaining a first location of one or more data resources in a respective data cluster based on the returned metadata in the first search result;
- reading contents of the one or more data resources from the first location of the respective data cluster;
- searching the metadata using the second metadata as a search query and returning metadata as a second search result;
- obtaining a second location in a respective data cluster based on the returned metadata in the second search result;
- writing the read contents of the one or more data resources to the obtained second location based on the second search result.
A method, executing a single instance of an application by a first computing device to perform operations comprising:
- obtaining parameters indicating first metadata, and second metadata in a respective data cluster;
- searching the metadata using the first metadata as a search query and returning metadata as a first search result;
- obtaining a first location of one or more data resources in a respective data cluster based on the returned metadata in the first search result;
- reading contents of the one or more data resources from the first location of the respective data cluster;
- searching the metadata using the second metadata as a search query and returning metadata as a second search result;
- obtaining a second location in a respective data cluster based on the returned metadata in the second search result;
- and
- writing the read contents of the one or more data resources to the obtained second location based on the second search result.
One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a first computing device, configure the first computing device to perform operations comprising:
- obtaining parameters indicating first metadata, and second metadata in a respective data cluster;
- searching the metadata using the first metadata as a search query and returning metadata as a first search result;
- obtaining a first location of one or more data resources in a respective data cluster based on the returned metadata in the first search result;
- reading contents of the one or more data resources from the first location of the respective data cluster;
- searching the metadata using the second metadata as a search query and returning metadata as a second search result;
- obtaining a second location in a respective data cluster based on the returned metadata in the second search result;
- and
- writing the read contents of the one or more data resources to the obtained second location based on the second search result.
Dependencies are inferred from the claim text.