Pluggable storage system for parallel query engines
AssigneeEMC IP Holding Company LLC
InventorsSameer Tiwari, Milind Arun Bhandarkar, Bhooshan Deepak Mogal
Life of this patent
Estimating the term…
Abstract
A method, article of manufacture, and apparatus for managing data. In some embodiments, this includes, receiving a query from a client, based on the received query, analyzing a catalog for location information, based on the analysis, determining a first storage system, an associated first file system and an associated first protocol, using the associated first protocol to communicate with the first storage system, and performing at least a portion of the query on the first storage system.
Claims18
A method for managing data, comprising:
- receiving, by a universal namenode, a query from a client;
- based at least in part on the received query, accessing a catalog service and searching a catalog provided by the catalog service for location information of one or more files responsive to the query;
- based at least in part on a result of the search of the catalog, determining to move at least one of the plurality of files from a second storage system to a first storage system, and determining to communicate with the first storage system in connection with the one or more files responsive to the queries;
- performing at least a portion of the query on the first storage system;
- and
- providing, to the client, results of the query such that in the event that various portions of the results correspond to query results stored on a set of a plurality of storage systems, the results of the query are presented in a unified view across the set of the plurality of storage systems and appear, from a perspective of the client, to exist from a single namespace, wherein the plurality of storage systems comprises at least the first storage system and the second storage system.
A system for managing data, comprising one or more processors configured to:
- receive a query from a client;
- based at least in part on the received query, accessing a catalog service and searching a catalog provided by the catalog service for location information of one or more files responsive to the query;
- based at least in part on a result of the search of the catalog, determine to move at least one of the plurality of files from a second storage system to a first storage system, and determine to communicate with the first storage system in connection with the one or more files responsive to the queries;
- perform at least a portion of the query on the first storage system;
- and
- provide, to the client, results of the query such that in the event that various portions of the results correspond to query results stored on a set of a plurality of storage systems, the results of the query are presented in a unified view across the set of the plurality of storage systems and appear, from a perspective of the client, to exist from a single namespace, wherein the plurality of storage systems comprises at least the first storage system and the second storage system.
A computer program product for processing data, comprising a non-transitory computer readable medium having program instructions embodied therein for:
- receiving a query from a client;
- based at least in part on the received query, accessing a catalog service and searching a catalog provided by the catalog service for location information of one or more files responsive to the query;
- based at least in part on a result of the search of the catalog, determining to move at least one of the plurality of files from a second storage system to a first storage system, and determining to communicate with the first storage system in connection with the one or more files responsive to the queries;
- performing at least a portion of the query on the first storage system;
- and
- providing, to the client, results of the query such that in the event that various portions of the results correspond to query results stored on a set of a plurality of storage systems, the results of the query are presented in a unified view across the set of the plurality of storage systems and appear, from a perspective of the client, to exist from a single namespace, wherein the plurality of storage systems comprises at least the first storage system and the second storage system.
Dependencies are inferred from the claim text.