Data system configured to transparently cache data of data sources and access the cached data

A method comprising:

• receiving a query configured for reading a data object stored at an external data source,
• wherein a first query result that satisfies the query includes the data object stored at the external data source;
• generating a query plan by parsing the query into a plurality of phases, each phase being configured to read a fragment of the data object from the external data source;
• generating a unit of hashing corresponding to an output of a hash algorithm based on an input indicative of the data object;
• using a unit of hashing to map fragments of the data object of the external data source to a cluster of nodes of a data system;
• generating a read request for the data object in accordance with the unit of hashing to read the fragments of the data object of the external data source;
• processing the read request by the cluster of nodes that divides the data object into discrete logical blocks, the read request using the unit of hashing to determine a link to one or more logical blocks of the external data source and using storage software to read the one or more logical blocks instead of reading the data object stored at the external data source;
• and
• returning a second query result that is determined to satisfy the query, the second query result including the data object obtained from the one or more logical blocks.

A method comprising:

• receiving a query configured for reading a data file stored at an external storage device, the data file being divided into multiple discrete logical blocks and associated with different instances of metadata stored at both a local cache storage device and the external storage device,
• wherein the different instances of metadata are mapped between the local cache storage device and the external storage device based on a unit of hashing, and
• wherein the unit of hashing corresponds to an output of a hash algorithm based on an input indicative of the data file;
• comparing a cached instance of a particular metadata at the local cache storage device with a stored instance of the particular metadata at the external storage device;
• determining that the cached instance of the particular metadata is different from the stored instance of the particular metadata;
• responsive to determining that the cached instance of the particular metadata is different from the stored instance of the particular metadata, reading the data file stored at the external storage device instead of reading the cached instance of the particular metadata from the local cache storage device;
• returning a query result including the data file obtained from the external storage device, the query result being determined to satisfy the query;
• and
• updating the multiple discrete logical blocks of the local cache storage device with the data file obtained from the external storage device, the updated multiple discrete logical blocks include an updated cached instance of the particular metadata.

A method comprising:

• receiving, by a data system, a read request configured for reading a data file that is stored at a local cache storage;
• wherein a copy of the data file is stored at an external data storage,
• wherein the data file is stored at a storage location of the local cache storage and mapped to the external data storage based on a unit of hashing, and
• wherein the unit of hashing corresponds to an output of a hash algorithm based on an input indicative of the data file;
• selecting, by the data system, a particular format reader of a plurality of format readers, the plurality of format readers being configured to read different types of data files, the particular format reader being selected based on a type of the data file in the read request;
• modifying, by using the particular format reader, the read request to include an attribute of the data file, the attribute depending on the type of the data file;
• parsing, by using the particular format reader, the data file for the read request into discrete logical blocks depending on the type of the data file;
• and
• reading, by using the particular format reader, data of the data file and the attribute stored at the external data storage unless the data file stored at the local cache storage is a current version of the data file stored and the attribute such that the data file and the attribute are read from the local cache storage instead of being read from the external data storage.