External data access with split index
AssigneeMicrosoft Technology Licensing, LLC
InventorsNikhil Teletia, Vinitha Reddy Gankidi, Alan D. Halverson, Jignesh M. Patel
Life of this patent
Estimating the term…
Abstract
A split-index can be employed for access to external data. The index can be created on a primary data storage system for data stored externally on a secondary data storage system. After creation, the index can be utilized to expedite at least query execution over the externally stored data. The index can be updated upon detection of changes to data. Further, even when the index is not completely up to date, the index can be exploited for query execution. Furthermore, hybrid execution is enabled with the index and without the index.
Claims17
A method, comprising:
- employing at least one processor configured to execute computer-executable instructions stored in memory to perform the following acts:
- creating an index in a primary data storage system describing data stored externally of the primary data storage system in a secondary data storage system, the secondary data storage system being a non-relational distributed file system;
- during a period of time during which the index is available for execution with queries of the data within the non-relational distributed file system:
- initiating execution of at least a portion of a query with the index over a first portion of the data described by a first portion of the index determined to be current and without the index over a second portion of the data described by a second portion of the index determined to be out of date by initiating a scan of the second portion of the data within the non-relational distributed file system to obtain a query response;
- and
- performing an incremental update to the index for the second portion of the index describing the second portion of the data for which the scan was initiated.
A system, comprising:
- a processor coupled to a memory, the processor configured to execute computer-executable instruction including the following computer-executable components stored in the memory:
- a first component configured to create an index in a primary data storage system describing data stored externally of the primary data storage system in a secondary data storage system, the secondary data storage system being a non-relational distributed file system;
- and
- a second component configured to, during a period of time during which the index is available for execution with queries of the data within the non-relational distributed file system:
- initiate execution of at least a portion of a query with the index over a first portion of the data described by a first portion of the index determined to be current and without the index over a second portion of the data described by a second portion of the index determined to be out of date by initiating a scan of the second portion of the data within the non-relational distributed file system to obtain a query response;
- the processor being further configured to execute the computer-executable instructions to perform an incremental update to the index for the second portion of the index describing the second portion of the data for which the scan was initiated.
A computer-readable storage device having instructions stored thereon that enable at least one processor to perform a method upon execution of the instructions, the method comprising:
- creating an index locally in a relational data warehouse system describing data stored externally of the relational data warehouse system in a non-relational distributed file system;
- during a period of time during which the index is available for execution with queries of the data within the non-relational distributed file system:
- initiating execution of at least a portion of a query with the index over a first portion of the data described by a first portion of the index determined to be current and without the index over a second portion of the data described by a second portion of the index determined to be out of date by initiating a scan of the second portion of the data within the non-relational distributed file system to obtain a query response;
- and
- performing an incremental update to the index for the second portion of the index describing the second portion of the data for which the scan was initiated.
Dependencies are inferred from the claim text.