# US 11,030,179 B2 · External data access with split index > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,179 B2 - Document ID: US11030179B2 - Status: Granted - Kind: B2 - Application number: 15/650,827 - Filed: 2017-07-14 - Published: 2017-11-02 - Granted: 2021-06-08 - Earliest claimed: 2014-01-31 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Nikhil Teletia; Vinitha Reddy Gankidi; Alan D. Halverson; Jignesh M. Patel - Web page: https://patentgrep.com/patent/US11030179B2 ## CPC - G06F 16/2272: {Management thereof} - G06F 16/245: Query processing - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 16/41: Indexing; Data structures therefor; Storage structures - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A split-index can be employed for access to external data. The index can be created on a primary data storage system for data stored externally on a secondary data storage system. After creation, the index can be utilized to expedite at least query execution over the externally stored data. The index can be updated upon detection of changes to data. Further, even when the index is not completely up to date, the index can be exploited for query execution. Furthermore, hybrid execution is enabled with the index and without the index. ## Claims ### Claim 1 A method, comprising: employing at least one processor configured to execute computer-executable instructions stored in memory to perform the following acts: creating an index in a primary data storage system describing data stored externally of the primary data storage system in a secondary data storage system, the secondary data storage system being a non-relational distributed file system; during a period of time during which the index is available for execution with queries of the data within the non-relational distributed file system: initiating execution of at least a portion of a query with the index over a first portion of the data described by a first portion of the index determined to be current and without the index over a second portion of the data described by a second portion of the index determined to be out of date by initiating a scan of the second portion of the data within the non-relational distributed file system to obtain a query response; and performing an incremental update to the index for the second portion of the index describing the second portion of the data for which the scan was initiated. ### Claim 2 (depends on 1) The method of claim 1, further comprising acquiring data from the secondary data storage system for at least the portion of the query based on the index. ### Claim 3 (depends on 2) The method of claim 2, further comprising bypassing a native data processing mechanism of the secondary data storage system in acquiring data from the secondary data storage system, wherein the native data processing mechanism of the secondary data storage system includes a MapReduce processing mechanism. ### Claim 4 (depends on 1) The method of claim 1, further comprising determining whether to execute the query using the index at the primary data storage system based on a selectivity of the query, wherein the selectivity is a probability that the query satisfies a filter expression. ### Claim 5 (depends on 1) The method of claim 1, wherein performing the incremental update includes updating the index based on one or more changes to the data stored externally in the secondary data storage system. ### Claim 6 (depends on 1) The method of claim 1, further comprising determining one or more updates to the index based on one or more changes to the data stored externally in the secondary data storage system concurrently with execution of the query at the primary data storage system. ### Claim 7 (depends on 6) The method of claim 6, wherein performing the incremental update includes committing the one or more updates to the index upon a pause in processing activity at the primary data storage system. ### Claim 8 (depends on 1) The method of claim 1, wherein the primary data storage system is a relational data warehouse system. ### Claim 9 A system, comprising: a processor coupled to a memory, the processor configured to execute computer-executable instruction including the following computer-executable components stored in the memory: a first component configured to create an index in a primary data storage system describing data stored externally of the primary data storage system in a secondary data storage system, the secondary data storage system being a non-relational distributed file system; and a second component configured to, during a period of time during which the index is available for execution with queries of the data within the non-relational distributed file system: initiate execution of at least a portion of a query with the index over a first portion of the data described by a first portion of the index determined to be current and without the index over a second portion of the data described by a second portion of the index determined to be out of date by initiating a scan of the second portion of the data within the non-relational distributed file system to obtain a query response; the processor being further configured to execute the computer-executable instructions to perform an incremental update to the index for the second portion of the index describing the second portion of the data for which the scan was initiated. ### Claim 10 (depends on 9) The system of claim 9, wherein the primary data storage system is a relational data warehouse system. ### Claim 11 (depends on 9) The system of claim 9, wherein: the second portion of the data has been modified since the index was created. ### Claim 12 (depends on 9) The system of claim 9, wherein: a signal indicates that the index is to be utilized in the execution of the query. ### Claim 13 (depends on 9) The system of claim 9, wherein the second component is configured to perform the incremental update to the index in view of changes to the data. ### Claim 14 A computer-readable storage device having instructions stored thereon that enable at least one processor to perform a method upon execution of the instructions, the method comprising: creating an index locally in a relational data warehouse system describing data stored externally of the relational data warehouse system in a non-relational distributed file system; during a period of time during which the index is available for execution with queries of the data within the non-relational distributed file system: initiating execution of at least a portion of a query with the index over a first portion of the data described by a first portion of the index determined to be current and without the index over a second portion of the data described by a second portion of the index determined to be out of date by initiating a scan of the second portion of the data within the non-relational distributed file system to obtain a query response; and performing an incremental update to the index for the second portion of the index describing the second portion of the data for which the scan was initiated. ### Claim 15 (depends on 14) The computer-readable storage device of claim 14, wherein the method further comprises acquiring at least the first portion of the data from the distributed file system for the portion of the query based on the index. ### Claim 16 (depends on 15) The computer-readable storage device of claim 15, wherein the method further comprises acquiring the second portion of the data from the distributed file system for the portion of the query not based on the index. ### Claim 17 (depends on 14) The computer-readable storage device of claim 14, wherein the index incrementally updated in response to one or more changes to the data.