# US 2019/0171677 A1 · Search service for a data fabric system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171677 A1 - Document ID: US20190171677A1 - Status: Published application - Kind: A1 - Application number: 16/264,441 - Filed: 2019-01-31 - Published: 2019-06-06 - Earliest claimed: 2016-09-26 - Assignees: Splunk Inc. - Applicants: Splunk Inc. - Inventors: Sourav Pal; Christopher Pride; Arindam Bhattacharjee; Xiaowei Wang; James Alasdair Robert Hodge; Mustafa Ahamed - Web page: https://patentgrep.com/patent/US20190171677A1 ## CPC - G06F 16/951: Indexing; Web crawling techniques - G06F 16/211: {Schema design and management} - G06F 16/212: {with details for data modelling support} - G06F 16/2455: Query execution - G06F 16/2471: {Distributed queries} - G06F 16/248: Presentation of query results - G06F 16/252: {between a Database Management System and a front-end application} - G06F 16/258: {Data format conversion from or to a database} - G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor - G06F 16/9024: {Graphs; Linked lists (G06F16/9027 takes precedence)} - G06F 16/90335: {Query processing} - G06F 16/9038: Presentation of query results - G06F 16/904: Browsing; Visualisation therefor (for navigating the web G06F16/954; browsing optimisation for the web G06F16/957) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed is a technique that can be performed in a distributed network. The technique can include a search service system that receives an indication of at least a portion of a search scheme to cause worker nodes to obtain search results from distributed data storage systems. The search scheme is defined by a data intake and query system. The search service system defines a search process based on the at least a portion of the search scheme and executes the search process to cause the worker nodes to obtain search results from the distributed data storage systems. The search service system receives a combination of search results based on the search results obtained by the worker nodes from the distributed data storage systems, and causes an output based on the combination of search results obtained by the data intake and query system in accordance with the search scheme. ## Claims ### Claim 1 A method comprising: receiving, by a search service, an indication of at least a portion of a search scheme to cause a plurality of worker nodes to obtain search results from a plurality of distributed data storage systems, the search scheme being defined by a data intake and query system, and the plurality of distributed data storage systems including at least one of an external data storage system communicatively coupled to the data intake and query system over a network or an internal data storage system of the data intake and query system; defining, by the search service, a search process based on the at least a portion of the search scheme defined by the data intake and query system; executing, by the search service, the search process to cause the plurality of worker nodes to obtain a plurality of search results from the plurality of distributed data storage systems; receiving, by the search service, a combination of search results based on the plurality of search results obtained by the plurality of worker nodes from the plurality of distributed data storage systems; and causing, by the search service, an output based on the combination of search results obtained by the data intake and query system in accordance with the search scheme. ### Claim 2 (depends on 1) The method of claim 1, wherein only a portion of the search scheme is received by the search service. ### Claim 3 (depends on 1) The method of claim 1, wherein the plurality of search results is obtained from the external data storage system and the internal data storage system of the data intake and query system. ### Claim 4 (depends on 1) The method of claim 1, wherein at least some of the plurality of search results are time-ordered events retrieved from the internal data storage system, each time-ordered event including a segment of raw machine data and an associated timestamp. ### Claim 5 (depends on 1) The method of claim 1, wherein the combination of search results is derived from the plurality of search results obtained from the external data storage system and the internal data storage system of the data intake and query system. ### Claim 6 (depends on 1) The method of claim 1, wherein the combination of search results includes a plurality of time-ordered events. ### Claim 7 (depends on 1) The method of claim 1, wherein the combination of search results is an aggregation of the plurality of search results collected by the plurality of worker nodes. ### Claim 8 (depends on 1) The method of claim 1, wherein the combination of search results includes structured data retrieved from the external data storage system. ### Claim 9 (depends on 1) The method of claim 1, wherein the combination of search results includes unstructured data retrieved from the internal data storage system. ### Claim 10 (depends on 1) The method of claim 1, wherein the combination of search results communicated to the data intake and query system is in a format specified in the search scheme. ### Claim 11 (depends on 1) The method of claim 1, wherein the combination of search results is based on structured data retrieved from the external data storage system and unstructured data retrieved from the internal data storage system. ### Claim 12 (depends on 1) The method of claim 1, wherein the combination of search results includes data retrieved in parallel from peer indexers of the internal data storage system. ### Claim 13 (depends on 1) The method of claim 1, wherein a device distinct from, and communicatively coupled to, the data intake and query system implements the search service over the network. ### Claim 14 (depends on 1) The method of claim 1, wherein the data intake and query system implements the search service. ### Claim 15 (depends on 1) The method of claim 1, wherein causing the output based on the combination of search results comprises: causing the combination of search results or data indicative of the combination of search results to be displayed on a display device. ### Claim 16 (depends on 1) The method of claim 1, wherein the output is rendered on a display device in a timeline visualization. ### Claim 17 (depends on 1) The method of claim 1, wherein the search scheme is based on user input expressed in a pipelined language. ### Claim 18 (depends on 1) The method of claim 1, wherein the search scheme is based on user input expressed in a structured language. ### Claim 19 (depends on 1) The method of claim 1, wherein the search scheme comprises a plurality of phases, and defining the search process comprises: defining, by the search service, the search process as a directed acyclic graph including at least some of the plurality of phases. ### Claim 20 (depends on 1) The method of claim 1, wherein the search scheme comprises a plurality of phases, and defining the search process comprises: defining, by the search service, the search process to include a first phase to cause the plurality of worker nodes to obtain at least some of the plurality of search results from the external data storage system and a second phase to cause the plurality of worker nodes to obtain at least some of the plurality of search results from the internal data storage system. ### Claim 21 (depends on 1) The method of claim 1, wherein the search scheme comprises a plurality of ordered search phases, and defining the search process comprises: defining, by the search service, the search process to include a first phase to cause the plurality of worker nodes to obtain at least some of the plurality of search results from the external data storage system followed by a second phase to cause the plurality of worker nodes to obtain at least some of the plurality of search results from the internal data storage system. ### Claim 22 (depends on 1) The method of claim 1, wherein the search scheme comprises a plurality of phases, and defining the search process comprises: defining, by the search service, the search process include a first phase to cause the plurality of worker nodes to obtain at least some of the plurality of search results from the external data storage system, followed by a second phase to cause the plurality of worker nodes to obtain at least some of the plurality of search results from the internal data storage system, followed by a third phase to cause the search service to obtain the plurality of search results from the plurality of worker nodes. ### Claim 23 (depends on 1) The method of claim 1, wherein the plurality of search results includes first search results obtained from the internal data storage system in a first format, second search results obtained from the external data storage system in a second format different from the first format, and the combination of search results is in a common format. ### Claim 24 (depends on 1) The method of claim 1, wherein the plurality of search results includes a plurality of time-indexed events obtained from the internal data storage system, data obtained from the external data storage system in a format other than time-indexed events, and the combination of search results is in a specified format. ### Claim 25 (depends on 1) The method of claim 1 further comprising: metering an amount of time and computing resources utilized to complete the search process. ### Claim 26 (depends on 1) The method of claim 1, wherein at least some of the plurality of search results are retrieved from the external data storage system in accordance with a first portion of the search scheme and at least some of the plurality of search results are retrieved from the internal data storage system in accordance with a second portion of the search scheme that is different from the first portion of the search scheme. ### Claim 27 (depends on 1) The method of claim 1, wherein executing the search process further comprises: causing, by the search service, the plurality of worker nodes to produce a plurality of aggregate search results based on a first plurality of partial search results obtained from the external data storage system with a second plurality of partial search results obtained from the internal data storage system. ### Claim 28 (depends on 1) The method of claim 1, wherein executing the search process further comprises: causing, by the search service, the plurality of worker nodes to produce a plurality of aggregate search results as an arrangement of time-ordered events by combining at least some of plurality of search results obtained from the external data storage system and obtained from the internal data storage system. ### Claim 29 A search service system comprising: a processor; and a memory having instructions executable by the processor to cause the search service system to: receive an indication of at least a portion of a search scheme to cause a plurality of worker nodes to obtain search results from a plurality of distributed data storage systems, the search scheme being defined by a data intake and query system, and the plurality of distributed data storage systems including at least one of an external data storage system communicatively coupled to the data intake and query system over a network or an internal data storage system of the data intake and query system; define a search process based on the at least a portion of the search scheme defined by the data intake and query system; execute the search process to cause the plurality of worker nodes to obtain a plurality of search results from the plurality of distributed data storage systems; receive a combination of search results based on the plurality of search results obtained by the plurality of worker nodes from the plurality of distributed data storage systems; and cause an output based on the combination of search results obtained by the data intake and query system in accordance with the search scheme. ### Claim 30 A non-transitory storage medium storing program instructions, execution of which in search service system causes actions comprising: receiving an indication of at least a portion of a search scheme to cause a plurality of worker nodes to obtain search results from a plurality of distributed data storage systems, the search scheme being defined by a data intake and query system, and the plurality of distributed data storage systems including at least one of an external data storage system communicatively coupled to the data intake and query system over a network or an internal data storage system of the data intake and query system; defining a search process based on the at least a portion of the search scheme defined by the data intake and query system; executing the search process to cause the plurality of worker nodes to obtain a plurality of search results from the plurality of distributed data storage systems; receiving a combination of search results based on the plurality of search results obtained by the plurality of worker nodes from the plurality of distributed data storage systems; and causing an output based on the combination of search results obtained by the data intake and query system in accordance with the search scheme.