# US 2025/0384084 A1 · Processing data using nodes in a scalable environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0384084 A1 - Document ID: US20250384084A1 - Status: Published application - Kind: A1 - Application number: 19/303,260 - Filed: 2025-08-18 - Published: 2025-12-18 - Earliest claimed: 2017-07-31 - Assignees: Splunk Inc. - Applicants: Splunk Inc. - Inventors: Alexandros BATSAKIS; Ashish Mathew; Christopher Madden Pride; Bharath Kishore Reddy Aleti; Sourav Pal; Arindam Bhattacharjee; James Monschke; Sai Krishna Sajja; Igor Stojanovski; Tameem Anwar; Paul J. Lucas; Eric Woo; Steve Wong - Web page: https://patentgrep.com/patent/US20250384084A1 ## CPC - G06F 16/2471: {Distributed queries} - G06F 16/901: Indexing; Data structures therefor; Storage structures (for retrieval from the web G06F16/951) - G06F 3/0604: {Improving or facilitating administration, e.g. storage management} - G06F 3/0644: {Management of space entities, e.g. partitions, extents, pools} - G06F 3/065: {Replication mechanisms} - G06F 3/0652: {Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket} - G06F 3/0653: {Monitoring storage devices or systems} - G06F 3/0656: {Data buffering arrangements} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 16/23: Updating - G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor - G06F 16/903: Querying (for retrieval from the web G06F16/953) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives raw machine data at an indexing system, and stores at least a portion of the raw machine data in buckets using containerized indexing nodes instantiated in a containerized environment. The data intake and query system stores the buckets in a shared storage system. ## Claims ### Claim 1 (canceled) ### Claim 2 A method comprising: receiving a query from a client device, the query including criteria for identifying search results from a set of data comprising messages published to a publish-subscribe messaging system; and defining, by one or more computing devices, a query processing scheme for obtaining and processing the set of data, the query processing scheme including a dynamic allocation of partitions to process the messages to identify the search results, wherein defining the query processing scheme includes: determining a number of messages queues on the publish-subscribe messaging system in which the messages are placed; dynamically allocating individual queues of the message queues to individual partitions, the individual partitions to repeatedly collect messages from a corresponding allocated individual queue over a plurality of time windows of a specified duration, and at the end of each time window, cause the messages received during a time window to be processed by a processing partition; and executing the query based on the query processing scheme. ### Claim 3 (depends on 2) The method of claim 2, wherein each time window is a predetermined length of time. ### Claim 4 (depends on 2) The method of claim 2, wherein dynamically allocating individual queues of the message queues to individual partitions comprises evenly allocating the individual queues among one or more worker node computing devices. ### Claim 5 (depends on 2) The method of claim 2, wherein a number of the individual partitions is equal to a number of message queues, and wherein dynamically allocating individual queues to individual partitions comprises allocating a single queue to each individual partition. ### Claim 6 (depends on 2) The method of claim 2, wherein a number of the individual partitions is less than a number of the message queues, and wherein dynamically allocating individual queues to individual processors comprises allocating multiple queues to each of the individual partitions. ### Claim 7 (depends on 2) The method of claim 2, wherein at least two of the individual partitions are implemented within a common worker node computing device. ### Claim 8 (depends on 2) The method of claim 2, wherein each of the individual partitions is implemented within a different worker node computing device. ### Claim 9 (depends on 2) The method of claim 2, wherein the set of data corresponds to a topic of the publish-subscribe messaging system. ### Claim 10 (depends on 2) The method of claim 2, wherein the set of data corresponds to a topic of the publish-subscribe messaging system, and wherein determining the number of messages queues on the publish-subscribe messaging system in which the messages are placed comprises querying the publish-subscribe messaging system for identifying information of one or more message queues corresponding to the topic. ### Claim 11 (depends on 2) The method of claim 2 further comprising identifying, from the criteria, that the query is directed to data on the publish-subscribe messaging system based at least in part on the criteria of the query. ### Claim 12 (depends on 2) The method of claim 2 further comprising identifying, from the criteria, that the query is directed to data on the publish-subscribe messaging system based at least in part on the criteria of the query. ### Claim 13 (depends on 2) The method of claim 2 further comprising transmitting search results generated from a collection of messages to a data destination and, subsequent to transmitting the search results search results generated from the collection of messages, transmitting an acknowledgement of each message within the collection of messages to the publish-subscribe messaging system. ### Claim 14 (depends on 2) The method of claim 2 further comprising transmitting search results generated from a collection of messages to a data destination and, subsequent to transmitting the search results search results generated from the collection of messages, transmitting an acknowledgement of each message within the collection of messages to the publish-subscribe messaging system, wherein transmitting the acknowledgement of each message within the collection of messages to the publish-subscribe messaging system comprises instructing individual partitions that collected each message within the collection of messages to transmit acknowledgements for each message to the publish-subscribe messaging system. ### Claim 15 (depends on 2) The method of claim 2, wherein the messages are associated with a topic on the publish-subscribe messaging system, wherein the method further comprising transmitting the search to the publish-subscribe messaging system as messages for a second topic. ### Claim 16 (depends on 2) The method of claim 2, wherein the publish-subscribe messaging system queues the messages for delivery to indexers that index data within the messages. ### Claim 17 (depends on 2) The method of claim 2, wherein the publish-subscribe messaging system queues the messages for delivery to indexers that index data within the messages, and wherein the indexers place indexed data generated from the messages within a common storage also accessible to the processors. ### Claim 18 A system comprising: a data store including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions, wherein execution of the computer-executable instructions causes the system to: receive a query from a client device, the query including criteria for identifying search results from a set of data comprising messages published to a publish-subscribe messaging system; and define, by one or more computing devices, a query processing scheme for obtaining and processing the set of data, the query processing scheme including a dynamic allocation of partitions to process the messages to identify the search results, wherein defining the query processing scheme includes: determining a number of messages queues on the publish-subscribe messaging system in which the messages are placed; dynamically allocating individual queues of the message queues to individual partitions, the individual partitions to repeatedly collect messages from a corresponding allocated individual queue over a plurality of time windows of a specified duration, and at the end of each time window, cause the messages received during a time window to be processed by a processing partition; and executing the query based on the query processing scheme. ### Claim 19 (depends on 18) The system of claim 18, wherein execution of the computer-executable instructions causes the system to transmit search results generated from a collection of messages to a data destination, and subsequent to transmitting the search results generated from the collection of messages, transmit an acknowledgement of each message within the collection of messages to the publish-subscribe messaging system. ### Claim 20 Non-transitory computer-readable media including computer-executable instructions that, when executed by a computing system, cause the computing system to: receive a query from a client device, the query including criteria for identifying search results from a set of data comprising messages published to a publish-subscribe messaging system; and define, by one or more computing devices, a query processing scheme for obtaining and processing the set of data, the query processing scheme including a dynamic allocation of partitions to process the messages to identify the search results, wherein defining the query processing scheme includes: determining a number of messages queues on the publish-subscribe messaging system in which the messages are placed; dynamically allocating individual queues of the message queues to individual partitions, the individual partitions to repeatedly collect messages from a corresponding allocated individual queue over a plurality of time windows of a specified duration, and at the end of each time window, cause the messages received during a time window to be processed by a processing partition; and executing the query based on the query processing scheme. ### Claim 21 (depends on 20) The non-transitory computer-readable media of claim 20, wherein execution of the computer-executable instructions causes the computing system to transmit search results generated from a collection of messages to a data destination, and subsequent to transmitting the search results generated from the collection of messages, transmit an acknowledgement of each message within the collection of messages to the publish-subscribe messaging system.