Processing data using nodes in a scalable environment

A method comprising:

• receiving a query from a client device, the query including criteria for identifying search results from a set of data comprising messages published to a publish-subscribe messaging system;
• and
• defining, by one or more computing devices, a query processing scheme for obtaining and processing the set of data, the query processing scheme including a dynamic allocation of partitions to process the messages to identify the search results, wherein defining the query processing scheme includes:
• determining a number of messages queues on the publish-subscribe messaging system in which the messages are placed;
• dynamically allocating individual queues of the message queues to individual partitions, the individual partitions to repeatedly collect messages from a corresponding allocated individual queue over a plurality of time windows of a specified duration, and at the end of each time window, cause the messages received during a time window to be processed by a processing partition;
• and
• executing the query based on the query processing scheme.

A system comprising:

• a data store including computer-executable instructions;
• and
• one or more processors configured to execute the computer-executable instructions, wherein execution of the computer-executable instructions causes the system to:
• receive a query from a client device, the query including criteria for identifying search results from a set of data comprising messages published to a publish-subscribe messaging system;
• and
• define, by one or more computing devices, a query processing scheme for obtaining and processing the set of data, the query processing scheme including a dynamic allocation of partitions to process the messages to identify the search results, wherein defining the query processing scheme includes:
• determining a number of messages queues on the publish-subscribe messaging system in which the messages are placed;
• dynamically allocating individual queues of the message queues to individual partitions, the individual partitions to repeatedly collect messages from a corresponding allocated individual queue over a plurality of time windows of a specified duration, and at the end of each time window, cause the messages received during a time window to be processed by a processing partition;
• and
• executing the query based on the query processing scheme.

Non-transitory computer-readable media including computer-executable instructions that, when executed by a computing system, cause the computing system to:

• receive a query from a client device, the query including criteria for identifying search results from a set of data comprising messages published to a publish-subscribe messaging system;
• and
• define, by one or more computing devices, a query processing scheme for obtaining and processing the set of data, the query processing scheme including a dynamic allocation of partitions to process the messages to identify the search results, wherein defining the query processing scheme includes:
• determining a number of messages queues on the publish-subscribe messaging system in which the messages are placed;
• dynamically allocating individual queues of the message queues to individual partitions, the individual partitions to repeatedly collect messages from a corresponding allocated individual queue over a plurality of time windows of a specified duration, and at the end of each time window, cause the messages received during a time window to be processed by a processing partition;
• and
• executing the query based on the query processing scheme.