Virtual streams

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• determining a plurality of data streams active on the server that match a query received from a client device;
• generating a virtual stream comprising data received from the plurality of data streams;
• transmitting to the client device, a unique identifier identifying the virtual stream and one or more segments of data of the virtual stream;
• detecting that a flow rate of data segments received by the virtual stream has decreased below a first threshold amount;
• notifying the client device to reduce one or more consumers to process the virtual stream;
• and
• decreasing a number of partitions of the virtual stream by at least one partition, in response to receiving a notice from the client device that the client device has reduced at least one consumer to process the virtual stream.

A computer-implemented method performed by a server, comprising:

• determining a plurality of data streams active on the server that match a query received from a client device;
• generating a virtual stream comprising data received from the plurality of data streams;
• transmitting to the client device, a unique identifier identifying the virtual stream and one or more segments of data of the virtual stream;
• detecting that a flow rate of data segments received by the virtual stream has increased above a first threshold amount;
• notifying the client device to add one or more consumers to process the virtual stream;
• and
• increasing a number of partitions of the virtual stream by at least one partition, in response to receiving a notice from the client device that the client device has added at least one consumer to process the virtual stream.