Predicting the next best compressor in a stream data platform

A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:

• receiving a data stream;
• collecting a sequence of one or more batches of data from the data stream;
• analyzing the batches of data in the sequence;
• obtaining compressor choices for the batches of data in the sequence;
• obtaining a new batch of data from the data stream;
• analyzing the new batch of data;
• based on the analyzing and the compressor choices for the batches of data in the sequence, and the analyzing of the new batch of data, generating a prediction that identifies a recommended data compressor for the new batch of data, and the generating of the prediction is performed by a compressor prediction model that was created and/or trained by a training module using a training data set comprising a sequence of (compressor, data pattern) pairs;
• and
• in response to a change in the data stream, reducing an amount of one or more computing system resources needed for handling of the new batch of data by compressing the new batch of data using the recommended data compressor.