Logging, streaming and analytics platforms using any S3 compatible store as primary store

A computerized method comprising:

• building an information technology operations analytics (ITOA) stack, wherein the ITOA stack uses a compatible object storage accessible through a web service interface as its primary storage;
• through the web service interface, ingesting an ingested data at rest;
• storing a set of metadata of the ingested data in a virtual machine or a container;
• storing a set of primary data of the ingested data as a set of objects in an object store;
• packaging one or more data units of the set of primary data by:
• writing over the set of objects at select intervals;
• breaking the set of objects into chunks to reduce overwrites;
• partitioning the primary data by time, wherein the partitioning by time ingests an arbitrary time series data and queries the arbitrary time series data to implement a time-based partitioning version of the primary data;
• filtering the time-based partitioning version of the primary data by implementing an isolation on the primary data by separating logs of the primary data by an environment type of the primary data, a tenant of the primary data, and a container orchestration system namespace to provide an isolation of the data at a bucket level;
• creating one or more application boundaries at the time of ingesting to generate a granular application data;
• querying the ingested data back using the set of metadata;
• using the metadata to fetch a correct object;
• and
• with the correct object, obtaining the granular application data, and wherein the granular application data is used to determine an Internet Protocol (IP) origin of the set of primary data;
• and
• caching gateway that caches a specified amount of the object to reduce an API cost that is charged by a respective cloud vendor.