System and method for use with a distributed event streaming environment for making services resilient of producer failures

A method for use with a distributed event streaming environment, for making services resilient of producer failures, comprising:

• providing at a computer comprising one or more processors a distributed event streaming environment, wherein the distributed event streaming environment includes a cluster of nodes and one or more partitions distributed across the nodes, and a producer that operates to receive an input data as messages and communicate with one or more brokers to write the data into partitioned topics;
• wherein when a determination is made that one or more messages could not be sent to a particular topic after a timeout error the system operates so that:
• the messages that could not be sent are stored in a centralized cache;
• a key-partitioner process is used to pre-compute a partition ID into which the message will be re-sent;
• the pre-computed partition ID is used to compute the key of the cache entry for the message as stored within the centralized cache;
• and
• a recurrent watchdog associated with a group of microservice resources operates to query the centralized cache for the messages to be re-sent into the partitions pertaining to the microservice resources,
• wherein the watchdog is configured to operate on a periodic basis to determine which messages the watchdog should attempt to resend to the producer, wherein the watchdog process looks in the centralized cache for messages stored thereon that are destined for its partitions;
• wherein upon a successful resend of the messages to a producer, those messages are removed from the centralized cache, and remaining messages are stored in the centralized cache for use during a subsequent execution of the watchdog.

A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform a method comprising:

• providing a distributed event streaming environment, wherein the distributed event streaming environment includes a cluster of nodes and one or more partitions distributed across the nodes, and a producer that operates to receive an input data as messages and communicate with one or more brokers to write the data into partitioned topics;
• wherein when a determination is made that one or more messages could not be sent to a particular topic after a timeout error the system operates so that:
• the messages that could not be sent are stored in a centralized cache;
• a key-partitioner process is used to pre-compute a partition ID into which the message will be re-sent;
• the pre-computed partition ID is used to compute the key of the cache entry for the message as stored within the centralized cache;
• and
• a recurrent watchdog associated with a group of microservice resources operates to query the centralized cache for the messages to be re-sent into the partitions pertaining to the microservice resources,
• wherein the watchdog is configured to operate on a periodic basis to determine which messages the watchdog should attempt to resend to the producer, wherein the watchdog process looks in the centralized cache for messages stored thereon that are destined for its partitions;
• wherein upon a successful resend of the messages to a producer, those messages are removed from the centralized cache, and remaining messages are stored in the centralized cache for use during a subsequent execution of the watchdog.