Collecting streaming event data from applications deployed in a distributed computing environment

A distributed computing system comprising a plurality of physical computers configured to perform operations comprising:

• receiving a plurality of feeds of streaming event data a plurality of feeds of streaming event data of a multi-tenant application having a plurality of tenants, wherein each feed of the plurality of feeds is generated by a respective tenant of the plurality of tenants of the multi-tenant application;
• routing feed data from each of the tenants of the multi-tenant application to a different respective channel of a plurality of channels, wherein each channel comprises a software process executing on one of the plurality of computers, wherein each channel of the plurality of channels is configured to receive and store feed data until the feed data is consumed by a data sink;
• computing a load metric for a first channel executing on a first computer of the plurality of computers, wherein the first channel is allocated to receive first feed data generated by a first tenant of the plurality of tenants of the multi-tenant application;
• determining that the load metric for the first channel exceeds a threshold;
• in response to determining that the load metric for the first channel exceeds the threshold, selecting, from among different quantities of allocable channels, a particular quantity of new channels for the first feed based on an amount by which the load metric exceeds the threshold;
• allocating the particular quantity of new channels for the first feed data generated by the first tenant of the plurality of tenants of the multi-tenant application;
• and
• redirecting feed data from the first tenant to the new channels instead of the first channel.

A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by a distributed computing system of a plurality of physical computers causes the distributed computing system to perform operations comprising:

• receiving a plurality of feeds of streaming event data a plurality of feeds of streaming event data of a multi-tenant application having a plurality of tenants, wherein each feed of the plurality of feeds is generated by a respective tenant of the plurality of tenants of the multi-tenant application;
• routing feed data from each of the tenants of the multi-tenant application to a different respective channel of a plurality of channels, wherein each channel comprises a software process executing on one of the plurality of computers, wherein each channel of the plurality of channels is configured to receive and store feed data until the feed data is consumed by a data sink;
• computing a load metric for a first channel executing on a first computer of the plurality of computers, wherein the first channel is allocated to receive first feed data generated by a first tenant of the plurality of tenants of the multi-tenant application;
• determining that the load metric for the first channel exceeds a threshold;
• in response to determining that the load metric for the first channel exceeds the threshold, selecting, from among different quantities of allocable channels, a particular quantity of new channels for the first feed based on an amount by which the load metric exceeds the threshold;
• allocating the particular quantity of new channels for the first feed data generated by the first tenant of the plurality of tenants of the multi-tenant application;
• and
• redirecting feed data from the first tenant to the new channels instead of the first channel.