System and method for conditional analysis of network traffic

An apparatus, comprising:

• an input circuit, which is configured to receive network traffic that carries content items for processing by an analytics system;
• and
• a processor coupled to a memory, which is configured to:
• extract a content item from the network traffic, wherein extracting the content item includes deriving a respective unique identifier for the content item;
• retrieve and output a cached analytics outcome of the content item in response to finding that the content item is a duplicate of a previous content that was already processed by the analytics system and cached;
• and
• count, for a given content item, a number of matching occurrences of the given content item, and caching the number of matching occurrences in the cache memory in association with the unique identifier of the given content item, wherein the number of matching occurrences is multiplied by a weight factor that is based on a processing time of the content item.

A non-transitory computer readable medium, having instructions stored thereon that, when executed by a computing system, cause the computing system to at least:

• receive network traffic that carries content items for processing by an analytics system;
• extract a content item from the network traffic, wherein extracting the content item includes deriving a respective unique identifier for the content item;
• retrieve and output a cached analytics outcome of the content item in response to finding that the content item is a duplicate of previous content that was already processed by the analytics system and cached;
• and
• count, for a given content item, a number of matching occurrences of the given content item, and caching the number of matching occurrences in the cache memory in association with the unique identifier of the given content item, wherein the number of matching occurrences is multiplied by a weight factor that is based on a processing time of the content item.