Method and system for identifying duplicate packets in flow-based network monitoring system

A network monitoring device configured to:

• collect a plurality of packets from one or more observation points of a network,
• store a plurality of packet values corresponding to the plurality of received packets in a buffer,
• determine a current packet arrival rate based on two or more of the plurality of packets, collect a new packet,
• compare a new packet value corresponding to the new packet with each of a number of samples of the plurality of packet values, wherein the number of samples is determined based on the current packet arrival rate;
• and
• determine, based on the comparison, whether the new packet is a duplicate packet.

A network monitoring device configured to:

• collect a plurality of received packets from one or more observation points of a network,
• store a queue of packet values corresponding to the plurality of received packets in an order in which the received packets are collected,
• determine a current packet arrival rate based on the plurality of received packets,
• determine a number of samples based on the current packet arrival rate,
• collect a new packet, compare a new packet value of the new packet with each packet value of the queue of packet values until at least one stop limit of a plurality of stop limits is reached, wherein the plurality of stop limits includes a first stop limit of finding a matching packet value in the queue of packet values that corresponds to the new packet value, and a second stop limit of comparing the new packet value with the number of samples of the queue of packet values, and
• determine, based on the comparison, whether the new packet is a duplicate packet.