Network recorders with entropy and value based packet truncation

A computer-implemented method of conserving storage space in a network recorder, the method comprising:

• receiving a computer packet including a header and payload data;
• estimating entropy of the payload data in the computer packet;
• determining if storage of the computer packet is of value or not based on the header of the computer packet;
• and
• storing all or a portion of the computer packet into a storage device based on the estimated entropy and the value determination.

A network recorder in a computer network, the network recorder comprising:

• a storage device,
• a motherboard,
• a plurality of daughter-cards coupled to the motherboard, each daugthercard including
• an input port to receive ingress computer data packets into the computer network from a wide area network and egress computer data packets out from the computer network to the wide area network;
• a packet analyzer coupled to the input port to receive the ingress and egress computer packets, the packet analyzer to identify header fields and a starting point of the payload data in the ingress and egress computer packets;
• an entropy calculator coupled to the packet analyzer to receive the starting point of the payload data, the entropy calculator to examine each octet of the payload data to generate an entropy estimate of the ingress and egress computer packets;
• and
• a comparator coupled to the entropy calculator to receive the entropy estimate of the ingress and egress computer packets, the comparator to compare the entropy estimate with an entropy truncation threshold to generate an entropy exceed signal to indicate that the payload data can be truncated to conserve storage space.