Method and system for processing a file to identify unexpected file types

A system for detecting unexpected file or packet types, the system comprising:

• a processor and a memory for storing program code, the program code comprising:
• a program code portion configured to receive a file or packet to be tested, the file or packet formed from a sequential series of information units, each information unit within the file or packet included within a predetermined set of information units;
• a program code portion configured to compute a first information unit-pair entropy density measurement for the received file or packet using a first probability matrix which tabulates probabilities of occurrence for each possible sequential pair of information units of the predetermined set of information units;
• and
• a program code portion configured to compare the first computed information unit-pair entropy density measurement with a threshold associated with a first expected file or packet type and determining whether the received file or packet is of an unexpected file or packet type or of the first expected file or packet type;
• and
• wherein the processor is configured to execute the program code portions stored in the memory.

A non-transitory computer readable medium for storing computer implementable instructions for causing a computer system to perform a method for detecting unexpected file or packet types, the method comprising the steps of:

• receiving a file or packet to be tested, the file or packet formed from a sequential series of information units, each information unit within the file or packet included within a predetermined set of information units;
• computing a first information unit-pair entropy density measurement for the received file or packet using a first probability matrix which tabulates probabilities of occurrence for each possible sequential pair of information units of the predetermined set of information units;
• and
• comparing the first computed information unit-pair entropy density measurement with a threshold associated with a first expected file or packet type and determining whether the received file or packet is of an unexpected file or packet type or of the first expected file or packet type.