Anomaly detection using sequences of system calls

A non-transitory computer readable storage medium comprising computer executable instructions, the computer executable instructions executable by a processor, the computer executable instructions comprising:

• instructions executable to receive a message indicating an invocation of a programmatic procedure of an operating system, the message including a programmatic procedure identifier, a sender process identifier, and a receiver process identifier, wherein the invocation of the programmatic procedure is one invocation in a series of invocations of operating system programmatic procedures made from a process identified by the sender process identifier;
• instructions executable to generate an invocation hash based on at least a portion of the message;
• instructions executable to translate the invocation hash to an invocation identifier;
• instructions executable to include the invocation identifier in a translated call sequence that comprises invocation identifiers for the series of invocations;
• instructions executable to determine whether the translated call sequence is included in a plurality of predetermined call sequences, each of the predetermined call sequences comprising corresponding invocation identifiers, wherein each of the corresponding invocation identifiers are mapped to invocation hashes;
• and
• instructions executable to identify the translated call sequence as an anomaly if the translated call sequence is not included in the predetermined call sequences and, otherwise, to determine the translated call sequence is not an anomaly.

A system to detect a call sequence anomaly in a message-based operating system, the system comprising:

• a processor configured to:
• receive a message indicating an invocation of a programmatic procedure of an operating system, the message including a programmatic procedure identifier, a sender process identifier, and a receiver process identifier, wherein the invocation of the programmatic procedure is one invocation in a series of invocations of operating system programmatic procedures made from a process identified by the sender process identifier;
• generate an invocation hash based on at least a portion of the message;
• translate the invocation hash to an invocation identifier;
• include the invocation identifier in a translated call sequence that comprises invocation identifiers for the series of invocations;
• determine whether the translated call sequence is included in a plurality of predetermined call sequences, each of the predetermined call sequences comprising corresponding invocation identifiers, wherein each of the corresponding invocation identifiers are mapped to invocation hashes;
• and
• identify the translated call sequence as an anomaly if the translated call sequence is not included in the predetermined call sequences and, otherwise, to determine the translated call sequence is not an anomaly.