System, method and computer program product for detecting encoded shellcode in network traffic

A computer program product embodied on a tangible computer readable medium, comprising:

• computer code for identifying network traffic that is encoded;
• computer code for determining a type of encoding associated with the network traffic;
• computer code for converting the network traffic that is encoded;
• and
• computer code for determining whether the network traffic that is encoded at least potentially includes shellcode;
• wherein the computer program product is operable such that the determination whether the network traffic includes the shellcode is conditionally performed based on a determination whether the network traffic comprises machine language instructions encoded as text.

A system, comprising:

• a client device for receiving encoded network traffic, comprising a processor;
• and
• a security application installed on the client device, the security application for determining a type of encoding associated with the network traffic, converting the network traffic that is encoded, and determining whether the encoded network traffic includes shellcode;
• wherein the security application is operable such that the determination whether the network traffic includes the shellcode is conditionally performed based on a determination whether the network traffic comprises machine language instructions encoded as text.