System and methods for advanced malware detection through placement of transition events

An electronic device comprising:

• one or more processors;
• a storage device communicatively coupled to the one or more processors and storing logic, the logic being executable by the one or more processors to perform operations including:
• instrumenting a virtual machine (VM) with a virtual machine monitor (VMM), the virtual machine to process an object to determine whether the object is associated with malware;
• performing a first analysis of memory allocated for the VM for a point of interest (POI), the point of interest being an address of an instruction of a set of instructions likely to be associated with malware, the set of instructions including one or more instructions;
• detecting a memory violation during processing of the object, the memory violation being an attempt to access a page in the memory allocated for the VM having a permission other than “execute” from which a process running within the VM is attempting to execute;
• responsive to detecting the memory violation, injecting a transition event at the point of interest on the page and setting the permission of the page to “execute only”;
• and
• responsive to continuing the processing of the object and detecting an attempted execution of the transition event, (i) emulating the instruction corresponding to the point of interest, and (ii) performing one or more malware detection routines.

A method for detecting whether an object is associated with malware through processing of the object within a virtual machine (VM), the method comprising:

• instrumenting the VM with a virtual machine monitor (VMM);
• performing a first analysis of memory allocated for the VM for a point of interest (POI), the point of interest being an address of an instruction of a set of instructions likely to be associated with malware, the set of instructions including one or more instructions;
• detecting a memory violation during processing of the object, the memory violation being an attempt to access a page in the memory allocated for a process within the VM having a permission other than “execute” from which a process running within the VM is attempting to execute;
• responsive to detecting the memory violation, injecting an transition event at the point of interest on the page and setting the permission of the page to “execute only”;
• and
• responsive to continuing the processing of the object and detecting an attempted execution of the transition event, (i) emulating the instruction corresponding to the point of interest, and (ii) performing one or more malware detection routines.

A method for detecting whether an object is associated with malware through processing of the object within a virtual machine (VM), the method comprising:

• performing a first analysis of memory allocated for the VM for one or more first points of interest (POIs), each point of interest being an address of an instruction of a set of instructions likely to be associated with malware, the set of instructions including one or more instructions;
• recording each of the one or more points of interest in one or more data structures in memory shared between the VM and a virtual machine monitor (VMM);
• and
• performing a second analysis of at least a portion of the memory allocated for a process within the VM for one or more second points of interest, the second analysis being performed in response to any of the following:
• (i) a new application starts within the VM, (ii) a running application within the VM terminates, or (iii) an executable module is loaded or unloaded for a running application within the VM, and each of the one or more second points of interest are recorded in the one or more data structures;
• injecting, by a virtual machine monitor introspection logic (VMMI), a transition event into a page of the memory allocated for the VM upon referencing the one or more data structures and setting a permission of the page to “execute only” in response to detection of a memory violation during processing of the object;
• and
• responsive to the processing the object and detecting an attempted execution of the transition event, (i) emulating the instruction corresponding to a point of interest of either the first points of interest or the second points of interest, and (ii) performing one or more malware detection routines.