Shadow stack violation enforcement at module granularity

A method, implemented at a computer system that includes a processor, for enforcing a shadow stack violation at module granularity, the method comprising:

• initiating execution of a thread at the processor, including initiating execution of first executable code of an application binary that calls second executable code of an external module, the application binary having been enabled for shadow stack enforcement;
• and
• based at least on execution of the thread at the processor, processing an exception triggered by a mismatch between a first return address popped from a call stack corresponding to the thread and a second return address popped from a shadow stack corresponding to the thread, including:
• determining that the exception resulted from execution of an instruction in the second executable code of the external module;
• determining whether or not the external module is enabled for shadow stack enforcement;
• and
• based on having determined whether the external module is enabled for shadow stack enforcement, perform one of:
• terminating execution of the thread based at least on the external module being enabled for shadow stack enforcement;
• or
• permitting the thread to continue executing based at least on the external module not being enabled for shadow stack enforcement.

A computer system comprising:

• a processor;
• and
• a computer-readable medium having stored thereon computer-executable instructions that are executable by the processor to cause the computer system to enforce a shadow stack violation at module granularity, the computer-executable instructions including instructions that are executable by the processor to cause the computer system to perform at least the following:
• initiate execution of a thread at the processor, including initiating execution of first executable code of an application binary that calls second executable code of an external module, the application binary having been enabled for shadow stack enforcement;
• and
• based at least on execution of the thread at the processor, process an exception triggered by a mismatch between a first return address popped from a call stack corresponding to the thread and a second return address popped from a shadow stack corresponding to the thread, including:
• determining that the exception resulted from execution of an instruction in the second executable code of the external module;
• determining whether or not the external module is enabled for shadow stack enforcement;
• identifying a rule based at least on an identity of the external module;
• and
• based on having determined whether the external module is enabled for shadow stack enforcement, perform one of:
• terminating execution of the thread based at least on the external module being enabled for shadow stack enforcement;
• terminating execution of the thread based at least on the external module being not being enabled for shadow stack enforcement, and on the rule specifying that a shadow stack violation should not be permitted;
• or
• permitting the thread to continue executing based at least on the external module not being enabled for shadow stack enforcement, and on the rule specifying that a shadow stack violation should be permitted.