# US 10,120,997 B2 · Code instrumentation for runtime application self-protection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,997 B2 - Document ID: US10120997B2 - Status: Granted - Kind: B2 - Application number: 15/535,732 - Filed: 2015-12-24 - Published: 2018-04-19 - Granted: 2018-11-06 - Earliest claimed: 2015-01-01 - Assignees: Checkmarx Ltd. - Applicants: Checkmarx Ltd. - Inventors: Shimon Eshkenazi; Maty Siman; Alexander Roichman - Web page: https://patentgrep.com/patent/US10120997B2 ## CPC - G06F 21/51: at application loading time, e.g. accepting, rejecting, starting or inhibiting executable software based on integrity or source reliability - G06F 8/75: Structural analysis for program understanding - G06F 9/328: {for runtime instruction patching} - G06F 9/44568: {Immediately runnable code} - G06F 21/52: during program execution, e.g. stack integrity {; Preventing unwanted data erasure; Buffer overflow} - G06F 21/12: Protecting executable software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for runtime self-protection of an application program includes, before running the application program, identifying input and output points in runtime code (24) of the program. The input points are instrumented so as to cause the program to sense and cache potentially malicious inputs to the program. The output points are instrumented so as to cause the program to detect outputs from the program corresponding to the cached inputs. While running the application program, upon detecting, at an instrumented output point, an output corresponding to a cached input, a vulnerability of a target of the output to the cached input is evaluated. A protective action is invoked upon determining that the output is potentially vulnerable to the cached input. ## Claims ### Claim 1 A method for runtime self-protection of an application program, comprising: before running the application program: identifying input and output points in runtime code of the program; instrumenting the input points so as to cause the program to sense and cache potentially malicious inputs to the program; and instrumenting the output points so as to cause the program to detect outputs from the program corresponding to the cached inputs; and while running the application program: upon detecting, at an instrumented output point, an output corresponding to a cached input, evaluating a vulnerability of a target of the output to the cached input; and invoking a protective action upon determining that the output is potentially vulnerable to the cached input. ### Claim 2 (depends on 1) The method according to claim 1, wherein instrumenting the input points comprises adding a sensor routine to each identified input point, wherein the sensor routine examines the input for syntax that is characteristic of an attack pattern. ### Claim 3 (depends on 2) The method according to claim 2, wherein the attack pattern is selected from a set of attack patterns consisting of SQL injection, cross-site scripting (XSS), file path manipulation, and JavaScript Object Notation (JSON) injection. ### Claim 4 (depends on 1) The method according to claim 1, wherein identifying the output points comprises identifying instructions submitted by the application program to a predefined set of targets. ### Claim 5 (depends on 4) The method according to claim 4, wherein the set of targets comprises at least one target type selected from a group of target types consisting of databases, file systems and markup language outputs of Web applications. ### Claim 6 (depends on 1) The method according to claim 1, wherein identifying the input and output points comprises applying a static analysis to source code of the program in order to identify one or more of the input and output points. ### Claim 7 (depends on 1) The method according to claim 1, wherein identifying the input points comprises identifying an encoded input to the application program, and wherein the method comprises identifying and instrumenting a decoding point in the runtime code, at which the encoded input is decoded, so as to cause the program to cache the decoded input for detection of the outputs corresponding thereto. ### Claim 8 (depends on 1) The method according to claim 1, wherein instrumenting the input and output points comprises replacing input and output methods in the runtime code with a general method RASP proxy, which wraps the methods. ### Claim 9 (depends on 1) The method according to claim 1, wherein detecting the output comprises finding an approximate match between the cached input and data contained in the output. ### Claim 10 (depends on 1) The method according to claim 1, wherein detecting the output comprises detecting a file-path parameter in the cached input, and wherein evaluating the vulnerability comprises converting the file-path parameter to a canonical form, and identifying a file system vulnerability when the canonical form does not match a file-path prefix of the output. ### Claim 11 (depends on 1) The method according to claim 1, wherein evaluating the vulnerability comprises parsing the output into a document object model (DOM), and examining an effect of the cached input on the DOM. ### Claim 12 (depends on 11) The method according to claim 11, wherein examining the effect comprises determining that the output is potentially vulnerable when the cached input extends over more than a single leaf of the DOM. ### Claim 13 (depends on 11) The method according to claim 11, wherein examining the effect comprises detecting an executable script in the DOM that contains the cached input. ### Claim 14 (depends on 1) The method according to claim 1, and comprising, upon determining that the output is not vulnerable to the cached input, taking no protective action. ### Claim 15 (depends on 1) The method according to claim 1, wherein invoking the protective action comprises sanitizing the cached input before permitting the output to proceed. ### Claim 16 (depends on 1) The method according to claim 1, wherein invoking the protective action comprises reporting the vulnerability to a user together with a stack trace associated with at least one of the cached input and the detected output. ### Claim 17 (depends on 1) The method according to claim 1, wherein invoking the protective action comprises identifying a portion of the source code corresponding to the output that was determined to be potentially vulnerable. ### Claim 18 (depends on 1) The method according to claim 1, wherein invoking the protective action comprises automatically generating rules for implementation by a Web Application Firewall (WAF). ### Claim 19 Apparatus for runtime self-protection of an application program, comprising: a memory, which is configured to receive runtime code of the program; and a processor, which is configured to identify, before the program is run, input and output points in the runtime code, to instrument the input points so as to cause the program, while the program is running, to sense and cache potentially malicious inputs to the program, and to instrument the output points with instrumentation that causes the program, while the program is running, to detect outputs from the program corresponding to the cached inputs, and upon detecting an output corresponding to a cached input, to evaluate a vulnerability of a target of the output to the cached input and to invoke protective action upon determining that the output is potentially vulnerable to the cached input. ### Claim 20 (depends on 19) The apparatus according to claim 19, wherein the processor is configured to instrument the input points by adding a sensor routine to each identified input point, wherein the sensor routine examines the input for syntax that is characteristic of an attack pattern. ### Claim 21 (depends on 20) The apparatus according to claim 20, wherein the attack pattern is selected from a set of attack patterns consisting of SQL injection, cross-site scripting (XSS), file path manipulation, and JavaScript Object Notation (JSON) injection. ### Claim 22 (depends on 19) The apparatus according to claim 19, wherein the identified output points comprises instructions submitted by the application program to a predefined set of targets. ### Claim 23 (depends on 22) The apparatus according to claim 22, wherein the set of targets comprises at least one target type selected from a group of target types consisting of databases, file systems and markup language outputs of Web applications. ### Claim 24 (depends on 19) The apparatus according to claim 19, wherein the processor is configured to apply a static analysis to source code of the program in order to identify one or more of the input and output points. ### Claim 25 (depends on 19) The apparatus according to claim 19, wherein the processor is configured to identify an encoded input to the application program, and to identify and instrument a decoding point in the runtime code, at which the encoded input is decoded, so as to cause the program to cache the decoded input for detection of the outputs corresponding thereto. ### Claim 26 (depends on 19) The apparatus according to claim 19, wherein the processor is configured to replace input and output methods in the runtime code with a general method RASP proxy, which wraps the methods. ### Claim 27 (depends on 19) The apparatus according to claim 19, wherein the instrumentation causes the program to find an approximate match between the cached input and data contained in the output. ### Claim 28 (depends on 19) The apparatus according to claim 19, wherein the instrumentation causes the program to detect a file-path parameter in the cached input, to convert the file-path parameter to a canonical form, and to identify a file system vulnerability when the canonical form does not match a file-path prefix of the output. ### Claim 29 (depends on 19) The apparatus according to claim 19, wherein the instrumentation causes the program to parse the output into a document object model (DOM), and to examine an effect of the cached input on the DOM. ### Claim 30 (depends on 29) The apparatus according to claim 29, wherein the instrumentation causes the program to determine that the output is potentially vulnerable when the cached input extends over more than a single leaf of the DOM. ### Claim 31 (depends on 29) The apparatus according to claim 29, wherein the instrumentation causes the program to detect an executable script in the DOM that contains the cached input. ### Claim 32 (depends on 19) The apparatus according to claim 19, wherein the instrumentation causes the program to take no protective action upon determining that the output is not vulnerable to the cached input. ### Claim 33 (depends on 19) The apparatus according to claim 19, wherein the protective action comprises sanitizing the cached input before permitting the output to proceed. ### Claim 34 (depends on 19) The apparatus according to claim 19, wherein the protective action comprises reporting the vulnerability to a user together with a stack trace associated with at least one of the cached input and the detected output. ### Claim 35 (depends on 19) The apparatus according to claim 19, wherein the protective action comprises identifying a portion of the source code corresponding to the output that was determined to be potentially vulnerable. ### Claim 36 (depends on 19) The apparatus according to claim 19, wherein the protective action comprises automatically generating rules for implementation by a Web Application Firewall (WAF). ### Claim 37 A computer software product for runtime self-protection of an application program, comprising a non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive runtime code of the program, and to identify, before the program is run, input and output points in the runtime code, and to instrument the input points so as to cause the program, while the program is running, to sense and cache potentially malicious inputs to the program, and to instrument the output points with instrumentation that causes the program, while the program is running, to detect outputs from the program corresponding to the cached inputs, and causes the program, upon detecting an output corresponding to a cached input, to evaluate a vulnerability of a target of the output to the cached input and to invoke protective action upon determining that the output is potentially vulnerable to the cached input.