# US 9,519,779 B2 · Methods and apparatus for control and detection of malicious content using a sandbox environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,779 B2 - Document ID: US9519779B2 - Status: Granted - Kind: B2 - Application number: 14/797,847 - Filed: 2015-07-13 - Published: 2015-11-12 - Granted: 2016-12-13 - Earliest claimed: 2011-12-02 - Assignees: Invincea, Inc. - Applicants: Invincea, Inc. - Inventors: Anup Ghosh; Scott Cosby; Alan Keister; Benjamin Bryant; Stephen Taylor - Web page: https://patentgrep.com/patent/US9519779B2 ## CPC - G06F 21/53: by executing in a restricted environment, e.g. sandbox or secure virtual machine - G06F 21/56: Computer malware detection or handling, e.g. anti-virus arrangements - G06F 21/566: {Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities} - G06F 2221/034: Test or assess a computer or a system - G06F 2221/2101: Auditing as a secondary aspect - G06F 2221/2141: Access rights, e.g. capability lists, access control lists, access tables, access matrices - G06F 2221/2149: Restricted operating environment - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A non-transitory processor-readable medium storing code representing instructions to cause a processor to perform a process includes code to cause the processor to receive a set of indications of allowed behavior associated with an application. The processor is also caused to initiate an instance of the application within a sandbox environment. The processor is further caused to receive, from a monitor module associated with the sandbox environment, a set of indications of actual behavior of the instance of the application in response to initiating the instance of the application within the sandbox environment. The processor is also caused to send an indication associated with an anomalous behavior if at least one indication from the set of indications of actual behavior does not correspond to an indication from the set of indications of allowed behavior. ## Claims ### Claim 1 A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to: initiate an instance of a first application within a sandbox environment; receive a set of indications of actual behavior of the instance of the first application including an indication that the instance of the first application is associating with an instance of a second application within the sandbox environment by at least one of (1) initiating the instance of the second application or (2) initiating a thread injection event with the instance of the second application; identify the instance of the first application associating with the instance of the second application as an anomalous behavior of the instance of the first application in response to an indication that the first application is allowed to associate with the second application not being within a set of indications of allowed behavior specific to the first application, send an indication associated with the anomalous behavior in response to identifying the anomalous behavior. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the code further comprises code to cause the processor to: add to an evaluation tree a node representing the second application as a child of a node representing the first application based on the instance of the first application initiating the instance of the second application. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the code further comprises code to cause the processor to: add a representation of the anomalous behavior to an evaluation tree as an attribute of a node associated with the first application. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the indication associated with the anomalous behavior includes a trace associated with a source of the anomalous behavior. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the set of indications of allowed behavior of the first application is based at least in part on a trust level associated with the first application. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the code to cause the processor to initiate includes code to cause the processor to initiate the instance of the first application within the sandbox environment by sending a signal via a network to a compute device such that the compute device executes the instance of the first application within the sandbox environment. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the code to cause the processor to send the indication associated with the anomalous behavior includes code to cause the processor to send the indication associated with the anomalous behavior such that the sandbox environment is restarted in response to the indication associated with the anomalous behavior. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the code to cause the processor to send the indication associated with the anomalous behavior includes code to cause the processor to send the indication associated with the anomalous behavior such that at least one of the instance of the first application or the instance of the second application is terminated in response to the indication associated with the anomalous behavior. ### Claim 9 (depends on 1) The non-transitory processor-readable medium of claim 1, wherein the instance of the second application is a first instance of the second application, the code further comprising code to cause the processor to: initiate an instance of a third application within the sandbox environment; receive a set of indications of actual behavior of the instance of the third application including an indication that the instance of the third application is associating with a second instance of the second application within the sandbox environment; and identify the instance of the third application associating with the second instance of the second application as a non-anomalous behavior of the instance of the third application in response to an indication that the third application is allowed to associate with the second application being within a set of indications of allowed behavior specific to the third application. ### Claim 10 (depends on 1) The non-transitory processor-readable medium of claim 1, wherein the code to cause the processor to initiate includes code to cause the processor to initiate the instance of the first application in the sandbox environment based on the first application being recognized by the sandbox environment, the code further comprising code to cause the processor to: initiate an instance of a third application outside the sandbox environment based on the third application not being recognized by the sandbox environment. ### Claim 11 (depends on 1) The non-transitory processor-readable medium of claim 1, the code further comprising code to cause the processor to: automatically update the set of indications of allowed behavior specific to the first application to include the indication that the first application is allowed to associate with the second application based on the anomalous behavior being identified as a false positive. ### Claim 12 An apparatus, comprising: a memory; and a hardware processor operatively coupled to the memory, the hardware processor configured to receive a set of indications of actual behavior of an instance of a first application within a sandbox environment, the hardware processor configured to compare the set of indications of actual behavior of the instance of the first application to a set of indications of allowed behavior specific to the first application to identify a behavior from the set of indications of actual behavior not within the set of indications of allowed behavior, the hardware processor configured to classify the behavior from the set of indications of actual behavior as an anomalous behavior of the instance of the first application in response to the behavior not being within the set of indications of allowed behavior, the anomalous behavior including at least one of the instance of the first application initiating an instance of a second application or the instance of the first application initiating a thread injection event with the instance of the second application, the hardware processor configured to send an indication (1) associated with the anomalous behavior of the instance of the first application and (2) including an identifier of the second application. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the hardware processor is configured to send the indication such that the sandbox environment is restarted in response to the indication. ### Claim 14 (depends on 12) The apparatus of claim 12, wherein the set of indications of allowed behavior of the first application is based at least in part on a trust level associated with the first application. ### Claim 15 (depends on 12) The apparatus of claim 12, wherein the hardware processor is configured to add to an evaluation tree a node representing the second application as a child of a node representing the first application based on the instance of the first application initiating the instance of the second application. ### Claim 16 (depends on 12) The apparatus of claim 12, wherein the hardware processor is configured to add a representation of the anomalous behavior to an evaluation tree as an attribute of a node associated with the first application. ### Claim 17 (depends on 12) The apparatus of claim 12, wherein the hardware processor is configured to send the indication such that at least one of the instance of the first application or the instance of the second application is terminated in response to the indication. ### Claim 18 A method, comprising: initiating an instance of a first application within a sandbox environment; monitoring behavior of the instance of the first application based on a list of approved behaviors specific to the first application; identifying that the instance of the first application is associating with an instance of a second application within the sandbox environment by at least one of (1) initiating the instance of the second application or (2) initiating a thread injection event with the instance of the second application; classifying the instance of the first application associating with the instance of the second application as an anomalous behavior of the instance of the first application in response to an indication that the first application is allowed to associate with the second application not being on the list of approved behaviors specific to the first application; and sending an indication associated with the anomalous behavior. ### Claim 19 (depends on 18) The method of claim 18, further comprising: restarting the sandbox environment in response to the anomalous behavior. ### Claim 20 (depends on 18) The method of claim 18, further comprising: adding to an evaluation tree a node representing the second application as a child of a node representing the first application based on the instance of the first application initiating the instance of the second application. ### Claim 21 (depends on 18) The method of claim 18, further comprising: adding a representation of the anomalous behavior to an evaluation tree as an attribute of a node associated with the first application. ### Claim 22 (depends on 18) The method of claim 18, wherein the indication associated with the anomalous behavior includes a trace associated with a source of the anomalous behavior. ### Claim 23 (depends on 18) The method of claim 18, further comprising: terminating at least one of the instance of the first application or the instance of the second application in response to the anomalous behavior.