# US 11,113,398 B2 · Deployment of machine learning models for discernment of threats > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,398 B2 - Document ID: US11113398B2 - Status: Granted - Kind: B2 - Application number: 16/813,529 - Filed: 2020-03-09 - Published: 2020-07-02 - Granted: 2021-09-07 - Earliest claimed: 2016-06-08 - Assignees: Cylance Inc. - Applicants: Cylance Inc. - Inventors: Kristopher William Harms; Renee Song; Raj Rajamani; Braden Rusell; Yoojin Sohn; Kiefer Ipsen - Web page: https://patentgrep.com/patent/US11113398B2 ## CPC - G06F 21/568: {eliminating virus, restoring damaged files} - G06F 21/51: at application loading time, e.g. accepting, rejecting, starting or inhibiting executable software based on integrity or source reliability - G06F 21/562: {Static detection} - G06F 21/566: {Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities} - G06N 20/00: Machine learning - H04L 63/1425: {Traffic logging, e.g. anomaly detection} - H04L 63/1441: {Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002)} - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G06F 2221/033: Test or assess software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A mismatch between model-based classifications produced by a first version of a machine learning threat discernment model and a second version of a machine learning threat discernment model for a file is detected. The mismatch is analyzed to determine appropriate handling for the file, and taking an action based on the analyzing. The analyzing includes comparing a human-generated classification status for a file, a first model version status that reflects classification by the first version of the machine learning threat discernment model, and a second model version status that reflects classification by the second version of the machine learning threat discernment model. The analyzing can also include allowing the human-generated classification status to dominate when it is available. ## Claims ### Claim 1 A method for implementation by one or more data processors forming part of at least one computing device, the method comprising: detecting, by at least one data processor, a mismatch between model-based classifications produced by a first version of a computer-implemented machine learning threat discernment model and a second version of a computer-implemented machine learning threat discernment model for a file, each of the first version of the machine learning threat discernment model and the second version of the machine learning threat discernment model for a file output a respective threat score, wherein the mismatch is based on a difference between the threat scores; executing or accessing, by at least one data processor, the file in response to the difference between the threat scores being below a pre-defined threshold; and analyzing, by at least one data processor, the mismatch to determine appropriate handling for the file in response to the threat score being equal to or above a pre-defined threshold including causing information relating to the mismatch to be displayed in a graphical user interface of an electronic visual display. ### Claim 2 (depends on 1) The method of claim 1 wherein: the analyzing comprises comparing a human-generated classification for a file, a first model version status that reflects classification by the first version of the machine learning threat discernment model, and a second model version status that reflects classification by the second version of the machine learning threat discernment model, the analyzing further comprises allowing the human-generated classification to dominate when it is available; and the method further comprises: taking, by at least one data processor, an action based on the analyzing. ### Claim 3 (depends on 2) The method of claim 2, wherein the action comprises: presenting, by at least one data processor, the file via user interface functionality of an administrative interface with a choice of creating a new human-generated classification of the file as safe or allowing the second model version classification to govern when the file is deemed safe by the first model version but unsafe by the second model version, and when the human-generated classification is unavailable for the file; quarantining, by at least one data processor, a file deemed unsafe by the first version of the machine learning threat discernment model but safe by the second version of the machine learning threat discernment model when the human-generated classification is unavailable for the file; and presenting, by at least one data processor, the file via the user interface functionality of the administrative interface with a second choice of designating the file for local analysis and/or allowing the second model version classification to continue to block use of the file until the human-generated classification becomes available when the file is deemed unsafe by the second model version and the human human-generated classification is unavailable and when the first model version has not previously classified the file. ### Claim 4 (depends on 2) The method of claim 2, wherein the action comprises: presenting, by at least one data processor, the file via user interface functionality of an administrative interface with a choice of creating a new human-generated classification of the file as safe or allowing the second model version classification to govern when the file is deemed safe by the first model version but unsafe by the second model version, and when the human-generated classification is unavailable for the file. ### Claim 5 (depends on 2) The method of claim 2, wherein the action comprises: quarantining, by at least one data processor, a file deemed unsafe by the first version of the machine learning threat discernment model but safe by the second version of the machine learning threat discernment model when the human-generated classification is unavailable for the file. ### Claim 6 (depends on 2) The method of claim 2, wherein the action comprises: presenting, by at least one data processor, the file via user interface functionality of an administrative interface with a second choice of designating the file for local analysis and/or allowing the second model version classification to continue to block use of the file until the human-generated classification becomes available when the file is deemed unsafe by the second model version and the human human-generated classification is unavailable and when the first model version has not previously classified the file. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the first version of a computer-implemented machine learning threat discernment model and the second version of a computer-implemented machine learning threat discernment model for the file are on different and remote computing systems. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the first version of a computer-implemented machine learning threat discernment model and the second version of a computer-implemented machine learning threat discernment model for the file are on the same computing system. ### Claim 9 A system comprising: at least one data processor; and memory storing instructions which, when executed by the at least one data processor, result in operations comprising: detecting a mismatch between model-based classifications produced by a first version of a computer-implemented machine learning threat discernment model and a second version of a computer-implemented machine learning threat discernment model for a file, each of the first version of the machine learning threat discernment model and the second version of the machine learning threat discernment model for a file output a respective threat score, wherein the mismatch is based on a difference between the threat scores; executing or accessing the file in response to the difference between the threat scores being below a pre-defined threshold; and analyzing the mismatch to determine appropriate handling for the file in response to the threat score being equal to or above a pre-defined threshold including causing information relating to the mismatch to be displayed in a graphical user interface of an electronic visual display. ### Claim 10 (depends on 9) The system of claim 9 wherein: the analyzing comprises comparing a human-generated classification for a file, a first model version status that reflects classification by the first version of the machine learning threat discernment model, and a second model version status that reflects classification by the second version of the machine learning threat discernment model, the analyzing further comprises allowing the human-generated classification to dominate when it is available; and the operations further comprise: taking an action based on the analyzing. ### Claim 11 (depends on 10) The system of claim 10, wherein the action comprises: presenting the file via user interface functionality of an administrative interface with a choice of creating a new human-generated classification of the file as safe or allowing the second model version classification to govern when the file is deemed safe by the first model version but unsafe by the second model version, and when the human-generated classification is unavailable for the file; quarantining a file deemed unsafe by the first version of the machine learning threat discernment model but safe by the second version of the machine learning threat discernment model when the human-generated classification is unavailable for the file; and presenting the file via the user interface functionality of the administrative interface with a second choice of designating the file for local analysis and/or allowing the second model version classification to continue to block use of the file until the human-generated classification becomes available when the file is deemed unsafe by the second model version and the human human-generated classification is unavailable and when the first model version has not previously classified the file. ### Claim 12 (depends on 10) The system of claim 10, wherein the action comprises: presenting the file via user interface functionality of an administrative interface with a choice of creating a new human-generated classification of the file as safe or allowing the second model version classification to govern when the file is deemed safe by the first model version but unsafe by the second model version, and when the human-generated classification is unavailable for the file. ### Claim 13 (depends on 10) The system of claim 10, wherein the action comprises: quarantining a file deemed unsafe by the first version of the machine learning threat discernment model but safe by the second version of the machine learning threat discernment model when the human-generated classification is unavailable for the file. ### Claim 14 (depends on 10) The system of claim 10, wherein the action comprises: presenting the file via user interface functionality of an administrative interface with a second choice of designating the file for local analysis and/or allowing the second model version classification to continue to block use of the file until the human-generated classification becomes available when the file is deemed unsafe by the second model version and the human human-generated classification is unavailable and when the first model version has not previously classified the file. ### Claim 15 (depends on 10) The system of claim 10, wherein the first version of a computer-implemented machine learning threat discernment model and the second version of a computer-implemented machine learning threat discernment model for the file are on different and remote computing systems. ### Claim 16 (depends on 10) The system of claim 10, wherein the first version of a computer-implemented machine learning threat discernment model and the second version of a computer-implemented machine learning threat discernment model for the file are on the same computing system. ### Claim 17 A computer program product comprising non-transitory media storing instructions which, when executed by the at least one data processor, result in operations comprising: detecting a mismatch between model-based classifications produced by a first version of a computer-implemented machine learning threat discernment model and a second version of a computer-implemented machine learning threat discernment model for a file, each of the first version of the machine learning threat discernment model and the second version of the machine learning threat discernment model for a file output a respective threat score, wherein the mismatch is based on a difference between the threat scores; executing or accessing the file in response to the difference between the threat scores being below a pre-defined threshold; and analyzing the mismatch to determine appropriate handling for the file in response to the threat score being equal to or above a pre-defined threshold including causing information relating to the mismatch to be displayed in a graphical user interface of an electronic visual display. ### Claim 18 (depends on 17) The computer program product of claim 17 wherein: the analyzing comprises comparing a human-generated classification for a file, a first model version status that reflects classification by the first version of the machine learning threat discernment model, and a second model version status that reflects classification by the second version of the machine learning threat discernment model, the analyzing further comprises allowing the human-generated classification to dominate when it is available; and the operations further comprise: taking an action based on the analyzing. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein the action comprises: presenting the file via user interface functionality of an administrative interface with a choice of creating a new human-generated classification of the file as safe or allowing the second model version classification to govern when the file is deemed safe by the first model version but unsafe by the second model version, and when the human-generated classification is unavailable for the file; quarantining a file deemed unsafe by the first version of the machine learning threat discernment model but safe by the second version of the machine learning threat discernment model when the human-generated classification is unavailable for the file; and presenting the file via the user interface functionality of the administrative interface with a second choice of designating the file for local analysis and/or allowing the second model version classification to continue to block use of the file until the human-generated classification becomes available when the file is deemed unsafe by the second model version and the human human-generated classification is unavailable and when the first model version has not previously classified the file. ### Claim 20 The computer program product of 18, wherein the action comprises: presenting the file via user interface functionality of an administrative interface with a choice of creating a new human-generated classification of the file as safe or allowing the second model version classification to govern when the file is deemed safe by the first model version but unsafe by the second model version, and when the human-generated classification is unavailable for the file.