# US 2020/0401941 A1 · System and Method for Machine Learning Model Determination and Malware Identification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401941 A1 - Document ID: US20200401941A1 - Status: Published application - Kind: A1 - Application number: 16/916,049 - Filed: 2020-06-29 - Published: 2020-12-24 - Earliest claimed: 2015-07-31 - Assignees: BluVector, Inc. - Applicants: Bluvector, Inc. - Inventors: Scott B. Miserendino; Robert H. Klein; Ryan V. Peters; Peter E. Kaloroumakis - Web page: https://patentgrep.com/patent/US20200401941A1 ## CPC - G06N 20/00: Machine learning - G06F 21/56: Computer malware detection or handling, e.g. anti-virus arrangements - G06F 21/566: {Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities} - G06N 5/04: Inference or reasoning models - H04W 12/128: Anti-malware arrangements, e.g. protection against SMS fraud or mobile malware - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 2221/033: Test or assess software - G06F 2221/2107: File encryption - G06F 2221/2115: Third party - H04L 63/1433: {Vulnerability analysis} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system and method for batched, supervised, in-situ machine learning classifier retraining for malware identification and model heterogeneity. The method produces a parent classifier model in one location and providing it to one or more in-situ retraining system or systems in a different location or locations, adjudicates the class determination of the parent classifier over the plurality of the samples evaluated by the in-situ retraining system or systems, determines a minimum number of adjudicated samples required to initiate the in-situ retraining process, creates a new training and test set using samples from one or more in-situ systems, blends a feature vector representation of the in-situ training and test sets with a feature vector representation of the parent training and test sets, conducts machine learning over the blended training set, evaluates the new and parent models using the blended test set and additional unlabeled samples, and elects whether to replace the parent classifier with the retrained version. ## Claims ### Claim 1 A method comprising: receiving information indicating a first plurality of attributes associated with a first plurality of files for machine learning training; determining, based on the information, a plurality of classifications for a second plurality of files; based on adjudicating one or more of the plurality of classifications, determining a second plurality of attributes associated with the second plurality of files; determining, based on training and using at least a portion of the first plurality of attributes and at least a portion of the second plurality of attributes, a machine learning model; and determining, based on the machine learning model, that a file comprises malicious content. ### Claim 2 (depends on 1) The method of claim 1, wherein the first plurality of attributes was extracted from the first plurality of files, wherein the first plurality of attributes comprises at least one of a file header property, a component of a file, or a binary sequence, and wherein the second plurality of attributes was extracted from the second plurality of files, wherein the second plurality of attributes comprises at least one of a file header property, a component of a file, or a binary sequence. ### Claim 3 (depends on 1) The method of claim 1, wherein the training is based on a machine learning algorithm, and wherein the machine learning algorithm comprises a decision tree, support vector machine, a k-nearest neighbor algorithm, an artificial neural network, or a Bayesian network. ### Claim 4 (depends on 1) The method of claim 1, wherein each classification of the plurality of classifications comprises at least one of malicious content or benign content. ### Claim 5 (depends on 1) The method of claim 1, wherein the adjudicating comprises: confirming classifications of the plurality of classifications that are correct; and adjusting classifications of the plurality of classifications that are not correct. ### Claim 6 (depends on 1) The method of claim 1, wherein the adjudicating indicates an accuracy of the machine learning model at determining the plurality of classifications and relevant attributes for determining the plurality of classifications. ### Claim 7 (depends on 1) The method of claim 1, wherein the training is triggered based on adjudicating a threshold number of classifications of the plurality of classifications. ### Claim 8 (depends on 1) The method of claim 1, wherein the second plurality of files are unique to an organization. ### Claim 9 A device comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the device to: receive information indicating a first plurality of attributes associated with a first plurality of files for machine learning training; determine, based on the information, a plurality of classifications for a second plurality of files; based on adjudicating one or more of the plurality of classifications, determine a second plurality of attributes associated with the second plurality of files; determine, based on training and using at least a portion of the first plurality of attributes and at least a portion of the second plurality of attributes, a machine learning model; and determine, based on the machine learning model, that a file comprises malicious content. ### Claim 10 (depends on 9) The device of claim 9, wherein the first plurality of attributes was extracted from the first plurality of files, wherein the first plurality of attributes comprises at least one of a file header property, a component of a file, or a binary sequence, and wherein the second plurality of attributes was extracted from the second plurality of files, wherein the second plurality of attributes comprises at least one of a file header property, a component of a file, or a binary sequence. ### Claim 11 (depends on 9) The device of claim 9, wherein the training is based on a machine learning algorithm, and wherein the machine learning algorithm comprises a decision tree, support vector machine, a k-nearest neighbor algorithm, an artificial neural network, or a Bayesian network. ### Claim 12 (depends on 9) The device of claim 9, wherein each classification of the plurality of classifications comprises at least one of malicious content or benign content. ### Claim 13 (depends on 9) The device of claim 9, wherein the adjudicating comprises: confirming classifications of the plurality of classifications that are correct; and adjusting classifications of the plurality of classifications that are not correct. ### Claim 14 (depends on 9) The device of claim 9, wherein the adjudicating indicates an accuracy of the machine learning model at determining the plurality of classifications and relevant attributes for determining the plurality of classifications. ### Claim 15 (depends on 9) The device of claim 9, wherein the training is triggered based on adjudicating a threshold number of classifications of the plurality of classifications. ### Claim 16 (depends on 9) The device of claim 9, wherein the second plurality of files are unique to an organization. ### Claim 17 A non-transitory computer-readable medium storing instructions that, when executed, cause: receiving information indicating a first plurality of attributes associated with a first plurality of files for machine learning training; determining, based on the information, a plurality of classifications for a second plurality of files; based on adjudicating one or more of the plurality of classifications, determining a second plurality of attributes associated with the second plurality of files; determining, based on training and using at least a portion of the first plurality of attributes and at least a portion of the second plurality of attributes, a machine learning model; and determining, based on the machine learning model, that a file comprises malicious content. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the first plurality of attributes was extracted from the first plurality of files, wherein the first plurality of attributes comprises at least one of a file header property, a component of a file, or a binary sequence, and wherein the second plurality of attributes was extracted from the second plurality of files, wherein the second plurality of attributes comprises at least one of a file header property, a component of a file, or a binary sequence. ### Claim 19 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the adjudicating comprises: confirming classifications of the plurality of classifications that are correct; and adjusting classifications of the plurality of classifications that are not correct. ### Claim 20 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the adjudicating indicates an accuracy of the machine learning model at determining the plurality of classifications and relevant attributes for determining the plurality of classifications.