# US 10,484,399 B1 · Systems and methods for detecting low-density training regions of machine-learning classification systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,399 B1
- Document ID: US10484399B1
- Status: Granted
- Kind: B1
- Application number: 15/434,086
- Filed: 2017-02-16
- Granted: 2019-11-19
- Earliest claimed: 2017-02-16
- Assignees: Symantec Corporation
- Applicants: Symantec Corporation
- Inventors: Ryan Curtin
- Web page: https://patentgrep.com/patent/US10484399B1

## CPC

- H04L 63/1408: {by monitoring network traffic (monitoring network traffic per se H04L43/00)}
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 20/00: Machine learning
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The disclosed computer-implemented method for detecting low-density training regions of machine-learning classification systems may include (i) receiving a training dataset that is used to train a classifier of a machine-learning classification system, (ii) calculating a density estimate of a distribution of the training dataset, (iii) receiving a sample that is to be classified by the classifier, (iv) using the density estimate to determine that the sample falls within a low-density region of the distribution of the training dataset, and (v) performing a security action in response to determining that the sample falls within the low-density region. Various other methods, systems, and computer-readable media are also disclosed.

## Claims

### Claim 1

A computer-implemented method for detecting low-density training regions of machine-learning classification systems, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
receiving a training dataset;
training, using the training dataset, a classifier of a machine-learning classification system;
calculating, using the training dataset, a density-estimation model comprising a density estimate of a distribution of the training dataset;
associating the density-estimation model with the classifier;
receiving a sample that is to be classified;
using, before classifying the sample using the classifier, the density-estimation model to determine that the sample falls within a low-density region of the distribution of the training dataset;
determining, based at least in part on the sample falling within the low-density region, that the classifier will not accurately classify the sample; and
performing a security action in response to determining that the classifier will not accurately classify the sample.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
calculating the density-estimation model comprises generating a density estimation tree using the training dataset; and
using the density-estimation model to determine that the sample falls within the low-density region of the distribution of the training dataset comprises:
determining that the sample maps to a node in the density estimation tree; and
determining that a density represented by the node is below a predetermined threshold.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein performing the security action comprises refraining from classifying the sample with the classifier.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising:
receiving an additional training dataset;
training, using the additional training dataset, an additional classifier of the machine-learning classification system;
calculating, using the additional training dataset, an additional density-estimation model comprising an additional density estimate of a distribution of the additional training dataset;
associating the additional density-estimation model with the additional classifier; and
using the additional density-estimation model to determine that the sample falls within a high-density region of the distribution of the additional training dataset, wherein performing the security action comprises using, based at least in part on determining that the sample falls within the low-density region and the sample falls within the high-density region, the additional classifier rather than the classifier to determine a classification of the sample.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein performing the security action comprises:
using the classifier to determine a classification of the sample; and
associating a lower confidence level to the classification.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein performing the security action comprises assigning a default classification to the sample without using the classifier in order to avoid an assignment of a false-positive classification to the sample by the classifier.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the density-estimation model comprises a density estimation tree.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the training dataset comprises at least one of:
malicious applications; and
benign applications.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the training dataset comprises at least one of:
malicious webpages; and
benign webpages.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the training dataset comprises at least one of:
malicious emails; and
benign emails.

### Claim 11

A system for detecting low-density training regions of machine-learning classification systems, the system comprising:
a receiving module, stored in memory, that receives:
a training dataset; and
a sample that is to be classified;
a training module, stored in memory, that trains, using the training dataset, a classifier;
an estimating module, stored in memory, that calculates, using the training dataset, a density-estimation model comprising a density estimate of a distribution of the training dataset;
a determining module, stored in memory, that uses the density-estimation model to determine that the sample falls within a low-density region of the distribution of the training dataset;
a security module, stored in memory, that:
determines, based at least in part on the sample falling within the low-density region, that the classifier will not accurately classify the sample; and
performs a security action in response to determining that the classifier will not accurately classify the sample; and
at least one physical processor that executes the receiving module, the training module, the estimating module, the determining module, and the security module.

### Claim 12 (depends on 11)

The system of claim 11, wherein:
the estimating module calculates the density-estimation model by generating a density estimation tree using the training dataset; and
the determining module uses the density-estimation model to determine that the sample falls within the low-density region of the distribution of the training dataset by:
determining that the sample maps to a node in the density estimation tree; and
determining that a density represented by the node is below a predetermined threshold.

### Claim 13 (depends on 11)

The system of claim 11, wherein
the security module performs the security action by refraining from classifying the sample with the classifier.

### Claim 14 (depends on 13)

The system of claim 13, wherein:
the receiving module further receives an additional training dataset;
the training module further trains, using the additional training dataset, an additional classifier;
the estimating module further calculates, using the additional training dataset, an additional density-estimation mode comprising an additional density estimate of a distribution of the additional training dataset;
the determining module further uses the additional density-estimation model to determine that the sample falls within a high-density region of the distribution of the additional training dataset; and
the security module performs the security action by using, based at least in part on the sample falling within the low-density region and the sample falling within the high-density region, the additional classifier rather than the classifier to determine a classification of the sample.

### Claim 15 (depends on 11)

The system of claim 11, wherein
the security module performs the security action by:
using the classifier to determine a classification of the sample; and
associating a lower confidence level to the classification.

### Claim 16 (depends on 11)

The system of claim 11, wherein the security module performs the security action by assigning a default classification to the sample without using the classifier in order to avoid an assignment of a false-positive classification to the sample by the classifier.

### Claim 17 (depends on 11)

The system of claim 11, wherein the density-estimation model comprises a density estimation tree.

### Claim 18 (depends on 11)

The system of claim 11, wherein the training dataset comprises at least one of:
malicious applications; and
benign applications.

### Claim 19 (depends on 11)

The system of claim 11, wherein the training dataset comprises at least one of:
malicious webpages; and
benign webpages.

### Claim 20

A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
receive a training dataset;
train, using the training dataset, a classifier of a machine-learning classification system;
calculate, using the training dataset, a density-estimation model comprising a density estimate of a distribution of the training dataset;
associate the density-estimation model with the classifier;
receive a sample that is to be classified;
use, before classifying the sample using the classifier, the density-estimation model to determine that the sample falls within a low-density region of the distribution of the training dataset;
determine, based at least in part on the sample falling within the low-density region, that the classifier will not accurately classify the sample; and
perform a security action in response to determining that the classifier will not accurately classify the sample.