# US 2016/0253498 A1 · Methods and Systems for On-Device High-Granularity Classification of Device Behaviors using Multi-Label Models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0253498 A1
- Document ID: US20160253498A1
- Status: Published application
- Kind: A1
- Application number: 14/837,936
- Filed: 2015-08-27
- Published: 2016-09-01
- Earliest claimed: 2015-02-27
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Andres Valencia; Vinay Sridhara; Yin Chen; Rajarshi Gupta
- Web page: https://patentgrep.com/patent/US20160253498A1

## CPC

- G06F 21/554: {involving event detection and direct action}
- G06F 21/55: Detecting local intrusion or implementing counter-measures
- G06F 21/566: {Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities}
- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06N 20/00: Machine learning
- G06F 2221/034: Test or assess a computer or a system
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Various aspects include methods and computing devices implementing the methods for evaluating device behaviors in the computing devices. Aspect methods may include using a behavior-based machine learning technique to classify a device behavior as one of benign, suspicious, and non-benign. Aspect methods may include using one of a multi-label classification and a meta-classification technique to sub-classify the device behavior into one or more sub-categories. Aspect methods may include determining a relative importance of the device behavior based on the sub-classification, and determining whether to perform robust behavior-based operations based on the determined relative importance of the device behavior.

## Claims

### Claim 1

A method of evaluating device behaviors in a computing device, comprising:
using a behavior-based machine learning technique to classify a device behavior as one of benign, suspicious, and non-benign;
using one of a multi-label classification technique and a meta-classification technique to sub-classify the device behavior into a sub-classification that includes one or more sub-categories;
determining a relative importance of the device behavior based on the sub-classification; and
determining whether to perform robust behavior-based operations based on the relative importance of the device behavior.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
using the behavior-based machine learning technique to classify the device behavior as one of benign, suspicious, and non-benign comprises classifying the device behavior as a suspicious device behavior; and
using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises sub-classifying the suspicious device behavior into a sub-category selected from the group consisting of:
Adware;
Key Logger;
Ransomware;
Spyware;
mRAT;
Botnet;
Phishing;
Trojan; and
SMS Fraudware.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the relative importance of the device behavior based on the sub-classification comprises determining a severity of risk posed by the device behavior to the proper functioning of the computing device.

### Claim 4 (depends on 3)

The method of claim 3, further comprising balancing tradeoffs between amounts of processing, memory, or energy resources of the computing device that are used to analyze the device behavior and the severity of risk posed by the device behavior.

### Claim 5 (depends on 1)

The method of claim 1, further comprising determining actuation operations based on the relative importance of the device behavior.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a high risk sub-category; and
immediately terminating a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the high risk sub-category.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a high risk sub-category;
performing robust analysis operations to determine with a high degree of confidence whether the device behavior is non-benign in response to determining that the device behavior is sub-classified into a high risk sub-category; and
automatically terminating a software application program associated with the device behavior in response to determining with the high degree of confidence that the device behavior is non-benign.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a low risk sub-category; and
displaying a prompt that requests that a user of the computing device choose whether to uninstall or whitelist a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the low risk sub-category.

### Claim 9 (depends on 1)

The method of claim 1, wherein using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises applying a behavior vector information structure to one of a multi-classification classifier model and a meta-classifier model.

### Claim 10

A computing device, comprising:
means for using a behavior-based machine learning technique to classify a device behavior as one of benign, suspicious, and non-benign;
means for using one of a multi-label classification technique and a meta-classification technique to sub-classify the device behavior into a sub-classification that includes one or more sub-categories;
means for determining a relative importance of the device behavior based on the sub-classification; and
means for determining whether to perform robust behavior-based operations based on the relative importance of the device behavior.

### Claim 11 (depends on 10)

The computing device of claim 10, wherein:
means for using the behavior-based machine learning technique to classify the device behavior as one of benign, suspicious, and non-benign comprises means for classifying the device behavior as a suspicious device behavior; and
means for using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises means for sub-classifying the suspicious device behavior into a sub-category selected from the group consisting of:
Adware;
Key Logger;
Ransomware;
Spyware;
mRAT;
Botnet;
Phishing;
Trojan; and
SMS Fraudware.

### Claim 12 (depends on 10)

The computing device of claim 10, wherein means for determining the relative importance of the device behavior based on the sub-classification comprises means for determining a severity of risk posed by the device behavior to the proper functioning of the computing device, the computing device further comprising:
means for balancing tradeoffs between amounts of processing, memory, or energy resources used to analyze the device behavior and the severity of risk posed by the device behavior.

### Claim 13 (depends on 10)

The computing device of claim 10, wherein means for determining the relative importance of the device behavior based on the sub-classification and means for determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
means for determining whether the device behavior is sub-classified into a high risk sub-category; and
means for immediately terminating a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the high risk sub-category.

### Claim 14 (depends on 10)

The computing device of claim 10, wherein means for determining the relative importance of the device behavior based on the sub-classification and means for determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
means for determining whether the device behavior is sub-classified into a high risk sub-category;
means for performing robust analysis operations to determine with a high degree of confidence whether the device behavior is non-benign in response to determining that the device behavior is sub-classified into a high risk sub-category; and
means for automatically terminating a software application program associated with the device behavior in response to determining with the high degree of confidence that the device behavior is non-benign.

### Claim 15 (depends on 10)

The computing device of claim 10, wherein means for determining the relative importance of the device behavior based on the sub-classification and means for determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
means for determining whether the device behavior is sub-classified into a low risk sub-category; and
means for displaying a prompt that requests that a user choose whether to uninstall or whitelist a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the low risk sub-category.

### Claim 16 (depends on 10)

The computing device of claim 10, wherein means for using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises means for applying a behavior vector information structure to one of a multi-classification classifier model and a meta-classifier model.

### Claim 17

A computing device, comprising:
a memory; and
a processor coupled to the memory and configured with processor-executable instructions to perform operations comprising:
using a behavior-based machine learning technique to classify a device behavior as one of benign, suspicious, and non-benign;
using one of a multi-label classification technique and a meta-classification technique to sub-classify the device behavior into a sub-classification that includes one or more sub-categories;
determining a relative importance of the device behavior based on the sub-classification; and
determining whether to perform robust behavior-based operations based on the relative importance of the device behavior.

### Claim 18 (depends on 17)

The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations such that:
using the behavior-based machine learning technique to classify the device behavior as one of benign, suspicious, and non-benign comprises classifying the device behavior as a suspicious device behavior; and
using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises sub-classifying the suspicious device behavior into a sub-category selected from the group consisting of:
Adware;
Key Logger;
Ransomware;
Spyware;
mRAT;
Botnet;
Phishing;
Trojan; and
SMS Fraudware.

### Claim 19 (depends on 17)

The computing device of claim 17, wherein:
the processor is configured with processor-executable instructions to perform operations such that determining the relative importance of the device behavior based on the sub-classification comprises determining a severity of risk posed by the device behavior to the proper functioning of the computing device; and
the processor is configured with processor-executable instructions to perform operations further comprising balancing tradeoffs between amounts of processing, memory, or energy resources used to analyze the device behavior and the severity of risk posed by the device behavior.

### Claim 20 (depends on 17)

The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations such that determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a high risk sub-category; and
immediately terminating a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the high risk sub-category.

### Claim 21 (depends on 17)

The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations such that determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a high risk sub-category;
performing robust analysis operations to determine with a high degree of confidence whether the device behavior is non-benign in response to determining that the device behavior is sub-classified into a high risk sub-category; and
automatically terminating a software application program associated with the device behavior in response to determining with the high degree of confidence that the device behavior is non-benign.

### Claim 22 (depends on 17)

The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations such that determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a low risk sub-category; and
displaying a prompt that requests that a user choose whether to uninstall or whitelist a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the low risk sub-category.

### Claim 23 (depends on 17)

The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations such that using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises applying a behavior vector information structure to one of a multi-classification classifier model and a meta-classifier model.

### Claim 24

A non-transitory computer readable storage medium having stored thereon processor-executable software instructions configured to cause a processor to perform operations for evaluating device behaviors in a computing device, the operations comprising:
using a behavior-based machine learning technique to classify a device behavior as one of benign, suspicious, and non-benign;
using one of a multi-label classification technique and a meta-classification technique to sub-classify the device behavior into a sub-classification that includes one or more sub-categories;
determining a relative importance of the device behavior based on the sub-classification; and
determining whether to perform robust behavior-based operations based on the relative importance of the device behavior.

### Claim 25 (depends on 24)

The non-transitory computer readable storage medium of claim 24, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that:
using the behavior-based machine learning technique to classify the device behavior as one of benign, suspicious, and non-benign comprises classifying the device behavior as a suspicious device behavior; and
using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises sub-classifying the suspicious device behavior into a sub-category selected from the group consisting of:
Adware;
Key Logger;
Ransomware;
Spyware;
mRAT;
Botnet;
Phishing;
Trojan; and
SMS Fraudware.

### Claim 26 (depends on 24)

The non-transitory computer readable storage medium of claim 24, wherein:
the stored processor-executable software instructions are configured to cause a processor to perform operations such that determining the relative importance of the device behavior based on the sub-classification comprises determining a severity of risk posed by the device behavior to the proper functioning of the computing device; and
the stored processor-executable software instructions are configured to cause a processor to perform operations further comprising balancing tradeoffs between amounts of processing, memory, or energy resources used to analyze the device behavior and the severity of risk posed by the device behavior.

### Claim 27 (depends on 24)

The non-transitory computer readable storage medium of claim 24, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a high risk sub-category; and
immediately terminating a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the high risk sub-category.

### Claim 28 (depends on 24)

The non-transitory computer readable storage medium of claim 24, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a high risk sub-category;
performing robust analysis operations to determine with a high degree of confidence whether the device behavior is non-benign in response to determining that the device behavior is sub-classified into a high risk sub-category; and
automatically terminating a software application program associated with the device behavior in response to determining with the high degree of confidence that the device behavior is non-benign.

### Claim 29 (depends on 24)

The non-transitory computer readable storage medium of claim 24, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that determining the relative importance of the device behavior based on the sub-classification and determining whether to perform the robust behavior-based operations based on the relative importance of the device behavior comprise:
determining whether the device behavior is sub-classified into a low risk sub-category; and
displaying a prompt that requests that a user of the computing device choose whether to uninstall or whitelist a software application program associated with the device behavior in response to determining that the device behavior is sub-classified into the low risk sub-category.

### Claim 30 (depends on 24)

The non-transitory computer readable storage medium of claim 24, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that using one of the multi-label classification technique and the meta-classification technique to sub-classify the device behavior into the one or more sub-categories comprises applying a behavior vector information structure to one of a multi-classification classifier model and a meta-classifier model.
