# US 12,003,535 B2 · Phishing URL detection using transformers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,535 B2
- Document ID: US12003535B2
- Status: Granted
- Kind: B2
- Application number: 17/246,352
- Filed: 2021-04-30
- Published: 2022-09-01
- Granted: 2024-06-04
- Earliest claimed: 2021-03-01
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Jack Wilson Stokes, III; Pranav Ravindra Maneriker; Arunkumar Gururajan; Diana Anca Carutasu; Edir Vinicio Garcia Lazo
- Web page: https://patentgrep.com/patent/US12003535B2

## CPC

- H04L 63/1483: {service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12)}
- H04L 63/168: {above the transport layer}
- G06F 21/554: {involving event detection and direct action}
- G06F 21/56: Computer malware detection or handling, e.g. anti-virus arrangements
- G06F 21/566: {Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities}
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06N 3/096: Transfer learning
- H04L 63/1466: {Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks}
- G06F 2221/2119: Authenticating web pages, e.g. with suspicious links
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The technology described herein can identify phishing URLs using transformers. The technology tokenizes useful features from the subject URL. The useful features can include the text of the URL and other data associated with the URL, such as certificate data for the subject URL, a referrer URL, an IP address, etc. The technology may build a joint Byte Pair Encoding for the features. The token encoding may be processed through a transformer, resulting in a transformer output. The transformer output, which may be described as a token embedding, may be input to a classifier to determine whether the URL is a phishing URL. Additional or improved URL training data may be generated by permuting token order, by simulating a homoglyph attack, and by simulating an a compound word attack.

## Claims

### Claim 1

One or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, causes the one or more processors to perform a method for detecting a malicious uniform resource locator (URL) the method comprising:
tokenizing a URL into URL tokens;
tokenizing metadata associated with the URL into metadata tokens;
forming a token encoding from the URL tokens and the metadata tokens by generating a joint Byte Pair Encoding (BPE) that combines the URL tokens and the metadata tokens;
inputting the token encoding into a transformer model;
in response to inputting the token encoding into the transformer model, receiving an embedding vector from the transformer model;
calculating a decision statistic from the embedding vector; and
based on the decision statistic indicating the URL is malicious, taking a remedial action that limits access to a resource associated with the URL.

### Claim 2 (depends on 1)

The media of claim 1, the BPE model uses a URL specific vocabulary.

### Claim 3 (depends on 1)

The media of claim 1, wherein forming the token encoding comprises using a word piece model.

### Claim 4 (depends on 1)

The media of claim 1, wherein the remedial action comprises preventing a computing device from accessing the URL.

### Claim 5 (depends on 1)

The media of claim 1, wherein the method further comprises adding compound word split URLs to a training data corpus, the compound word split URLs labeled as phishing URLs.

### Claim 6 (depends on 1)

The media of claim 1, wherein the token encoding comprises a separator token between the URL tokens and the metadata tokens.

### Claim 7

A method for detecting a malicious uniform resource locator (URL) the method comprising:
tokenizing a URL into URL tokens;
forming a token encoding from the URL tokens by using a Byte Pair Encoding (BPE) model;
inputting the token encoding into a transformer model;
in response to inputting the token encoding into the transformer model, receiving an embedding vector from the transformer model;
calculating a decision statistic from the embedding vector; and
based on the decision statistic indicating the URL is malicious, taking a remedial action that limits access to a resource associated with the URL.

### Claim 8 (depends on 7)

The method of claim 7, wherein forming the token encoding comprises using a word piece model.

### Claim 9 (depends on 7)

The method of claim 7, wherein the token encoding comprises token positions and corresponding tokens.

### Claim 10 (depends on 7)

The method of claim 7, further comprising adding compound word split URLs to a training data corpus, the compound word split URLs labeled as phishing URLs.

### Claim 11 (depends on 7)

The method of claim 7, wherein the transformer model is a bidirectional encoder representation trained with URL training data.

### Claim 12 (depends on 7)

The method of claim 7, wherein the transformer model is a bidirectional encoder representation trained on a natural language vocabulary and optimized with URL training data.

### Claim 13 (depends on 7)

The method of claim 7, comprising preventing a computing device from accessing the URL.

### Claim 14

A system comprising:
one or more hardware processors; and
one or more computer-readable media having executable instructions embodied thereon, which, when executed by the one or more processors, cause the one or more hardware processors to execute actions comprising:
tokenizing a uniform resource locator (URL) into URL tokens;
tokenizing metadata associated with the URL into metadata tokens;
forming a token encoding from the URL tokens and the metadata tokens by generating a Byte Pair Encoding (BPE) that uses a URL specific vocabulary, the token encoding comprising a separator token between the URL tokens and the metadata tokens;
inputting the token encoding into a transformer model;
in response to inputting the token encoding into the transformer model, receiving an embedding vector from the transformer model;
calculating a decision statistic from the embedding vector; and
based on the decision statistic indicating the URL is malicious, taking a remedial action that limits access to a resource associated with the URL.

### Claim 15 (depends on 14)

The system of claim 14, wherein the transformer model is a bidirectional encoder representation trained with URL training data.

### Claim 16 (depends on 14)

The system of claim 14, wherein the decision statistic is calculated with a machine classifier comprising a single fully connected layer.

### Claim 17 (depends on 14)

The system of claim 14, wherein the metadata is certificate authority for the URL.

### Claim 18 (depends on 1)

The media of claim 1, wherein the method further comprises causing the transformer model to perform a mask pretraining task to determine a correlation between features associated with the URL and features associated with the metadata.

### Claim 19 (depends on 1)

The media of claim 1, wherein the token encoding includes word tokens corresponding to the URL tokens and position information associated with the URL tokens in the URL.

### Claim 20 (depends on 14)

The system of claim 14, wherein the token encoding includes URL tokens and the metadata tokens and position information associated with the URL tokens in the URL.