# US 2017/0286544 A1 · Using hash signatures of dom objects to identify website similarity

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286544 A1
- Document ID: US20170286544A1
- Status: Published application
- Kind: A1
- Application number: 15/625,877
- Filed: 2017-06-16
- Published: 2017-10-05
- Earliest claimed: 2015-09-16
- Assignees: RiskIQ, Inc.
- Applicants: RiskIQ, Inc.
- Inventors: Adam Hunt; David Pon; Chris Kiernan; Ben Adams; Jonas Edgeworth; Elias Manousos; Joseph Linn
- Web page: https://patentgrep.com/patent/US20170286544A1

## CPC

- H04L 63/0876: {based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint}
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/953: Querying, e.g. by the use of web search engines
- G06F 16/958: Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
- G06F 21/128: {involving web programs, i.e. using technology especially used in internet, generally interacting with a web browser, e.g. hypertext markup language [HTML], applets, java}
- H04L 9/3247: {involving digital signatures}
- H04L 63/123: {received data contents, e.g. message integrity}
- H04L 63/1416: {Event detection, e.g. attack signature detection}
- H04L 63/1483: {service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12)}
- G06F 2221/2119: Authenticating web pages, e.g. with suspicious links
- H04L 63/10: {for controlling access to devices or network resources}
- H04L 63/1425: {Traffic logging, e.g. anomaly detection}
- H04L 2463/103: applying security measure for protecting copyright
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments are directed to using a hash signature of a rendered DOM object of a website to find similar content and behavior on other websites. Embodiments break a DOM into a large number of data portions (i.e., “shingles”), apply a hashing algorithm to the shingles, select a predetermined number of hashes from the hashed shingles according to a selection criteria to create a hash signature, and compare the hash signature to that of a reference page to determine similarity of website DOM object content. Embodiments can be used to identify phishing websites, defaced websites, spam websites, significant changes in the content of a webpage, copyright infringement, and any other suitable purposes related to the similarity between website DOM object content.

## Claims

### Claim 1

A method comprising, at a computer system:
rendering a document object model (DOM) object of a document;
generating, by a hardware processor of the computer system, a hash signature of the DOM object by:
determining a plurality of hashed data portions by applying a number of hashing functions to each of a plurality of data portions that are separated from content within the DOM object of the document, wherein applying the number of hashing functions results in a number of values for each of the plurality of data portions; and
generating a hash signature of the DOM object of the document based on selecting a number of hashed data portions of the plurality of hashed data portions;
comparing the hash signature of the DOM object to a known hash signature of a DOM object associated with a known document having a classification, wherein comparing the hash signature of the DOM object to the known hash signature of the DOM object associated with the known document includes comparing each of the plurality of hashed data portions to a plurality of known hashed data portions of the known hash signature;
calculating a similarity measurement between the hash signature of the DOM object and the known hash signature of the DOM object associated with the known document;
comparing the similarity measurement to a threshold; and
determining that the document has the classification based on the similarity measurement satisfying the threshold.

### Claim 2 (depends on 1)

The method of claim 1, wherein the document is an email document.

### Claim 3 (depends on 1)

The method of claim 1, wherein the document is a web-based document.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
receiving information from a server computer corresponding to a website, wherein the DOM object of the document is rendered using the information received from the server computer.

### Claim 5 (depends on 1)

The method of claim 1, wherein each of a plurality of data portions has a fixed length, and wherein the number of hashing functions is generated based on a seed value.

### Claim 6 (depends on 1)

The method of claim 1, wherein the classification of the known document is one of a trademark classification for trademark content or a copyright classification for copyrighted content.

### Claim 7 (depends on 1)

The method of claim 1, wherein the classification of the known document is a website revision classification.

### Claim 8 (depends on 1)

The method of claim 1, wherein the classification of the known document is a content classification for unauthorized content.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
determining that the document does not have the classification based on the similarity measurement not satisfying the threshold.

### Claim 10 (depends on 1)

The method of claim 1, wherein the threshold is based on the classification of the known document.

### Claim 11 (depends on 1)

The method of claim 1, wherein the number of hashing functions is based on the threshold for the classification of the known document.

### Claim 12 (depends on 1)

The method of claim 1, wherein the known hash signature is a first known hash signature of the DOM object associated with the known document, wherein the known document is a first known document, wherein the DOM object associated with the first known document is a first DOM object, wherein the classification is a first classification, wherein the similarity measurement is a first similarity measurement, wherein the threshold is a first threshold based on the first classification, and wherein the method further comprises:
comparing the hash signature of the DOM object to a second known hash signature of a second DOM object associated with a second known document having a second classification;
calculating a second similarity measurement between the hash signature of the DOM object and the second known hash signature of the second DOM object associated with the second known document;
comparing the second similarity measurement to a second threshold based on the second classification; and
determining whether the document has the second classification based on whether the second similarity measurement satisfies the second threshold.

### Claim 13 (depends on 12)

The method of claim 12, wherein the first classification is different from the second classification, and wherein the first threshold is different from the second threshold.

### Claim 14 (depends on 12)

The method of claim 12, wherein the second known hash signature is different from the first known hash signature.

### Claim 15 (depends on 12)

The method of claim 12, wherein the document does not have the second classification based on the second similarity measurement not satisfying the second threshold.

### Claim 16 (depends on 15)

The method of claim 15, wherein the first classification is a phishing classification, and wherein the second classification is one of a trademark classification for trademark content or a copyright classification for copyrighted content.

### Claim 17

A computer system comprising:
one or more processors; and
a memory accessible to the one or more processors, wherein the memory stores one or more instructions that, upon execution by the one or more processors, causes the one or more processors to:
render a document object model (DOM) object of a document;
generate a hash signature of the DOM object by:
determining a plurality of hashed data portions by applying a number of hashing functions to each of a plurality of data portions that are separated from content within the DOM object of the document, wherein applying the number of hashing functions results in a number of values for each of the plurality of data portions; and
generating a hash signature of the DOM object of the document based on selecting a number of hashed data portions of the plurality of hashed data portions;
compare the hash signature of the DOM object to a known hash signature of a DOM object associated with a known document having a classification, wherein comparing the hash signature of the DOM object to the known hash signature of the DOM object associated with the known document includes comparing each of the plurality of hashed data portions to a plurality of known hashed data portions of the known hash signature;
calculate a similarity measurement between the hash signature of the DOM object and the known hash signature of the DOM object associated with the known document;
compare the similarity measurement to a threshold; and
determine that the document has the classification based on the similarity measurement satisfying the threshold.

### Claim 18 (depends on 17)

The computer system of claim 17, wherein the document is an email document, wherein the threshold is based on the classification, and wherein the classification of the known document is one of a trademark classification for trademark content or a copyright classification for copyrighted content.

### Claim 19

An apparatus comprising:
one or more hardware processors configured to:
render a document object model (DOM) object of a document;
generate a hash signature of the DOM object by:
determining a plurality of hashed data portions by applying a number of hashing functions to each of a plurality of data portions that are separated from content within the DOM object of the document, wherein applying the number of hashing functions results in a number of values for each of the plurality of data portions; and
generating a hash signature of the DOM object of the document based on selecting a number of hashed data portions of the plurality of hashed data portions;
compare the hash signature of the DOM object to a known hash signature of a DOM object associated with a known document having a classification, wherein comparing the hash signature of the DOM object to the known hash signature of the DOM object associated with the known document includes comparing each of the plurality of hashed data portions to a plurality of known hashed data portions of the known hash signature;
calculate a similarity measurement between the hash signature of the DOM object and the known hash signature of the DOM object associated with the known document;
compare the similarity measurement to a threshold; and
determine that the document has the classification based on the similarity measurement satisfying the threshold.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein the known hash signature is a first known hash signature of the DOM object associated with the known document, wherein the known document is a first known document, wherein the DOM object associated with the first known document is a first DOM object, wherein the classification is a first classification, wherein the similarity measurement is a first similarity measurement, wherein the threshold is a first threshold based on the first classification, and wherein the one or more hardware processors is further configured to:
compare the hash signature of the DOM object to a second known hash signature of a second DOM object associated with a second known document having a second classification, wherein the first classification is different from the second classification;
calculate a second similarity measurement between the hash signature of the DOM object and the second known hash signature of the second DOM object associated with the second known document;
compare the second similarity measurement to a second threshold based on the second classification, wherein the first threshold is different from the second threshold; and
determine whether the document has the second classification based on whether the second similarity measurement satisfies the second threshold.
