# US 12,701,126 B2 · Visual deep learning for inline phishing detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,701,126 B2 - Document ID: US12701126B2 - Status: Granted - Kind: B2 - Application number: 18/651,537 - Filed: 2024-04-30 - Published: 2025-10-30 - Granted: 2026-08-04 - Earliest claimed: 2024-04-30 - Assignees: Palo Alto Networks, Inc. - Applicants: Palo Alto Networks, Inc. - Inventors: Lucas Hu; Yu Zhang; Jingwei Fan; Wei Wang - Web page: https://patentgrep.com/patent/US12701126B2 ## CPC - H04L 63/1416: {Event detection, e.g. attack signature detection} - G06F 21/56: Computer malware detection or handling, e.g. anti-virus arrangements - G06N 3/08: Learning methods - G06V 30/19: Recognition using electronic means - H04L 63/1483: {service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12)} - G06F 2221/2119: Authenticating web pages, e.g. with suspicious links - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Techniques for visual deep learning for inline phishing detection are disclosed. In some embodiments, a system/process/computer program product for visual deep learning for inline phishing detection includes extracting a logo from a screenshot of a web page; detecting phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page; and performing a remedial action in response to determining that the web page is associated with phishing. ## Claims ### Claim 1 A system, comprising: a processor configured to: extract a logo from a screenshot of a web page; detect phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page, comprising to: input the extracted logo into the visual deep learning model to output a first feature vector and a second feature vector, wherein the visual deep learning model passes the extracted logo to at least two different sub-networks to generate the first feature vector and the second feature vector; input a combination of the first feature vector and the second feature vector into a neural network to generate a logo embedding; determine a similarity of the logo embedding associated with the logo and a reference logo embedding associated with the at least one reference logo; and in response to a determination that the similarity fails to equal or exceed a similarity threshold, determine that the web page is associated with phishing; and perform a remedial action in response to determining that the web page is associated with phishing; and a memory coupled to the processor and configured to provide the processor with instructions. ### Claim 2 (depends on 1) The system of claim 1, wherein a set of candidate reference logos are embedded as feature vectors using the visual deep learning model and then clustered, filtered, and deduplicated to generate the plurality of reference logos. ### Claim 3 (depends on 1) The system of claim 1, wherein the remedial action includes injecting a warning in the web page, and wherein the detecting phishing is performed using a browser extension, browser plugin, enterprise browser, or a remote browser isolation (RBI) environment. ### Claim 4 (depends on 1) The system of claim 1, wherein the remedial action includes blocking the response. ### Claim 5 (depends on 1) The system of claim 1, wherein performing the remedial action includes adding a Uniform Resource Locator (URL) associated with a detected phishing server to a block list. ### Claim 6 (depends on 1) The system of claim 1, wherein a login form is detected in the screenshot of the web page. ### Claim 7 (depends on 1) The system of claim 1, wherein a Uniform Resource Locator (URL) is extracted from a request for the web page that was determined to be associated with phishing activity, and wherein the URL is added to a URL block list. ### Claim 8 (depends on 1) The system of claim 1, wherein an output of the visual deep learning model includes an embedding vector that is compared with a plurality of embedding vectors for the plurality of reference logos. ### Claim 9 (depends on 1) The system of claim 1, wherein the plurality of reference logos includes a plurality of embedded reference logos, and wherein a new reference logo is added as a new embedded reference logo for potentially matching with the extracted logo without retraining of the visual deep learning model. ### Claim 10 (depends on 1) The system of claim 1, wherein performing the remedial action includes providing an explanation of the phishing detection that includes a matching reference logo and information indicating that the web page is not associated with an entity that matches the logo extracted from the web page. ### Claim 11 A method, comprising: extracting a logo from a screenshot of a web page; detecting phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page, comprising: inputting the extracted logo into the visual deep learning model to output a first feature vector and a second feature vector, wherein the visual deep learning model passes the extracted logo to at least two different sub-networks to generate the first feature vector and the second feature vector; inputting a combination of the first feature vector and the second feature vector into a neural network to generate a logo embedding; determining a similarity of the logo embedding associated with the logo and a reference logo embedding associated with the at least one reference logo; and in response to a determination that the similarity fails to equal or exceed a similarity threshold, determining that the web page is associated with phishing; and performing a remedial action in response to determining that the web page is associated with phishing. ### Claim 12 (depends on 11) The method of claim 11, wherein a set of candidate reference logos are embedded as feature vectors using the visual deep learning model and then clustered, filtered, and deduplicated to generate the plurality of reference logos. ### Claim 13 (depends on 11) The method of claim 11, wherein the remedial action includes injecting a warning in the web page, and wherein the detecting phishing is performed using a browser extension, browser plugin, enterprise browser, or a remote browser isolation (RBI) environment. ### Claim 14 (depends on 11) The method of claim 11, wherein the remedial action includes blocking the response. ### Claim 15 (depends on 11) The method of claim 11, wherein performing the remedial action includes adding a Uniform Resource Locator (URL) associated with a detected phishing server to a block list. ### Claim 16 (depends on 11) The method of claim 11, wherein a Uniform Resource Locator (URL) is extracted from a request for the web page that was determined to be associated with phishing activity, and wherein the URL is added to a URL block list. ### Claim 17 A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for: extracting a logo from a screenshot of a web page; detecting phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page, comprising: inputting the extracted logo into the visual deep learning model to output a first feature vector and a second feature vector, wherein the visual deep learning model passes the extracted logo to at least two different sub-networks to generate the first feature vector and the second feature vector; inputting a combination of the first feature vector and the second feature vector into a neural network to generate a logo embedding; determining a similarity of the logo embedding associated with the logo and a reference logo embedding associated with the at least one reference logo; and in response to a determination that the similarity fails to equal or exceed a similarity threshold, determining that the web page is associated with phishing; and performing a remedial action in response to determining that the web page is associated with phishing. ### Claim 18 (depends on 17) The computer program product of claim 17, wherein a set of candidate reference logos are embedded as feature vectors using the visual deep learning model and then clustered, filtered, and deduplicated to generate the plurality of reference logos. ### Claim 19 (depends on 17) The computer program product of claim 17, wherein the remedial action includes injecting a warning in the web page, and wherein the detecting phishing is performed using a browser extension, browser plugin, enterprise browser, or a remote browser isolation (RBI) environment. ### Claim 20 (depends on 17) The computer program product of claim 17, wherein the remedial action includes blocking the response.