# US 11,310,268 B2 · Systems and methods using computer vision and machine learning for detection of malicious actions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,268 B2 - Document ID: US11310268B2 - Status: Granted - Kind: B2 - Application number: 16/404,140 - Filed: 2019-05-06 - Published: 2020-11-12 - Granted: 2022-04-19 - Earliest claimed: 2019-05-06 - Assignees: Secureworks Corp. - Applicants: Secureworks Corp. - Inventors: William Parke Bowditch; Raul Garcia Calvo - Web page: https://patentgrep.com/patent/US11310268B2 ## CPC - H04L 63/1483: {service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12)} - G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} - G06F 18/24: Classification techniques - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06N 3/096: Transfer learning - G06N 20/00: Machine learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06N 3/08: Learning methods - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The present disclosure provides systems and methods for classifying or determined whether a request for a user's information is malicious or safe/legitimate. Request information related to a request for a user's information can be received, and one or more screenshots associated with the request can be obtained and provided to a machine learning model. The machine learning model can generate a probability or confidence level that the request is malicious. ## Claims ### Claim 1 A system for detection of security threats and/or malicious actions, comprising: one or more processors and at least one memory having a plurality of instructions that when executed by the one or more processors implement one or more components are configured to: receive a request seeking information or data related to a user's credentialing or personal information, and determine whether the request is associated with a webpage or requestor that is known to be malicious or known to be safe; if the request is not determined to be associated with a known malicious or known safe webpage or requestor, obtain a screenshot related to the request, wherein the screenshot comprises an image of a web page and wherein the one or more components comprises a web automation framework configured to, in isolation of a system or network of the user, securely navigate to the webpage and obtain the screenshot of the webpage; submit the screenshot to a machine learning model, and generate a probability that the request is malicious or non-malicious based on screenshot information identified and extracted from the screenshot using the machine learning model; and if the probability that the request is malicious exceeds a prescribed threshold: determine, based on the probability and on a series of automated analytics, an adjustment to the probability thereby defining an adjusted probability, and if the adjusted probability that the request is malicious exceeds the prescribed threshold, classify, in real-time, the request and/or webpage and/or requestor associated therewith as a malicious and/or generate and provide an alarm, alert, or notification. ### Claim 2 (depends on 1) The system of claim 1, wherein the request comprises at least one of a webpage or an email requesting the user's information or credentials. ### Claim 3 (depends on 1) The system of claim 1, wherein the information or data sought by the request for information includes Iterate URLs, POST requests, links to webpages, emails forwarded by user, webpages with a login form, or combinations thereof. ### Claim 4 (depends on 1) The system of claim 1, wherein the one or more components include a detection and extraction processor configured to compare the information or data sought by the request for information to information or data of a Blacklist and/or a Whitelist to determine whether the request for information is associated with a site or requestor that is known to be malicious or known to be safe. ### Claim 5 (depends on 4) The system of claim 4, wherein the detection and extraction processor is configured to compute or extract one or more features from the information or data sought by the request, including domain reputation, IP analysis, keywords in an email, a domain registration age, a domain registrar, a domain's SSL certificate details, or combinations thereof, to determine whether the request is a known safe or known malicious request. ### Claim 6 (depends on 1) The system of claim 1, wherein the screenshot information includes user interface elements, logos, slogans, trademarks, phrases, keywords, images, indicia, or combinations thereof. ### Claim 7 (depends on 1) The system of claim 1, wherein the machine learning model includes a machine learning algorithm including neutral network with a regional proposal algorithm. ### Claim 8 (depends on 1) The system of claim 1, wherein the one or more components are further configured to generate and/or update a Blacklist of known malicious requests based on output of the machine learning model. ### Claim 9 (depends on 1) The system of claim 1, wherein the request and the screenshot are received from a user's browser extension corresponding to the system. ### Claim 10 (depends on 1) The system of claim 1, wherein the machine learning model is trained with a first dataset including labeled images and with a second dataset including trademark images. ### Claim 11 (depends on 10) The system of claim 10, wherein the labeled images are included in a COCO dataset. ### Claim 12 (depends on 10) The system of claim 10, wherein the trademark images include an amount of images less than that of the labeled images and selected from companies that are frequently targeted by phishing attacks. ### Claim 13 (depends on 1) The system of claim 1, wherein the alarm, alert, or notification includes a pop-up window, the pop-up window including the probability or confidence level that the request is malicious and a user selectable option to thereby allow a user to prevent the webpage from creating additional dialogues. ### Claim 14 (depends on 1) The system of claim 1, wherein the system includes a RESTful service that provides a webpage, the webpage accessible by the user and to enable the user to submit the request seeking information. ### Claim 15 A method for detecting or classifying malicious activities, comprising: receiving request information related to a request for a user's credentialing information; obtaining one or more screenshots associated with the request; providing the one or more screenshots and the request information to a classifier including a machine learning model; using the machine learning model, computing or extracting one or more features from the request information, including domain reputation, IP analysis, keywords in an email, a domain registration age, a domain registrar, a domain's SSL certificate details, or combinations thereof; generating with the machine learning model a probability or confidence level that the request is malicious based at least in part on identified screenshot information from the one or more screenshots and the one or more features extracted from the request information; if the probability or confidence level that the request is malicious exceeds a prescribed threshold, determine, based on the probability and on a series of automated analytics, an adjustment to the probability thereby defining an adjusted probability; and if the adjusted probability that the request is malicious exceeds the prescribed threshold: classifying the request and/or an actor associated with the request as malicious, generating an alert, notification, or alarm and/or blocking further communication with the request and/or the actor associated therewith, and generating or compiling a blacklist of known malicious requests based on output of the machine learning model. ### Claim 16 (depends on 15) The method of claim 15, wherein the request compromises a webpage or email requesting a user's login information or credentials. ### Claim 17 (depends on 16) The method of claim 16, wherein the information or data related to the request includes Iterate URLs, POST requests, email data in a data center, emails forwarded by user, webpages with a login form, or combinations thereof. ### Claim 18 (depends on 16) The method of claim 16, further comprising: comparing the information or data related to the request information to information or data in a blacklist and/or a whitelist to determine whether the request is a known malicious or known safe request. ### Claim 19 (depends on 16) The method of claim 16, wherein the screenshot information includes user interface elements, logos, slogans, trademarks, phrases, keywords, images, indicia, or combinations thereof. ### Claim 20 (depends on 16) The method of claim 16, wherein the machine learning model includes a machine learning algorithm including neutral network with a regional proposal algorithm. ### Claim 21 (depends on 15) The method of claim 15, wherein blocking further communication with the request and/or the actor associated therewith includes preventing generation of additional dialogues from the request and/or actor. ### Claim 22 A non-transitory computer-readable medium including instructions for detecting or classifying malicious activities, which instructions, when executed by one or more processors, cause the one or more processors to perform a method comprising: receiving request information related to a request for a user's credentialing information; comparing the request information to information or data in a blacklist and/or a whitelist to determine whether the request is a known malicious or known safe request; if the request is not determined to be a known malicious or known safe request, obtaining one or more screenshots associated with the request, wherein the screenshot comprises an image of a web page obtained using a web automation framework configured to securely navigate to the webpage and obtain the screenshot of the webpage in isolation of a system or network of the user; providing the one or more screenshots and the request information to a classifier including a machine learning model; computing or extracting one or more features from the information related to the request, including domain reputation, IP analysis, keywords in an email, a domain registration age, a domain registrar, a domain's SSL certificate details, or combinations thereof, to determine whether the request is a known safe or known malicious request; generating with the machine learning model a probability or confidence level that the request is malicious based at least in part on identified screenshot information from the one or more screenshots and the request information; and if the probability or confidence level that the request is malicious exceeds a prescribed threshold: determine, based on the probability and on a series of automated analytics, an adjustment to the probability thereby defining an adjusted probability, if the adjusted probability that the request is malicious exceeds the prescribed threshold, classifying the request and/or an actor associated with the request as malicious, and generating an alert, notification, or alarm and/or blocking further communication with the request and/or the actor associated therewith.