# US 2024/0267307 A1 · Abstracting Network Traffic as Video for Representation Learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0267307 A1
- Document ID: US20240267307A1
- Status: Published application
- Kind: A1
- Application number: 18/165,860
- Filed: 2023-02-07
- Published: 2024-08-08
- Earliest claimed: 2023-02-07
- Assignees: Micro Focus LLC
- Applicants: Micro Focus LLC
- Inventors: Manish Marwah; Martin Fraser Arlitt
- Web page: https://patentgrep.com/patent/US20240267307A1

## CPC

- H04L 43/026: using flow identification
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/47: {Detecting features for summarising video content}
- H04L 43/028: by filtering
- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A plurality of captured packets are received. The plurality of captured packets are from a plurality of packet flows. A packet flow is a communication session between two devices. For example, a packet flow may be a communication session between a client and a server. The plurality of captured packets are sorted into individual packet flows. The individual packet flows are converted into individual videos. For example, each packet from each packet flow is stored as a separate video frame in an individual video. A machine learning algorithm is applied to the individual videos to perform analytic tasks on the individual videos. For example, the machine learning algorithm may be used to identify anomalies within a packet flow and/or between packet flows.

## Claims

### Claim 1

A system comprising:
a microprocessor; and
a computer readable medium, coupled with the microprocessor and comprising microprocessor readable and executable instructions that, when executed by the microprocessor, cause the microprocessor to:
receive a plurality of captured packets, wherein the plurality of captured packets are from a plurality of packet flows;
sort the plurality of captured packets into individual packet flows;
convert the individual packet flows into individual videos; and
apply a machine learning algorithm to the individual videos to classify the individual videos.

### Claim 2 (depends on 1)

The system of claim 1, wherein classifying the individual videos comprise one or more of:
clustering the individual videos;
identifying an anomaly in a specific individual video;
identifying similar individual videos; and
training the machine learning model.

### Claim 3 (depends on 1)

The system of claim 1, wherein converting the individual packet flows into the individual videos comprises: placing all bytes of each packet in each of the individual packet flows to create each of the individual videos.

### Claim 4 (depends on 3)

The system of claim 3, wherein each byte of each packet in the individual packet flows is represented as a pixel in an induvial video frame.

### Claim 5 (depends on 3)

The system of claim 3, wherein each protocol header starts in a fixed location in each frame in the individual videos.

### Claim 6 (depends on 1)

The system of claim 1, wherein converting the individual packet flows into the individual videos comprises a coarse transformation that only discriminates between header and payload information.

### Claim 7 (depends on 1)

The system of claim 1, wherein converting the individual packet flows into individual videos comprises a medium transformation that uses pooling to convert the individual packet flows into the individual videos.

### Claim 8 (depends on 1)

The system of claim 1, wherein converting the individual packet flows into individual videos comprises a video summarization.

### Claim 9 (depends on 1)

The system of claim 1, wherein one or more of the classified individual videos are displayed visually to a user.

### Claim 10 (depends on 9)

The system of claim 9, wherein individual headers and/or a payload are displayed to the user in different colors and/or grey scale for identifying differences between the individual videos.

### Claim 11 (depends on 9)

The system of claim 9, wherein the user can select the individual headers and/or payload and wherein the selection causes the individual headers and/or payload to be converted back to original packet data for display.

### Claim 12 (depends on 1)

The system of claim 1, wherein a first individual video is compared to a second individual video and wherein the second individual video is retrieved from a malicious flow database.

### Claim 13 (depends on 1)

The system of claim 1, wherein frames in the individual videos comprise metadata and wherein the metadata comprises one or more of: a source IP address, a destination IP address, a source port number, a destination port number, a type of frame, a multiple frame packet indicator, a packet number, a cryptographic hash offset, failure information, packet header locations, an event, a Media Access (MAC) address, and a network address.

### Claim 14

A method comprising:
receiving, by a microprocessor, a plurality of captured packets, wherein the plurality of captured packets are from a plurality of packet flows;
sorting, by the microprocessor, the plurality of captured packets into individual packet flows;
converting, by the microprocessor, the individual packet flows into individual videos; and
applying, by the microprocessor, a machine learning algorithm to the individual videos to classify the individual videos.

### Claim 15 (depends on 14)

The method of claim 14, wherein converting the individual packet flows into the individual videos comprises: placing all bytes of each packet in each of the individual packet flows to create each of the individual videos.

### Claim 16 (depends on 15)

The method of claim 15, wherein each byte of each packet in the individual packet flows is represented as a pixel in an induvial video frame.

### Claim 17 (depends on 14)

The method of claim 14, wherein converting the individual packet flows into the individual videos comprises a coarse transformation that only discriminates between header and payload information.

### Claim 18 (depends on 14)

The method of claim 14, wherein converting the individual packet flows into individual videos comprises a medium transformation that uses pooling to convert the individual packet flows into the individual videos.

### Claim 19 (depends on 14)

The method of claim 14, wherein converting the individual packet flows into individual videos comprises a video summarization.

### Claim 20

A non-transient computer readable medium having stored thereon instructions that cause a processor to execute a method, the method comprising instructions to:
receive a plurality of captured packets, wherein the plurality of captured packets are from a plurality of packet flows;
sort the plurality of captured packets into individual packet flows;
convert the individual packet flows into individual videos; and
apply a machine learning algorithm to the individual videos to classify the individual videos.
