Machine learning based classification of anomalous or malicious objects using transformer encoders with dimensionality reduction

A computing device, comprising:

• a processor;
• and
• a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations including:
• obtaining an object including a text portion, wherein the object is to be classified as being associated with anomalous or malicious behavior;
• generating a token sequence by tokenizing the text portion of the object;
• deploying a machine learning model with the token sequence as input to the machine learning model resulting in a contextualized embedding sequence, wherein the contextualized embedding sequence is formed of a plurality of contextualized embedding vectors;
• reducing a dimension of a first contextualized embedding vector of the plurality of contextualized embedding vectors with a multi-layer transformer encoder including a plurality of linear layers resulting in a low-dimensionality vector, including a first linear layer configured to reduce the dimension of the first contextualized embedding vector from a first dimensionality to a second dimensionality, and a second linear layer configured to further reduce the dimension of the first contextualized embedding vector from the second dimensionality to a third dimensionality resulting in the low-dimensionality vector having two-dimensions;
• applying an activation function to the low-dimensionality vector resulting in a classification prediction of association with the anomalous or malicious behavior or no association;
• and
• responsive to the classification prediction being the object is associated with the anomalous or malicious behavior, causing one or more remediation or preventive efforts.

A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processor to perform operations including:

• obtaining an object including a text portion, wherein the object is to be classified as being associated with anomalous or malicious behavior;
• generating a token sequence by tokenizing the text portion of the object;
• deploying a machine learning model with the token sequence as input to the machine learning model resulting in a contextualized embedding sequence, wherein the contextualized embedding sequence is formed of a plurality of contextualized embedding vectors;
• reducing a dimension of a first contextualized embedding vector of the plurality of contextualized embedding vectors with a multi-layer transformer encoder including a plurality of linear layers resulting in a low-dimensionality vector, including a first linear layer configured to reduce the dimension of the first contextualized embedding vector from a first dimensionality to a second dimensionality, and a second linear layer configured to further reduce the dimension of the first contextualized embedding vector from the second dimensionality to a third dimensionality resulting in the low-dimensionality vector having two-dimensions;
• applying an activation function to the low-dimensionality vector resulting in a classification prediction of association with the anomalous or malicious behavior or no association;
• and
• responsive to the classification prediction being the object is associated with the anomalous or malicious behavior, causing one or more remediation or preventive efforts.