System and method for robust natural language classification under character encoding

A computer-implemented method, comprising:

• accepting a plurality of images representing a plurality of characters under various character encoding schemes for a target language character;
• creating a distribution of text similarity probabilities for the plurality of characters based on a text similarity of each of the plurality of characters to the target language character;
• utilizing the distribution of text similarity probabilities to identify a set of candidate characters among the plurality of characters that are likely to be swapped with the target language character in a character swap attack;
• accepting both the distribution of text similarity probabilities of the plurality of characters and a true text corpus as input, wherein the true text corpus comprises a set of text words of an electronic message without any character swapping being present;
• generating a synthetic training corpus by substituting one or more of the set of candidate characters with the target character in the set of text words of the true text corpus based on the distribution of text similarity probabilities of the plurality of characters;
• utilizing the synthetic training corpus to train one or more natural language processing (NLP) models for electronic message classification and identification of the character swap attack;
• and
• classifying an actual incoming text-based electronic message that contains the character swap attack using the trained one or more NLP models trained with the synthetic training corpus.

A non-transitory storage medium having software instructions stored thereon that when executed cause a system to:

• accept a plurality of images representing a plurality of characters under various character encoding schemes for a target language character;
• create a distribution of text similarity probabilities for the plurality of characters based on a probability of similarity of each of the plurality of characters to the target language character;
• utilize the distribution of text similarity probabilities to identify a set of candidate characters among the plurality of characters that are likely to be swapped with the target language character in a character swap attack;
• accept both the distribution of text similarity probabilities of the plurality of characters and a true text corpus as input, wherein the true text corpus comprises a set of text words of an electronic message without any character swapping being present;
• generate a synthetic training corpus by substituting one or more of the set of candidate characters with the target character in the set of text words of the true text corpus based on the distribution of text similarity probabilities of the plurality of characters;
• utilize the synthetic training corpus to train one or more natural language processing (NLP) models for electronic message classification and identification of the character swap attack;
• and
• classify an actual incoming text-based electronic message that contains the character swap attack using the trained one or more NLP models trained with the synthetic training corpus.