Detection of machine learning model attacks obfuscated in unicode

A computer implemented method comprising:

• receiving, by an analysis engine, a prompt for ingestion by a generative artificial intelligence (GenAI) model comprising unicode;
• identifying and removing, by the analysis engine, at least one repeating sequence within the prompt to result in a modified prompt;
• determining, by the analysis engine, whether ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner;
• passing, by the analysis engine, the modified prompt to the GenAI model when it is determined that ingestion of the modified prompt will not result in the GenAI model behaving in an undesired manner;
• and
• initiating, by a remediation engine, at least one remediation action when it is determined that ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner.

A system comprising:

• a model computing environment comprising one or more computing devices each having hardware processors and memory collectively executing a generative artificial intelligence (GenAI) model and a proxy;
• a monitoring computing environment, separate and distinct from the model computing environment, comprising one or more computing devices having hardware processors and memory collectively executing an analysis engine and a remediation engine;
• wherein;
• the proxy receives a prompt for the GenAI model comprising unicode;
• the analysis engine identifies and removes at least one repeating sequence from the prompt to result in a modified prompt;
• the analysis engine determines whether ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner;
• the analysis engine passes the modified prompt to the GenAI model when it is determined that ingestion of the modified prompt will not result in the GenAI model behaving in an undesired manner;
• and
• the remediation engine initiates at least one remediation action when it is determined that ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner.