Embedding based anomaly detection for machine learning models

A computing system comprising:

• data processing hardware;
• and
• memory in communication with the data processing hardware, the memory storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to:
• obtain, from a user computing device, a first input, wherein the first input comprises a request to generate an output using a machine learning model;
• obtain, from a data store, a set of data based on obtaining the first input, wherein the set of data comprises one or more vector embeddings of anomalous data associated with the machine learning model;
• generate, using an embedding model, a first vector embedding of the first input;
• compare, using a vector search, the first vector embedding and the one or more vector embeddings;
• determine a manner of processing the first input based on comparing the first vector embedding and the one or more vector embeddings, wherein the manner of processing the first input indicates to drop the first input, route the first input to a first destination, or filter the first input;
• route the first input to the machine learning model based on the manner of processing the first input;
• obtain a first output of the machine learning model based on routing the first input to the machine learning model;
• generate, using the embedding model, a second vector embedding of the first output;
• compare, using the vector search, the second vector embedding and the one or more vector embeddings;
• determine a manner of processing the first output based on comparing the first vector embedding and the one or more vector embeddings, wherein the manner of processing the first output indicates to drop the first output, route the first output to a second destination, or filter the first output;
• and
• process the first output according to the manner of processing the first output.