Intelligent detection of bias within an artificial intelligence model

A non-transitory computer readable medium having instructions that, when executed by one or more processors, cause the one or more processors to:

• determine an AI model to test, wherein the AI model is a trained deep learning model such that a gradient of the trained deep learning model with respect to each token of the sentence is calculable;
• receive a text sentence and an identification of a class;
• determine a bias direction with respect to the class for an embedding model used by an artificial intelligence model to be analyzed, wherein to determine the bias direction includes to:
• obtain a corpus of token pairs related to the class;
• determine token embeddings for each token of the pairs of the corpus using the embedding model;
• perform principal component analysis on the embeddings such that variance associated with the class comes to lie on a first principal component that expresses a biased portion of the meaning of the tokens;
• and
• determine the bias direction using the first principal component;
• for each token in the text sentence, determine a protected gradient score, wherein determining the protected gradient score includes to:
• determine a directional derivative based on an output of the artificial intelligence model based on the token and with respect to the bias direction to generate a protected gradient;
• and
• determine a dot product of the protected gradient and the bias direction, wherein the protected gradient score is based on the dot product;
• aggregate the protected gradient scores of the tokens to form a sentence-level protected gradient score for the sentence;
• determine a fairness indicator of the AI model based on the sentence-level protected gradient score, wherein determining the fairness indicator of the AI model based on the sentence-level protected gradient score includes determining the fairness indicator of the AI model based on multiple sentence-level protected gradient scores determined for respective different sentences;
• and
• responsive to the fairness indicator being below a threshold, remediate the AI model to form a remediated AI model and deploy the remediated AI model in production.

A system comprising:

• one or more processors;
• and
• memory having instructions that, when executed by the one or more processors, cause the one or more processors to:
• determine an AI model to test, wherein the AI model is a trained deep learning model such that a gradient of the trained deep learning model with respect to each token of the sentence is calculable;
• receive a text sentence and an identification of a class;
• determine a bias direction with respect to the class for an embedding model used by an artificial intelligence model to be analyzed, wherein to determine the bias direction includes to:
• obtain a corpus of token pairs related to the class;
• determine token embeddings for each token of the pairs of the corpus using the embedding model;
• perform principal component analysis on the embeddings such that variance associated with the class comes to lie on a first principal component that expresses a biased portion of the meaning of the tokens;
• and
• determine the bias direction using the first principal component;
• for each token in the text sentence, determine a protected gradient score,
• wherein determining the protected gradient score includes to:
• determine a directional derivative based on an output of the artificial intelligence model based on the token and with respect to the bias direction to generate a protected gradient;
• and
• determine a dot product of the protected gradient and the bias direction, wherein the protected gradient score is based on the dot product;
• aggregate the protected gradient scores of the tokens to form a sentence-level protected gradient score for the sentence;
• determine a fairness indicator of the AI model based on the sentence-level protected gradient score, wherein determining the fairness indicator of the AI model based on the sentence-level protected gradient score includes determining the fairness indicator of the AI model based on multiple sentence-level protected gradient scores determined for respective different sentences;
• and
• responsive to the fairness indicator being below a threshold, remediate the AI model to form a remediated AI model and deploy the remediated AI model in production.