Increase quality in artificial intelligence with reference tracking

A computing system comprising:

• at least one processor;
• at least one memory storing instructions that, when executed by the at least one processor, cause operations comprising:
• training a machine learning model with a training dataset, wherein the machine learning model comprises a plurality of layers;
• monitoring, during training, values of a plurality of coefficients of one or more layers;
• responsive to detecting a change of a given coefficient by more than a threshold during a given training run, storing a given reference to a given input dataset of the given training run;
• responsive to detecting an output error of a trained version of the machine learning model:
• retrieving the given reference to the given input dataset in response to the given coefficient being located on a backward path providing more than a threshold contribution to the output error;
• and
• providing the given reference to an application analyzing the trained version of the machine learning model to determine a cause of the output error.

A non-transitory computer readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:

• training a machine learning model with a training dataset, wherein the machine learning model comprises a plurality of layers;
• monitoring, during training, values of a plurality of coefficients of one or more layers;
• responsive to detecting a change of a given coefficient by more than a threshold during a given training run, storing a given reference to a given input dataset of the given training run;
• responsive to detecting an output error of a trained version of the machine learning model:
• retrieving the given reference to the given input dataset in response to the given coefficient being located on a backward path providing more than a threshold contribution to the output error;
• and
• providing the given reference to an application analyzing the trained version of the machine learning model to determine a cause of the output error.