System and method for testing machine learning

A method for automated machine learning failure mode discrimination for an interactive machine learning model data architecture experiencing a failure for an unknown reason, the method comprising:

• instantiating a discriminator machine learning network machine comprising a pre-processing discriminator machine learning network, a first discriminator machine learning network, and a second discriminator machine learning network that interoperate together to generate intermediate outputs representing values indicative of whether a null hypothesis for at least one failure type can be rejected, wherein the pre-processing discriminator machine learning network, the first discriminator machine learning network and the second discriminator machine learning network operate in parallel to improve computational efficiency and speed;
• upon detection of a failure of an iterative machine learning model data architecture, communicating across a coupled message bus to iteratively retrieve message frames or packets comprising, as an input data set, n-tuples representative of model training of an iterative machine learning model data architecture, the n-tuples representing validation loss curves extracted from the model training over a number of epochs;
• processing the input data set using a first discriminator machine learning network to generate a first intermediate data value representative of a first probability value of rejecting a first null hypothesis for a first failure type indicative of a conditional dependency in the iterative machine learning model data architecture between X and Y in respect of Z=(X, Y), where Z is a data set of the iterative machine learning model data architecture, X is an input of the iterative machine learning model data architecture, and Y is a target or label of the iterative machine learning model data architecture;
• processing the input data set using a second discriminator machine learning network to generate a set of three loss curves, a first loss curve directed to an original validation dataset Zvalid, a second loss curve directed to a new validation data set spawned from an original training set Z′valid, and a third loss curve directed to a target shuffled validation dataset Z″valid;
• generating a second intermediate data value representative of a second probability value of rejecting a second null hypothesis for a second failure type that Ztrain,i and Zvalid,i are equal in distribution where Ztrain,i and Zvalid,i are data sets generated from a training set and the validation set for the iterative machine learning model data architecture, respectively;
• processing the input data set using the second discriminator machine learning network to generate a third intermediate data value representative of a third probability value of rejecting a third null hypothesis for a third failure type that convergence rates Rtrain VS Rvalid are equal in distribution, indicative of whether measuring an amount of learnt conditional dependency from Ztrain generalizes to Zvalid;
• generating an output data structure encapsulating the first, second, and third probability values, wherein the output data structure indicates a failure type value representing one of the first failure type, the second failure type, and the third failure type;
• exposing the output data structure through an application programming interface to facilitate communication and processing of the output data structure by downstream systems to debug and rectify the failure based on the failure type value indicated by the output data structure, wherein the iterative machine learning model data architecture can be re-trained using the output data structure to improve at least one of accuracy, convergence rate, and reduced total number of processing cycles of the iterative machine learning model data architecture;
• and
• controlling rendering of a graphical user interface including a plurality of interactive graphical interface control elements having graphical rendering characteristics controlled based on the first probability value, the second probability value, and the third probability value.

A non-transitory computer readable medium storing machine interpretable instructions for automated machine learning failure discrimination, the machine interpretable instructions, when executed by a processor, cause the processor to perform steps of a method for automated machine learning failure mode discrimination for an interactive machine learning model data architecture experiencing a failure for an unknown reason, the method comprising:

• instantiating a discriminator machine learning network machine comprising a pre-processing discriminator machine learning network, a first discriminator machine learning network, and a second discriminator machine learning network that interoperate together to generate intermediate outputs representing values indicative of whether a null hypothesis for at least one failure type can be rejected, wherein the pre-processing discriminator machine learning network, the first discriminator machine learning network and the second discriminator machine learning network operate in parallel to improve computational efficiency and speed;
• upon detection of a failure of an iterative machine learning model data architecture, communicating across a coupled message bus to iteratively retrieve message frames or packets comprising, as an input data set, n-tuples representative of model training of an iterative machine learning model data architecture, the n-tuples representing validation loss curves extracted from the model training over a number of epochs;
• processing the input data set using a first discriminator machine learning network to generate a first intermediate data value representative of a first probability value of rejecting a first null hypothesis for a first failure type indicative of a conditional dependency in the iterative machine learning model data architecture between X and Y in respect of Z=(X, Y), where Z is a data set of the iterative machine learning model data architecture, X is an input of the iterative machine learning model data architecture, and Y is a target or label of the iterative machine learning model data architecture;
• processing the input data set using a second discriminator machine learning network to generate a set of three loss curves, a first loss curve directed to an original validation dataset Zvalid, a second loss curve directed to a new validation data set spawned from an original training set Z′valid, and a third loss curve directed to a target shuffled validation dataset Z″valid;
• generating a second intermediate data value representative of a second probability value of rejecting a second null hypothesis for a second failure type that Ztrain,i and Zvalid,i are equal in distribution where Ztrain,i and Zvalid,i are data sets generated from a training set and the validation set for the iterative machine learning model data architecture, respectively;
• processing the input data set using the second discriminator machine learning network to generate a third intermediate data value representative of a third probability value of rejecting a third null hypothesis for a third failure type that convergence rates Rtrain VS Rvalid are equal in distribution, indicative of whether measuring an amount of learnt conditional dependency from Ztrain generalizes to Zvalid;
• generating an output data structure encapsulating the first, second, and third probability values, wherein the output data structure indicates a failure type value representing one of the first failure type, the second failure type, and the third failure type;
• and
• exposing to the output data structure through an application programming interface to facilitate communication and processing of the output data structure by downstream systems to debug and rectify the failure based on the failure type value indicated by the output data structure, wherein the iterative machine learning model data architecture can be re-trained using the output data structure to improve at least one of accuracy, convergence rate, and reduced total number of processing cycles of the iterative machine learning model data architecture;
• and
• controlling rendering of a graphical user interface including a plurality of interactive graphical interface control elements having graphical rendering characteristics controlled based on the first probability value, the second probability value, and the third probability value.