Automatic and unsupervised detached subgraph detection in deep learning programs

A computer-implemented method comprising:

• receiving a deep learning program with a neural network;
• running a pre-trained machine learning model with the deep learning program with the neural network as input, the pre-trained machine learning model detecting whether the deep learning program creates at runtime a detaching subgraph in the neural network;
• responsive to detecting that the deep learning program creates a detaching subgraph in the neural network, debugging automatically the deep learning program by at least outputting a location of the deep learning program causing the detaching subgraph;
• running the neural network in training mode;
• monitoring weight gradients associated with training of the neural network;
• based on the monitoring, detecting whether the neural network includes a detaching subgraph, wherein the neural network is determined as including a detaching subgraph responsive to determining that changes in the weight gradients are below a predefined threshold through training iterations;
• responsive to detecting that the neural network includes a detaching subgraph, outputting the detaching subgraph detected during the monitoring of the weight gradients, and correcting automatically the deep learning program by modifying a line of code in the deep learning program that creates the detaching subgraph;
• and
• pre-training the pre-trained machine learning model in unsupervised manner, wherein unlabeled training data including a plurality of deep learning programs are labeled automatically based on monitoring weight gradients associated with computing graphs of the plurality of deep learning programs and used to pre-train the pre-trained machine learning model,
• the pre-trained machine learning model identifying a location of the line of code in the deep learning program that creates the detaching subgraph, wherein
• fragments of source codes of deep learning programs as being constructed via an integrated programming environment are is-input to the pre-trained machine learning model for the pre-trained machine learning model to detect and output the location of the line of code in the source codes of deep learning programs that creates the detaching subgraph.

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a device to cause the device to:

• receive a deep learning program with a neural network;
• detect whether the deep learning program creates at runtime a detaching subgraph in the neural network;
• and
• responsive to detecting that the deep learning program creates a detaching subgraph in the neural network, debugging automatically the deep learning program by at least outputting a report indicating that an anomaly in the deep learning program is detected in that the neural network includes a detaching subgraph, and correcting automatically the deep learning program by modifying a line of code in the deep learning program that creates the detaching subgraph,
• wherein the device is caused to detect whether the neural network includes a detaching subgraph by inputting the deep learning program with neural network to a pre-trained machine learning model and running the pre-trained machine learning model, the pre-trained machine learning model outputting an indication whether the neural network includes a detaching subgraph,
• the pre-trained machine learning model identifying a location of the line of code in the deep learning program that creates the detaching subgraph, wherein
• fragments of source codes of deep learning programs as being constructed via an integrated programming environment are input to the pre-trained machine learning model for the pre-trained machine learning model to detect and output the location of the line of code in the source codes of deep learning programs that creates the detaching subgraph.