Systems and Methods for Holistic Extraction of Features from Neural Networks

A method for identifying informative features within input data using a neural network data structure, comprising:

• a network interface;
• a processor, and;
• a memory, containing:
• a feature application;
• a data structure describing a neural network that comprises a plurality of neurons;
• wherein the processor is configured by the feature application to:
• determining contributions of individual neurons to activation of a target neuron by comparing activations of a set of neurons to their reference values, where the contributions are computed by dynamically backpropagating an importance signal through the data structure describing the neural network;
• extracting aggregated features detected by the target neuron by:
• segmenting the determined contributions to the target neuron;
• clustering the segmented contributions into clusters of similar segments;
• aggregating data within clusters of similar segments to identify aggregated features of input data that contribute to the activation of the target neuron;
• and
• displaying the aggregated features of input data to highlight important features of the input data relied upon by the neural network.