Visual Analytics System to Assess, Understand, and Improve Deep Neural Networks

A system for visualizing an operation of a neural network model, the system comprising:

• a display screen;
• a memory configured to store (i) a plurality of images, (ii) a plurality of outputs, each output in the plurality of outputs having been outputted by the neural network model responsive to a corresponding image in the plurality of images, and (iii) a plurality of sets of visual features, each set of visual features having been extracted from a corresponding image in the plurality of images;
• and
• a processor operably connected to the display screen and the memory, the processor being configured to:
• read, from the memory, (i) the plurality of images, (ii) the plurality of outputs, and (iii) the plurality of sets of visual features;
• and
• generate, and operate the display screen to display, a graphical user interface including a graphical depiction of the plurality of outputs, the graphical depiction including a plurality of graphical elements encoding the plurality of outputs which are visually arranged depending on the plurality of sets of visual features.

A non-transitory computer readable medium for visualizing an operation of a neural network model, the computer-readable medium storing program instructions that, when executed by a processor, cause the processor to:

• receive a plurality of images;
• receive a plurality of outputs, each output in the plurality of outputs having been outputted by the neural network model responsive to a corresponding image in the plurality of images;
• receive a plurality of sets of visual features, each set of visual features having been extracted from a corresponding image in the plurality of images;
• generate, and operate a display screen to display, a graphical user interface including a graphical depiction of the plurality of outputs, the graphical depiction including a plurality of graphical elements encoding the plurality of outputs which are visually arranged depending on the plurality of sets of visual features.