Systems and methods for debugging neural networks with coverage guided fuzzing

A computing device comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing device to:
• obtain one or more inputs from an input corpus;
• mutate the one or more inputs;
• provide the one or more mutated inputs to a neural network;
• execute the neural network to process the one or more mutated inputs as input and producing one or more outputs;
• obtain, as a result of the neural network processing the one or more mutated inputs, an activation vector that describes, for each of a plurality of neurons of the neural network, whether the neuron was activated during processing of the one or more mutated inputs by the neural network, wherein the plurality of neurons of the neural network comprises at least one neuron in a layer prior to a final layer of the neural network;
• obtain a predetermined threshold distance;
• perform a nearest neighbors algorithm to determine whether a Euclidean distance between the activation vector and a nearest previous activation vector associated with a previous input to the neural network satisfies the predetermined threshold distance;
• upon determining that the Euclidean distance between the activation vector and the nearest previous activation vector satisfies the predetermined threshold distance, determine that the one or more mutated inputs provide new coverage;
• and
• upon determining that the one or more mutated inputs provide new coverage, add the one or more mutated inputs to the input corpus.

One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations, the operations comprising:

• obtaining one or more inputs from an input corpus;
• mutating the one or more inputs;
• providing the one or more mutated inputs to a neural network;
• executing the neural network to process the one or more mutated inputs as input and producing one or more outputs;
• obtaining as a result of the neural network processing the one or more mutated inputs, an activation vector that describes, for each of a plurality of neurons of the neural network, whether the neuron was activated during processing of the one or more mutated inputs by the neural network, wherein the plurality of neurons of the neural network comprises at least one neuron in a layer prior to a final layer of the neural network;
• obtaining a predetermined threshold distance;
• performing a nearest neighbors algorithm to determine whether a Euclidean distance between the activation vector and a nearest previous activation vector associated with a previous input to the neural network satisfies the predetermined threshold distance;
• upon determining that the Euclidean distance between the activation vector and the nearest previous activation vector satisfies the predetermined threshold distance, determining that the one or more mutated inputs provide new coverage;
• and
• upon determining that the one or more mutated inputs provide new coverage, adding the one or more mutated inputs to the input corpus.