System and method for iterative classification using neurophysiological signals

A system for training an image classification neural network, comprising:

• a neurophysiological signal collection system, configured for collecting neurophysiological signals from a brain of an observer;
• and
• a data processor, communicating with said neurophysiological signal collection system and being configured for:
• presenting a first plurality of images to an observer as a visual stimulus, while collecting neurophysiological signals from a brain of said observer;
• processing said neurophysiological signals to identify a neurophysiological event indicative of a detection of a target by said observer in at least one image of said first plurality of images;
• training the image classification neural network to identify the target in the image, based on said identification of said neurophysiological event;
• and
• storing said trained image classification neural network in a non-transitory computer-readable storage medium;
• wherein said data processor is configured to apply unsupervised clustering to a second plurality of images, and to select said first plurality of images from said second plurality of images based on said unsupervised clustering;
• wherein said image classification neural network comprises:
• a first neural subnetwork configured for receiving and processing said neurophysiological data, a first separate output layer for said first neural subnetwork outputting a first score, a second neural subnetwork configured for receiving and processing said second plurality of images, second separate output layer for said second neural subnetwork outputting a second score, and a shared subnetwork having a neural network layer receiving and combining outputs from both said first neural subnetwork and said second neural subnetwork;
• and
• wherein the data processor is also configured to combine said first score with said second score as a weighted sum of said first and said second score, to label said image with said weighted sum, and to use said label in at least one iteration of said training.

A method of image classification, comprising:

• presenting to an observer a stimulus describing a target;
• following said presentation of said stimulus describing a target, training an image classification neural network to identify said target in images;
• applying said trained image classification neural network to the image to detect therein candidate image regions suspected as being occupied by said target;
• presenting to an observer each candidate image region as a visual stimulus, while collecting neurophysiological signals from a brain of said observer;
• and
• determining an existence of said target in the image is based, at least in part, on said identification of said neurophysiological event.

A system for image classification, comprising:

• a neurophysiological signal collection system, configured for collecting neurophysiological signals from a brain of an observer;
• and
• a data processor, communicating with said neurophysiological signal collection system and being configured for:
• applying a trained image classification neural network to the image to detect therein candidate image regions suspected as being occupied by a target;
• presenting to an observer each candidate image region as a visual stimulus, while collecting neurophysiological signals from a brain of said observer;
• and
• determining an existence of said target in the image is based, at least in part, on said identification of said neurophysiological event;
• wherein said trained image classification neural network comprises an autoencoder subnetwork for unsupervised feature learning.

A method of training an image classification neural network, the method comprising:

• presenting a first plurality of images to an observer as a visual stimulus, while collecting neurophysiological signals from a brain of said observer;
• processing said neurophysiological signals to identify a neurophysiological event indicative of a detection of a target by said observer in at least one image of said first plurality of images;
• training the image classification neural network to identify the target in the image, based on said identification of said neurophysiological event;
• and
• storing said trained image classification neural network in a non-transitory computer-readable storage medium;
• wherein said image classification neural network comprises an autoencoder subnetwork for unsupervised feature learning.

A method of training an image classification neural network, the method comprising:

• presenting a first plurality of images to an observer as a visual stimulus, while collecting neurophysiological signals from a brain of said observer;
• processing said neurophysiological signals to identify a neurophysiological event indicative of a detection of a target by said observer in at least one image of said first plurality of images;
• training the image classification neural network to identify the target in the image, based on said identification of said neurophysiological event;
• and
• storing said trained image classification neural network in a non-transitory computer-readable storage medium;
• wherein the method comprises applying unsupervised clustering to a second plurality of images, and selecting said first plurality of images from said second plurality of images based on said unsupervised clustering;
• and
• wherein said image classification neural network comprises:
• a first neural subnetwork configured for receiving and processing said neurophysiological data,
• a second neural subnetwork configured for receiving and processing said second plurality of images,
• a shared subnetwork having a neural network layer receiving and combining outputs from both said first neural subnetwork and said second neural subnetwork,
• a first separate output layer for said first neural subnetwork outputting a first score, and
• a second separate output layer for said second neural subnetwork outputting a second score;
• and wherein the method comprises combining said first score with said second score as a weighted sum of said first and said second score, labeling said image with said weighted sum, and using said label in at least one iteration of said training.