Hyperspectral recovery from two images

A hyperspectral imaging device, comprising:

• one or more imaging lenses,
• one or more sensors positioned to receive light associated with an object from the one or more imaging lenses, and
• a spectral filter, wherein:
• the hyperspectral imaging device is configured to capture a first image produced without using the spectral filter and to capture a second image produced with the spectral filter,
• the first image and the second image have different spectral contents,
• the first and the second images are provided to a processor configured to process information corresponding to the first and the second images using a trained neural network for producing hyperspectral imaging data associated with the first and second images, and
• the spectral filter is positioned in an illumination path of the hyperspectral imaging device between an illumination source and the object.

A hyperspectral imaging device, comprising:

• an imaging lens,
• a sensor positioned to receive light associated with an object from the imaging lens, and
• an illumination source configurable to produce illumination with changeable spectral characteristics, wherein:
• the hyperspectral imaging device is configured to capture a first image of the object using a first illumination output from the illumination source, and to capture a second image of the object using a second illumination output from the illumination source,
• the first and second illumination outputs have different spectral characteristics, and
• the first and second images are provided to a processor configured to process information associated with the first and the second images using a trained neural network for producing hyperspectral imaging data associated with the first and second images.

A method for producing hyperspectral images from a plurality of images, the method comprising:

• performing a set of operations to simultaneously train a neural network and obtain designed spectral characteristics for a spectral filter or an illumination light source, the set of operations including:
• obtaining a plurality of training images, each training image having a hyperspectral ground-truth representation, and for each of the training images:
• (a) producing a first image having a first spectral content and a second image having a second spectral content that is different from the first spectral content,
• (b) providing the first and the second images as inputs to the neural network and obtaining an estimated hyperspectral image at an output of the neural network,
• comparing each estimated hyperspectral image to the corresponding hyperspectral ground-truth representation,
• upon a determination that a predetermined error criteria or similarity measure is not reached, producing the first image with modified first spectral content or the second image with modified second spectral content and repeating operation (b) and the comparing until the predetermined error criteria or the similarity measure is reached, and
• upon reaching the predetermined error criteria or the similarity measure, which is also indicative of completion of training of the neural network, using the first or the second spectral contents of the first or the second images to define spectral characteristics of the spectral filter or the illumination light source for subsequent generation of one or more hyperspectral images for at least one object whose image is not part of the plurality of training images.