Optical sensing with nonlinear optical neural networks

A method, comprising:

• receiving light from a visual scene by a first optical linear layer in a nonlinear optical neural network (ONN) apparatus;
• linearly transforming the light into first optical outputs by the first optical linear layer that is trained to perform a first optical linear operation;
• nonlinearly generating second optical outputs based on the first optical outputs by an optical nonlinear layer in the ONN apparatus;
• and
• linearly transforming the second optical outputs into third optical outputs by a second optical linear layer in the ONN apparatus that is trained to perform a second optical linear operation,
• wherein the first optical linear layer, the optical nonlinear layer, and the second optical linear layer are sequentially arranged in series in the ONN apparatus.

The method of claim wherein the optical nonlinear layer comprises an optical nonlinear device,

• wherein the method further comprises:
• calibrating the optical nonlinear device to determine at least one property of the optical nonlinear device, and
• wherein calibrating the optical nonlinear device comprises:
• measuring an input intensity into the optical nonlinear device;
• measuring an output intensity from the optical nonlinear device;
• and
• determining a mathematical function or model between the input intensity and the output intensity.

An optical neural network (ONN) apparatus, comprising:

• a first optical linear layer configured to receive light from a visual scene and linearly transform the light into first optical outputs according to a first optical linear operation;
• an optical nonlinear layer coupled to the first optical linear layer and configured to nonlinearly generate second optical outputs based on the first optical outputs;
• and
• a second optical linear layer coupled to the optical nonlinear layer and configured to linearly transform the second optical outputs into third optical outputs according to a second optical linear operation,
• wherein the first optical linear layer, the optical nonlinear layer, and the second optical linear layer are sequentially coupled in series in the ONN apparatus.

A system, comprising:

• an optical neural network (ONN) apparatus comprising:
• a first optical linear layer configured to receive light from a visual scene and linearly transform the light into first optical outputs according to a first optical linear operation;
• an optical nonlinear layer coupled to the first optical linear layer and configured to nonlinearly generate second optical outputs based on the first optical outputs;
• and
• a second optical linear layer coupled to the optical nonlinear layer and configured to linearly transform the second optical outputs into third optical outputs according to a second optical linear operation,
• wherein the first optical linear layer, the optical nonlinear layer, and the second optical linear layer are sequentially coupled in series in the ONN apparatus;
• an optoelectronic device coupled to the ONN apparatus and configured to perform an optical-to-electric (OE) conversion of an optical signal output of the ONN apparatus into an electric data signal, the optical signal output of the ONN apparatus being associated with the third optical outputs;
• and
• a computing device coupled to the optoelectronic device and configured to process digital data associated with the electric data signal to generate a digital output corresponding to the visual scene.