Back to results
US 11,238,336 B2Granted
Optical convolutional neural network accelerator
AssigneeThe George Washington University
InventorsArmin Mehrabian, Volker J. Sorger, Tarek El-Ghazawi, Mario Miscuglio
Life of this patent
Estimating the term…
Abstract
An accelerator for modern convolutional neural networks applies the Winograd filtering algorithm in a wavelength division multiplexing integrated photonics circuit modulated by a memristor-based analog memory unit.
Claims21
A photonic accelerator for a convolutional neural network, comprising:
- an input buffer configured to receive one or more feature maps;
- a first digital-to-analog converter coupled to the input buffer, the first digital-to-analog converter configured to generate one or more analog signals corresponding to the one or more feature maps;
- a coherent light source coupled to the first digital-to-analog converter, the coherent light source configured to receive the one or more analog signals from the first digital-to-analog converter and output one or more light signals based, at least in part, on the one or more analog signals;
- an analog memory configured to generate one or more voltage signals;
- a photonic element-wise matrix multiplication circuit coupled to the coherent light source and the analog memory, wherein the photonic element-wise matrix multiplication circuit configured to:
- receive the one or more voltage signals from the analog memory;
- modulate the one or more light signals responsive to the one or more voltage signals, the modulated one or more light signals having one or more wavelengths having one or more wavelength-values;
- perform integration on the one or more wavelength-values of the modulated one or more light signals;
- and
- generate an analog electrical signal corresponding to the integrated one or more wavelength-values of the modulated light signals;
- and
- an analog-to-digital converter coupled to the photonic element-wise matrix multiplication circuit, the analog-to-digital converter configured to receive the analog electrical signal from the photonic element-wise matrix multiplication circuit, convert the analog electrical signal to a digital electrical signal, and output the digital electrical signal,
- wherein the one or more voltage signals correspond to one or more filter signals.
10 more elements
A neural network, the neural network including a system on chip, wherein the system on chip comprises:
- an input buffer configured to receive one or more feature maps;
- a first digital-to-analog converter coupled to the input buffer, the first digital-to-analog converter configured to generate one or more analog signals corresponding to the one or more feature maps;
- a coherent light source coupled to the first digital-to-analog converter, the coherent light source configured to receive the one or more analog signals from the first digital-to-analog converter and output one or more light signals based, at least in part, on the one or more analog signals;
- an analog memory configured to generate one or more voltage signals;
- a photonic element-wise matrix multiplication circuit coupled to the coherent light source and the analog memory, wherein the photonic element-wise matrix multiplication circuit configured to:
- receive the one or more voltage signals from the analog memory and modulate the one or more light signals responsive to the one or more voltage signals;
- and
- generate an analog electrical signal based, at least in part, on the one or more modulated light signals;
- and
- an analog-to-digital converter coupled to the photonic element-wise matrix multiplication circuit, the analog-to-digital converter configured to receive the analog electrical signal from the photonic element-wise matrix multiplication circuit, convert the analog electrical signal to a digital electrical signal, and output the digital electrical signal,
- wherein the one or more voltage signals correspond to one or more filter signals.
8 more elements
A method for accelerating computations for a convolutional neural network, comprising:
- receiving one or more feature maps at an input buffer;
- generating, using a first digital-to-analog converter coupled to the input buffer, one or more analog signals corresponding to the one or more feature maps;
- receiving, at a coherent light source coupled to the first digital-to-analog converter, the one or more analog signals from the first digital-to-analog converter;
- outputting, using the first digital-to-analog converter, one or more light signals based, at least in part, on the one or more analog signals;
- generating, using an analog memory, one or more voltage signals;
- receiving, at a photonic element-wise matrix multiplication circuit, the one or more voltage signals from the analog memory;
- modulating, using the photonic element-wise matrix multiplication circuit, the one or more light signals responsive to the one or more voltage signals, the one or more light signals corresponding to one or more wavelength-values;
- integrally summing, using the photonic element-wise matrix multiplication circuit, the one or more wavelength-values of the one or more light signals;
- and
- generating, using the photonic element-wise matrix multiplication circuit, an analog electrical signal corresponding to the one or more light signals so integrated;
- and
- receiving the analog electrical signal at an analog-to-digital converter;
- converting, using the analog electrical signal at an analog-to-digital converter, the analog electrical signal to a digital electrical signal;
- and outputting the digital electrical signal from the analog-to-digital converter,
- wherein the one or more voltage signals correspond to one or more analog Winograd domain filter signals.
12 more elements
A photonic element-wise matrix multiplication circuit configured to:
- receive one or more voltage signals from an analog memory;
- modulate one or more light signals responsive to the one or more voltage signals, the modulated one or more light signals having one or more wavelengths having one or more wavelength-values;
- perform integration on the one or more wavelength-values of the modulated one or more light signals;
- and
- generate an analog electrical signal corresponding to the integrated one or more wavelength-values of the modulated one or more light signals;
- and
- output the analog electrical signal to an analog-to-digital converter,
- wherein the one or more voltage signals correspond to one or more filters.
5 more elements
Dependencies are inferred from the claim text.