# US 2019/0356394 A1 · Photonic processing systems and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0356394 A1
- Document ID: US20190356394A1
- Status: Published application
- Kind: A1
- Application number: 16/412,098
- Filed: 2019-05-14
- Published: 2019-11-21
- Earliest claimed: 2018-05-15
- Assignees: Lightmatter, Inc.
- Applicants: Lightmatter, Inc.
- Inventors: Darius Bunandar; Nicholas C. Harris; Carl Ramey
- Web page: https://patentgrep.com/patent/US20190356394A1

## CPC

- H04B 10/548: Phase or frequency modulation
- G02F 1/21: by interference
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0675: {using electro-optical, acousto-optical or opto-electronic means}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- H04B 10/70: Photonic quantum communication
- H04B 10/801: {using optical interconnects, e.g. light coupled isolators, circuit board interconnections}
- H04J 14/02: Wavelength-division multiplex systems
- H04J 14/0279: {WDM point-to-point architectures}
- G02F 1/212: {Mach-Zehnder type}
- G02F 2203/48: Variable attenuator
- G02F 2203/50: Phase-only modulation
- G06N 3/047: Probabilistic or stochastic networks
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- H04B 10/61: Coherent receivers
- H04B: TRANSMISSION
- G02F: OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04J: MULTIPLEX COMMUNICATION (transmission in general H04B; peculiar to transmission of digital information H04L5/00; systems for the simultaneous or sequential transmission of more than one television signal H04N7/08; in exchanges H04Q11/00; stereophonic systems H04S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Aspects relate to a photonic processing system, a photonic processor, and a method of performing matrix-vector multiplication. An optical encoder may encode an input vector into a first plurality of optical signals. A photonic processor may receive the first plurality of optical signals; perform a plurality of operations on the first plurality of optical signals, the plurality of operations implementing a matrix multiplication of the input vector by a matrix; and output a second plurality of optical signals representing an output vector. An optical receiver may detect the second plurality of optical signals and output an electrical digital representation of the output vector.

## Claims

### Claim 1

A photonic processor comprising:
a first array of interconnected variable beam splitters (VBSs) comprising a first plurality of optical inputs and a first plurality of optical outputs;
a second array of interconnected VBSs comprising a second plurality of optical inputs and a second plurality of optical outputs; and
a plurality of controllable optical elements, each of the plurality of these controllable optical elements coupling a single one of the first plurality of optical outputs of the first array to a respective single one of the second plurality of optical inputs of the second array.

### Claim 2 (depends on 1)

The photonic processor of claim 1, wherein each of the VBSs of the first and second array comprises a Mach-Zehnder interferometer comprising:
a first beam splitter;
a second beam splitter; and
at least one phase modulator configured to modulate a phase of light of an optical mode that couples the first beam splitter and the second beam splitter.

### Claim 3 (depends on 1)

The photonic processor of claim 1, wherein each of the controllable optical elements comprises a variable attenuator and at least one phase modulator.

### Claim 4 (depends on 3)

The photonic processor of claim 3, wherein each of the variable attenuators of the controllable optical elements comprises a Mach-Zehnder interferometer.

### Claim 5 (depends on 1)

The photonic processor of claim 1, further comprising at least one fold between the first array of interconnected VBSs and the second array of interconnected VBSs, wherein a propagation direction of an optical mode of each of the first plurality of optical outputs is opposite a propagation direction of an optical mode of a respective optical input of the second plurality of inputs.

### Claim 6 (depends on 4)

The photonic processor of claim 4, further comprising a plurality of columns of VBSs, wherein at least a subset of the plurality of columns comprises at least one of the VBSs of the first array of interconnected VBSs and at least one of the VBSs of the second array of interconnected VBSs such that at least a portion of the first array of interconnected VBSs is interleaved with at least a portion of the second array of interconnected VBSs.

### Claim 7

A photonic processing system comprising:
an optical encoder configured to encode an input vector into a first plurality of optical signals;
a photonic processor configured to:
receive the first plurality of optical signals, each of the first plurality of signals received by a respective input spatial mode of a plurality of input spatial modes of the photonic processor,
perform a plurality of operations on the first plurality of optical signals, the plurality of operations implementing a matrix multiplication of the input vector by a matrix; and
output a second plurality of optical signals representing an output vector, each of the second plurality of signals transmitted by a respective output spatial mode of a plurality of output spatial modes of the photonic processor; and
an optical receiver configured to detect the second plurality of optical signals and output an electrical digital representation of the output vector.

### Claim 8 (depends on 7)

The photonic processing device of claim 7, wherein the optical receiver is configured to detect the second plurality of optical signals using phase sensitive detectors.

### Claim 9 (depends on 8)

The photonic processing device of claim 8, further comprising a light source configured to:
provide first light to the optical encoder for use in encoding the first plurality of optical signals; and
provide second light to the optical receiver for use as a local oscillator by the phase sensitive detectors, wherein:
the local oscillator is phase coherent with each of the first plurality of optical signals; and
a first path length of the first plurality of optical signals from the light source to the optical receiver is substantially equal to a second path length of the local oscillator from the light source to the optical receiver.

### Claim 10 (depends on 7)

The photonic processing device of claim 7, wherein the optical encoder is configured to:
encode an absolute value of a vector component of the input vector into an amplitude of a respective optical signal of the first plurality of optical signals; and
encode a phase of the vector component of the input vector into a phase of the respective optical signal of the first plurality of optical signals.

### Claim 11 (depends on 7)

The photonic processing device of claim 7, wherein the matrix is an arbitrary unitary matrix.

### Claim 12 (depends on 7)

The photonic processing device of claim 7, further comprising:
a plurality of frontends, wherein each of the plurality of frontends is associated with one input spatial mode of the plurality of input spatial modes of the photonic processor, wherein each of the plurality of frontends comprises:
a plurality of optical encoders, each of the optical encoders configured to encode a respective component of an input vector into an optical signal, wherein each optical encoder is configured to output an optical signals of a wavelength different from wavelengths output by the other optical encoders; and
an input wavelength division multiplexer (WDM) configured to receive each of the optical signals from each of the plurality of optical encoders in a separate spatial mode and output each of the optical signals in a single spatial mode connected to a respective input spatial mode of the plurality of input spatial modes of the photonic processor; and
a plurality of backends, wherein each of the plurality of backends is associated with one output spatial mode of a plurality of output spatial modes of the photonic processor, wherein each of the plurality of backends comprises:
an output wavelength division multiplexer (WDM) configured to receive optical signals of different wavelengths from a respective one of the plurality of output spatial modes of the photonic processor and output each of the optical signals of different wavelengths in a respective spatial mode of a plurality of output spatial modes of the WDM;
a plurality of optical receivers, each of the optical receivers configured to determine a respective component of an output vector by detecting a respective optical signal associated with a respective output spatial mode of the WDM.

### Claim 13 (depends on 7)

The photonic processing device of claim 7, wherein the photonic processor comprises:
a first array of interconnected variable beam splitters (VBSs) comprising a first plurality of optical inputs corresponding to the first plurality of input spatial modes and a first plurality of optical outputs;
a second array of interconnected VBSs comprising a second plurality of optical inputs and a second plurality of optical outputs corresponding to the plurality of output spatial modes; and
a plurality of controllable optical elements, each of the plurality of these controllable optical elements coupling a single one of the first plurality of optical outputs of the first array to a respective single one of the second plurality of optical inputs of the second array.

### Claim 14 (depends on 13)

The photonic processing device of claim 13, further comprising a controller configured to:
perform a singular value decomposition (SVD) of the matrix to determine a first, second, and third SVD matrix;
control the first plurality of interconnected VBSs to implement the first SVD matrix;
control the second plurality of interconnected VBSs to implement the second SVD matrix
control the plurality of controllable optical elements to implement the third SVD matrix, wherein the third SVD matrix is a diagonal matrix.

### Claim 15 (depends on 14)

The photonic processing device of claim 14, wherein the controller further comprises at least one digital-to-analog converter (DAC) to adjust one or more parameters of the first plurality of interconnected VBSs and the second plurality of interconnected VBSs.

### Claim 16 (depends on 15)

The photonic processing device of claim 15, wherein:
each of the VBSs of the first plurality of interconnected VBSs and each of the VBSs of the second plurality of interconnected VBSs is associated with a respective address; and
the at least one DAC includes a single DAC that controls a plurality of the VBSs of the first and/or second plurality of interconnected VBSs using the addresses.

### Claim 17 (depends on 7)

The photonic processing device of claim 7, wherein the matrix is a first matrix and the photonic processing device further comprises a controller configured to control the photonic processing device to perform multiplication of a second matrix by the first matrix by:
(a) determining a plurality of input vectors from each column of the second matrix;
(b) selecting an input vector from the plurality of input vectors;
(c) encoding the selected input vector into the first plurality of optical signals using the optical encoder;
(d) performing the plurality of operations on the first plurality of optical signals associated with the first input vector;
(e) detecting the second plurality of optical signals associated with the selected input vector;
(f) storing digital detection results based on the detected second plurality of optical signals;
(g) repeating acts (b)-(f) for the other input vectors of the plurality of input vectors;
(h) digitally combine the digital detection results to determine a resulting matrix resulting from the multiplication of the second matrix by the first matrix.

### Claim 18 (depends on 7)

The photonic processing device of claim 7, wherein the optical receiver comprises a low-pass filter configured to perform an analog summation of multiple subsequent signals associated with each output spatial mode of the plurality of output spatial modes of the photonic processor.

### Claim 19

A method of optically performing matrix-vector multiplication, the method comprising:
receiving a digital representation of an input vector;
encoding, using an optical encoder, the input vector into a first plurality of optical signals;
performing, using a processor, a singular value decomposition (SVD) of a matrix to determine a first, second, and third SVD matrix;
controlling photonic processor comprising a plurality of variable beam splitters (VBS) to optically implement the first, second, and third SVD matrix;
propagating the first plurality of optical signals through the photonic processor;
detecting a second plurality of optical signals received from the photonic processor; and
determining an output vector based on the detected second plurality of optical signals, wherein the output vector represents a result of the matrix-vector multiplication.

### Claim 20 (depends on 19)

The method of claim 19, wherein the detecting of the second plurality of optical signals is performed using phase-sensitive detectors.

### Claim 21 (depends on 20)

The method of claim 20, further comprising:
providing first light, from a light source, to the optical encoder for encoding the first plurality of optical signals;
providing second light, from the light source, to the phase-sensitive detectors, wherein the second light is used as a local oscillator by the phase-sensitive detectors.

### Claim 22 (depends on 21)

The method of claim 21, wherein:
the local oscillator is phase coherent with each of the first plurality of optical signals; and
a first path length of the first plurality of optical signals from the light source to the phase-sensitive detectors is substantially equal to a second path length of the local oscillator from the light source to the phase-sensitive detectors.

### Claim 23 (depends on 19)

The method of claim 19, wherein encoding the input vector comprises:
encoding an absolute value of a vector component of the input vector into an amplitude of a respective optical signal of the first plurality of optical signals; and
encoding a phase of the vector component of the input vector into a phase of the respective optical signal of the first plurality of optical signals.

### Claim 24 (depends on 19)

The method of claim 19, wherein the matrix is an arbitrary unitary matrix.

### Claim 25 (depends on 19)

The method of claim 19, further comprising performing multiple matrix-vector multiplications simultaneously using wavelength division multiplexing.

### Claim 26 (depends on 25)

The method of claim 25, wherein:
the input vector is one of a plurality of input vectors:
the method further comprises encoding each of the plurality of input vectors into a respective one of a first plurality of optical signal of a particular wavelength, wherein each wavelength associated with each one of the first plurality of optical signals is different from the other wavelengths of the other ones of the first plurality of optical signals.

### Claim 27 (depends on 19)

The method of claim 19, wherein the photonic processor comprises:
a first array of interconnected variable beam splitters (VBSs) comprising a first plurality of optical inputs corresponding to the first plurality of input spatial modes and a first plurality of optical outputs;
a second array of interconnected VBSs comprising a second plurality of optical inputs and a second plurality of optical outputs corresponding to the plurality of output spatial modes; and
a plurality of controllable optical elements, each of the plurality of these controllable optical elements coupling a single one of the first plurality of optical outputs of the first array to a respective single one of the second plurality of optical inputs of the second array.

### Claim 28 (depends on 27)

The method of claim 27, wherein:
the first plurality of interconnected VBSs implements the first SVD matrix;
the second plurality of interconnected VBSs implements the second SVD matrix
the plurality of controllable optical elements implements the third SVD matrix, wherein the third SVD matrix is a diagonal matrix.

### Claim 29 (depends on 27)

The method of claim 27, wherein:
each of the VBSs of the first plurality of interconnected VBSs and each of the VBSs of the second plurality of interconnected VBSs is associated with a respective address; and
the VBSs of the first and/or second plurality are controlled by at least one digital to analog converter (DAC) that controls a plurality of the VBSs using the addresses.

### Claim 30 (depends on 7)

The method of claim 7, wherein the matrix-vector multiplication is one of a plurality of matrix-vector multiplications performed to perform matrix-matrix multiplication, wherein the matrix is a first matrix and the matrix-matrix multiplication comprises multiplication of a second matrix by the first matrix by, the method further comprising:
(a) determining a plurality of input vectors from each column of the second matrix;
(b) selecting an input vector from the plurality of input vectors;
(c) encoding the selected input vector into the first plurality of optical signals using the optical encoder;
(d) performing the plurality of operations on the first plurality of optical signals associated with the first input vector;
(e) detecting the second plurality of optical signals associated with the selected input vector;
(f) storing digital detection results based on the detected second plurality of optical signals;
(g) repeating acts (b)-(f) for the other input vectors of the plurality of input vectors;
(h) digitally combine the digital detection results to determine a resulting matrix resulting from the multiplication of the second matrix by the first matrix.