# US 2022/0044092 A1 · Coherent photonic computing architectures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044092 A1
- Document ID: US20220044092A1
- Status: Published application
- Kind: A1
- Application number: 17/395,903
- Filed: 2021-08-06
- Published: 2022-02-10
- Earliest claimed: 2020-08-06
- Assignees: Celestial AI Inc.
- Applicants: Celestial AI Inc.
- Inventors: Nikolaos Pleros; David Lazovsky; George Giamougiannis; Apostolos Tsakyridis; Angelina Totovic
- Web page: https://patentgrep.com/patent/US20220044092A1

## CPC

- H04B 10/27: Arrangements for networking
- G02F 1/225: in an optical waveguide structure
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/067: using optical means
- G06N 3/0675: {using electro-optical, acousto-optical or opto-electronic means}
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- H04B 10/07: Arrangements for monitoring or testing transmission systems; Arrangements for fault measurement of transmission systems
- H04Q 11/0005: {Switch and router aspects}
- G06N 3/048: Activation functions
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 10/40: Physical realisations or architectures of quantum processors or components for manipulating qubits, e.g. qubit coupling or qubit control
- H04B: TRANSMISSION
- G02F: OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04Q: SELECTING (switches, relays, selectors H01H; wireless communication networks H04W)

## Abstract

Disclosed are coherent photonic circuit architectures that optically implement linear algebraic computations. In neuromorphic applications of such photonic circuit architectures, individual neural network layers can be implemented by coherent optical linear neurons in a crossbar configuration, integrated with electronic circuitry at the interfaces between neural network layers to determine the neuron inputs to one layer based on the neuron outputs of the preceding layer. Wavelength division multiplexing can be used to efficiently implement certain specific network models, optionally in conjunction with electro-optic switches to render a generic hardware configuration programmable.

## Claims

### Claim 1

A computing system comprising:
a photonic integrated circuit (PIC) implementing one or more computational layers, the PIC comprising:
one or more coherent photonic crossbars each configured to modulate a plurality of optical input signals to impart computational weights associated with a plurality of sets of weights onto the optical input signals to create weighted optical input signals, and to coherently combine the weighted optical input signals to generate optical output signals encoding computational outputs associated with the plurality of sets of weights,
at least one of the one or more coherent photonic crossbars comprising a plurality of electronically controlled modulators to impart computational inputs onto coherent optical carrier signals to generate the plurality of optical input signals, and
at least one of the one or more coherent photonic crossbars comprising a plurality of photodetectors to generate analog electronic output signals based on the optical output signals; and
a mixed-signal electronic integrated circuit (EIC) comprising:
driver circuitry to control the modulators in the PIC in accordance with the computational inputs, and
processing circuitry to process the analog electronic output signals, the processing being performed at least partially digitally.

### Claim 2 (depends on 1)

The computing system of claim 1, wherein the one or more coherent photonic crossbars implement multiple computational layers.

### Claim 3 (depends on 2)

The computing system of claim 2, wherein the analog electronic output signals associated with one of the multiple computational layers are processed to generate control signals for the driver circuitry associated with the computational inputs of another one of the computational layers.

### Claim 4 (depends on 3)

The computing system of claim 3, wherein at least one of the one or more coherent photonic crossbars comprises a plurality of additional electronically controlled modulators to impart the computational weights onto the optical input signals, and wherein the EIC further comprises additional driver circuitry to control the additional modulators.

### Claim 5 (depends on 4)

The computing system of claim 4, wherein the one of the computational layers and the other one of the computational layers are implemented by a single one of the one or more coherent photonic crossbars, and wherein the additional driver circuitry controls the additional modulators in accordance with one set of computational weights to implement the one of the computational layers and in accordance with another set of computational weights to implement the other one of the computational layers.

### Claim 6 (depends on 4)

The computing system of claim 4, wherein the EIC further comprises memory storing the computational weights and digital-to-analog converters to generate, from the computational weights, control signals for the additional driver circuitry.

### Claim 7 (depends on 4)

The computing system of claim 4, wherein the additional electronically controlled modulators comprise pairs of an amplitude modulator and a phase shifter to impart signed computational weights onto electric fields of the optical input signals.

### Claim 8 (depends on 3)

The computing system of claim 3, wherein the one of the computational layers and the other one of the computational layers are implemented by two coherent photonic crossbars.

### Claim 9 (depends on 2)

The computing system of claim 2, wherein the multiple computational layers are implemented by multiple coherent photonic crossbars, and wherein the optical output signals of at least one of the coherent photonic crossbars are provided to another one of the coherent photonic crossbars as the optical input signals.

### Claim 10 (depends on 1)

The computing system of claim 1, further comprising at least one of optical amplifiers in the PIC to amplify the optical output signals or electronic amplifiers in the processing circuitry to amplify the analog electronic output signal.

### Claim 11 (depends on 1)

The computing system of claim 1, wherein the one or more computational layers represent one or more neural network layers, the plurality of sets of weights represent a plurality of neurons, and the computational outputs are linear neuron outputs.

### Claim 12 (depends on 11)

The computing system of claim 11, wherein the processing circuitry applies a non-linear electronic activation function to the analog electronic signals either in an analog domain or in a digital domain following analog-to-digital conversion.

### Claim 13 (depends on 11)

The computing system of claim 11, wherein the PIC further comprises non-linear activation units associated with the neurons of at least one of the one or more coherent photonic crossbars, the non-linear optical activation units configured to apply an optical non-linearity to the optical output signals encoding the linear neurons outputs to thereby generate optical output signals encoding non-linear neuron outputs.

### Claim 14 (depends on 11)

The computing system of claim 11, wherein the digital processing comprises a neural-network-layer operation.

### Claim 15 (depends on 14)

The computing system of claim 14, wherein the neural-network-layer operation implements one of a pooling layer or a fully connected layer.

### Claim 16 (depends on 11)

The computing system of claim 11, wherein the digital processing comprises an image pre-processing operation for image input to the one or more neural network layers or a post-processing operation of output generated by the one or more neural network layers.

### Claim 17 (depends on 16)

The computing system of claim 16, wherein the digital processing comprises adjusting stored neural network weights to enable structural sparsity.

### Claim 18 (depends on 11)

The computing system of claim 11, wherein at least one of the PIC or the EIC is configured to perform mixed-precision operations.

### Claim 19 (depends on 11)

The computing system of claim 11, wherein the EIC further comprises a graphic processing unit (GPU) core to perform at least part of the digital processing.

### Claim 20 (depends on 1)

The computing system of claim 1, wherein the EIC further comprises on-chip memory.

### Claim 21 (depends on 1)

The computing system of claim 1, further comprising an electronic interface between the PIC and the EIC, the electronic interface having a power consumption of less than 3 pJ per bit of data converted from an optical domain to a digital electronic domain or from a digital electronic domain to an optical domain.

### Claim 22 (depends on 1)

The computing system of claim 1, the PIC further comprising an optical splitter that receives carrier light from a light engine and splits it into the coherent optical carrier signals.

### Claim 23 (depends on 22)

The computing system of claim 22, wherein the light engine comprises a plurality of lasers generating light at multiple wavelengths and a multiplexer to combine the light at the multiple wavelengths into multiplexed carrier light.

### Claim 24 (depends on 1)

The computing system of claim 1, the PIC further comprising an optical interface that receives carrier light from a source external to the PIC, the optical interface comprising at least one of an edge coupler, inverted taper coupler, or grating coupler.

### Claim 25 (depends on 24)

The computing system of claim 24, wherein the optical interface comprises at least one of an edge coupler, an inverted taper coupler, or a grating coupler.

### Claim 26 (depends on 1)

The computing system of claim 1, wherein the digital processing comprises operations performed at higher bit precision than operations performed in the PIC.
