Photoelectric computing unit, photoelectric computing array, and photoelectric computing method

A photoelectric computing method executed by a photoelectric computing unit, the photoelectric computing unit comprising at least one light emitting unit and at least one photoelectric computing unit, the photoelectric computing unit comprising one semiconductor multifunctional area structure, the semiconductor multifunctional area comprising a carrier control region, a coupled region and a photo-generated carrier collection and readout region, wherein the method comprises steps of:

• configuring the light emitting unit to emit light, irradiating the light onto the photoelectric computing unit, and generating, under the control of the carrier control region, photo-generated carriers in a collection region in a photo-generated carrier collection and readout region as a first operation quantity for the photoelectric computing unit;
• generating an electrical operation quantity in one region in the multifunctional area, and inputting corresponding carriers, the carriers being used as a second operation quantity for the photoelectric computing unit;
• affecting carriers in the photo-generated carrier readout region by using both the photo-generated carriers as the first operation quantity and the carriers as the second operation quantity, the affected carriers being used as the result of photoelectric operation;
• and
• outputting, from an output terminal of the readout region in the photo-generated carrier collection and readout region, the carriers as the result of photoelectric operation.

A neural network algorithm acceleration device, comprising a photoelectric computation unit, into which operation quantities are input in two manners, i.e., optical inputting and electrical inputting, the photoelectric computation unit comprising one semiconductor multifunctional area structure, wherein the semiconductor multifunctional area comprises at least one carrier control region, at least one coupled region and at least one photo-generated carrier collection and readout region, wherein:

• an operation quantity to be input optically, i.e., an optical input quantity, is input by converting incident photons into photo-generated carriers, and an operation quantity to be input electrically, i.e., an electrical input quantity, is input by directly injecting carriers;
• the carrier control region is configured to control and modulate carriers in the photoelectric computation unit and used as an electrical input terminal of the photoelectric computation unit to be input with one operation quantity as an electrical input quantity;
• or, the carrier control region is configured to just control and modulate carriers in the photoelectric computation unit, with an electrical input quantity being input from other regions;
• the coupled region is configured to connect a collection region with a readout region in the photo-generated carrier collection and readout region, so that photo-generated carriers generated by incidence of photons affect carriers in the photoelectric computation unit to form an operation relationship;
• and
• in the photo-generated carrier collection and readout region, the collection region is configured to absorb incident photons and collect the generated photo-generated carriers and is used as an optical input terminal of the photoelectric computation unit to be input with one operation quantity as an optical input quantity;
• the readout region is configured as an electrical input terminal of the photoelectric computation unit to be input with one operation quantity as an electrical input quantity and is also configured as an output terminal of the photoelectric computation unit to output carriers affected by the optical input quantity and the electrical input quantity as a unit output quantity;
• or, the electrical input quantity is input from other regions, and the readout region is used only as an output terminal of the photoelectric computation unit to output carriers affected by the optical input quantity and the electrical input quantity as a unit output quantity,
• the photoelectric computation unit further comprising one light emitting unit optically corresponding to the photoelectric computation unit, the light emitting unit being configured to emit light to generate photo-generated carriers in the photoelectric computation unit, the photo-generated carriers being used as an optical input quantity for the photoelectric computation unit and interacting with an electrical input quantity that is input into the photoelectric computation unit from the electrical input terminal to obtain a result of photoelectric operation,
• wherein the light emitting unit is driven by one signal conversion driver, and the signal conversion driver is configured to convert a digital signal into the pulse width of a driving current pulse of the light emitting unit and drive a light emitting array formed by a plurality of light emitting units;
• or, a particular related light emitting unit is driven by addressing so that the related light emitting unit generates an optical signal of a corresponding duration, and the optical signal is set as an optical input quantity for the corresponding photoelectric computation unit, and
• the photoelectric computation unit is arranged in a two-dimensional or three-dimensional array to form a photoelectric computation module so as to realize various particular operations,
• the photoelectric computation unit further comprising a plurality of light emitting units and a plurality of photoelectric computation units to form a serial matrix vector multiplier, each of the photoelectric computation units at least comprising a carrier control region, a coupled region and a photo-generated carrier collection and readout region, wherein:
• the light emitting units are configured to emit optical signals representing data in a matrix to be multiplied;
• the photoelectric computation units are arranged in a unit array having the same number of rows and columns as the matrix to be multiplied, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein output terminals of the photo-generated carrier collection and readout regions of all units in each column of the unit array are connected to each other, and the carrier control regions of all units in each row of the unit array are connected to each other;
• the carrier control regions of all units in each row of the unit array are configured to control and modulate carriers in the unit and input, by column, carriers representing elements in vectors, and are used as vector data input terminals of the matrix vector multiplier, wherein, after the data of each element of the vector is converted into binarized data, the carriers representing the binarized data are input serially and bitwise into the carrier control region of each row;
• the coupled regions of the photoelectric computation units are responsible for connecting the photo-generated carrier collection and readout regions, and are configured to affect carriers in the readout regions by using photo-generated carriers in the collection regions in the photo-generated carrier collection and readout regions;
• and
• in the photo-generated carrier collection and readout regions of the plurality of photoelectric computation units, each unit comprises one optical input terminal and at least one result output terminal, wherein the optical input terminals of the photoelectric computation units are configured to receive optical signals emitted by the corresponding light emitting units and input the data in the matrix, and are used as matrix data input terminals of the matrix vector multiplier;
• the result output terminals are configured to output carriers in the readout regions in the photo-generated carrier collection and readout regions affected by the matrix data and the vector data, the carriers are output in form of current under the drive of a constant voltage, and the current is then converged by column, and the result is output;
• the output result is shifted according to the input bits, and then accumulated to obtain a final result vector;
• or
• the photoelectric computation unit is used to form a parallel matrix vector multiplier and comprises a plurality of light emitting units and a plurality of photoelectric computation units, each of the photoelectric computation units at least comprising a carrier control region, a coupled region and a photo-generated carrier collection and readout region, wherein:
• the light emitting units are configured to emit optical signals representing data in a matrix to be multiplied;
• the photoelectric computation units are divided into multiple groups, and the units in each group are arranged in a unit array having the same number of rows and columns as the matrix to be multiplied, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein output terminals of the photo-generated carrier collection and readout regions of all units in each column of the unit array are connected to each other, and the carrier control regions of all units in each row of the unit array are connected to each other;
• the carrier control regions of all units in each row of each unit array are configured to control and modulate carriers in the unit and input, in parallel, by group and by column, carriers representing corresponding bit data of the elements in vectors after binarization, and the carrier control regions are used as vector data input terminals of the matrix vector multiplier;
• the coupled regions of the plurality of photoelectric computation units are responsible for connecting the photo-generated carrier collection and readout regions, and are configured to affect carriers in the readout regions by using photo-generated carriers in the collection regions in the photo-generated carrier collection and readout regions;
• and
• in the photo-generated carrier collection and readout regions of the plurality of photoelectric computation units, each unit comprises one optical input terminal and at least one result output terminal, wherein the optical input terminals of the plurality of photoelectric computation units are configured to receive the optical signals emitted by the plurality of light emitting units and input the data in the matrix, and are used as matrix data input terminals of the matrix vector multiplier;
• the plurality of result output terminals are configured to output carriers in the readout regions in the photo-generated carrier collection and readout regions affected by the matrix data and the vector data, the carriers are output in form of current under the drive of a constant voltage, the current is then converged by column, shifted according to the input bits and accumulated to obtain a final result vector,
• the photoelectric computation unit is used to form an average pooling calculator that performs a pooling operation on a matrix at least comprising two elements, the photoelectric matrix vector multiplier comprising one matrix input terminal, one vector input terminal and a result output terminal, wherein:
• the photoelectric matrix vector multiplier, having one column and a same number of rows as the number of elements in the matrix to be pooled, is used as an average pooling calculator for the matrix to be pooled;
• the vector input terminal of the photoelectric matrix vector multiplier, as an electrical input terminal to be input with different elements in the matrix to be pooled, is used as a to-be-pooled matrix input terminal of the pooling unit;
• and, the matrix input terminal, as an optical input terminal to be input with a matrix, which has one column and a same number of rows as the number of elements in the matrix to be pooled and in which each element is a reciprocal of the number of the elements in the matrix to be pooled, is used as an average denominator input terminal;
• and
• the result output terminal is configured to output a final average pooling result of the matrix to be pooled, and
• the photoelectric computation unit forms a serial convolver, each of the photoelectrical computation units at least comprising a carrier control region, a coupled region and a photo-generated carrier collection and readout region, wherein:
• the light emitting units are configured to emit optical signals representing data in a convolution kernel;
• the photoelectric computation units are arranged into an array having the same number of rows and columns as the convolution kernel, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein the output terminals of the photo-generated carrier collection and readout regions of all units in the unit array are connected to each other to form an output terminal;
• the carrier control regions of the photoelectric computation units in the array are configured to control and modulate carriers in the unit, and serially input carriers representing corresponding bit data of the elements in a matrix to be convoluted obtained after zero padding and the elements in a small matrix corresponding to the original position of the convolution kernel, and are used as to-be-convoluted matrix data input terminals of the convolver;
• and, the data in the small matrix is converted into binarized data, and the carriers representing the binarized data are input serially and bitwise into the carrier control regions of the units;
• the coupled regions of the photoelectric computation units in the array connect the photo-generated carrier collection and readout regions, and are configured to make affect carriers in the readout regions by using photo-generated carriers in the collection regions in the photo-generated carrier collection and readout regions;
• and
• the photo-generated carrier collection and readout region of each photoelectric computation unit in the array comprises one optical input terminal and at least one result output terminal, wherein the optical input terminal is configured to receive the optical signal emitted by the light emitting unit and input the data in the convolution kernel, and is used as a convolution kernel input terminal of the convolver;
• the output terminal of each unit is configured to output carriers in the readout region in the photo-generated carrier collection and readout region affected by the corresponding bit data of the small matrix split from the matrix to be convoluted and the convolution kernel data;
• the carriers are output in form of current under the drive of a constant voltage, and the output current from all the photoelectric computation units in the array are converged by the output terminals connected in parallel, then shifted according to the input bits and accumulated to obtain a result of convolution operation corresponding to the current position of the convolutional kernel;
• the convolution kernel is moved according to the step size required by the convolution operation, data of the small matrix split from the data of the matrix to be convoluted corresponding to the current position of the convolution kernel is input again to obtain an output value, and then the convolution kernel is continuously moved until the whole convolution operation is completed;
• and, all the output values are recombined to obtain a matrix having a corresponding dimension, i.e., a final result of convolution operation;
• or
• the photoelectric computation unit forms a parallel convolver, the photoelectrical computation unit at least comprising a carrier control region, a coupled region and a photo-generated carrier collection and readout region, wherein:
• the light emitting units are configured to emit optical signals representing data in a convolution kernel;
• the photoelectric computation units are divided into multiple groups, and the photoelectric computation units in each group are arranged in an array having the same number of rows and columns as the convolution kernel, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein the output terminals of the photo-generated carrier collection and readout regions of all units in the array of each group of photoelectric computation units are connected to each other to form an output terminal;
• the carrier control regions of the photoelectric computation units in the array of each group of photoelectric computation units are configured to control and modulate carriers in the unit, and serially input carriers representing corresponding bit data of the elements in a matrix to be convoluted obtained after zero padding and the elements in a small matrix corresponding to the original position of the convolution kernel, and are used as to-be-convoluted matrix data input terminals of the convolver;
• and, the data in the small matrix is converted into binarized data, and the corresponding bit data is input into the unit array of the corresponding group;
• the coupled regions of the photoelectric computation units in the array of each group of photoelectric computation units are responsible for connecting the photo-generated carrier collection and readout regions, and are configured to affect carriers in the readout regions by using photo-generated carriers in the collection regions in the photo-generated carrier collection and readout regions;
• and
• the photo-generated carrier collection and readout region of each photoelectric computation unit in the array of each group of photoelectric computation units comprises one optical input terminal and at least one result output terminal, wherein the optical input terminal is configured to receive the optical signal emitted by the light emitting unit and input the data in the convolution kernel, and is a convolution kernel input terminal of the convolver, and the arrays of different groups of photoelectric computation units receive the same optical input data;
• the output terminal of each unit is configured to output carriers in the readout region in the photo-generated carrier collection and readout region affected by the corresponding bit data of the small matrix split from the matrix to be convoluted and the convolution kernel data;
• the carriers are output in form of current under the drive of a constant voltage, and the output current from all the photoelectric computation units in the array are converged by the output terminals connected in parallel, then shifted according to the input bits and accumulated to obtain a result of convolution operation corresponding to the current position of the convolutional kernel;
• the convolution kernel is moved according to the step size required by the convolution operation, data of the small matrix split from the data of the matrix to be convoluted corresponding to the current position of the convolution kernel is input again to obtain an output value, and the convolution kernel is continuously moved until the whole convolution operation is completed;
• and, all the output values are recombined to obtain a matrix having a corresponding dimension, i.e., a final result of convolution operation, wherein:
• a matrix input terminal of the matrix vector multiplier is configured to be input with a network weight, and output data or initial data of an upper-level network is input from a vector input terminal;
• an average denominator input terminal of the average pooling calculator is configured to be input with a reciprocal of the number of elements in a matrix to be pooled, and a to-be-pooled matrix input terminal is configured to be input with data to be pooled;
• a convolution kernel input terminal of the convolver is configured to be input with data in a convolution kernel, and small matrix data in a matrix to be convoluted corresponding to the current position of the convolution kernel is input from a to-be-convoluted matrix input terminal;
• and
• a nonlinear function module consists of electrical computation components, and is configured to operate nonlinear functions;
• and
• a general logic operation module comprises an electrical operator and/or is based on the photoelectric computing unit, and is configured to integrate and control operation functions of the matrix vector multiplier, the average pooling calculator and the convolver.

A method for accelerating neural network operations, which uses a photoelectric computation method executed by a photoelectric computation unit, the photoelectric computation unit comprising at least one light emitting unit and at least one photoelectric computation unit, the photoelectric computation unit comprising one semiconductor multifunctional area structure, the semiconductor multifunctional area comprising a carrier control region, a coupled region and a photo-generated carrier collection and readout region, wherein the method comprises steps of:

• configuring the light emitting unit to emit light, irradiating the light onto the photoelectric computation unit, and generating, under the control of the carrier control region, photo-generated carriers in a collection region in a photo-generated carrier collection and readout region as a first operation quantity for the photoelectric computation unit;
• generating an electrical operation quantity in one region in the multifunctional area, and inputting corresponding carriers, the carriers being used as a second operation quantity for the photoelectric computation unit;
• affecting carriers in the photo-generated carrier readout region by using both the photo-generated carriers as the first operation quantity and the carriers as the second operation quantity, the affected carriers being used as the result of photoelectric operation;
• and
• outputting, from an output terminal of the readout region in the photo-generated carrier collection and readout region, the carriers as the result of photoelectric operation,
• the photoelectric computing method is used to perform a matrix vector multiplication operation, wherein:
• photo-generated carriers generated by photons emitted by the light emitting units are used as optical input terminal data of a matrix vector multiplier, i.e., matrix data to be multiplied;
• the photoelectric computation units are arranged in a unit array having the same number of rows and columns as the matrix to be multiplied, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein output terminals of the photo-generated carrier collection and readout regions of all units in each column of the unit array are connected to each other, and the carrier control regions of all units in each row of the unit array are connected to each other;
• each element of the vector data is converted into binarized data, and data not representing different elements are injected into different rows serially from the connected carrier control regions in the same row, in form of modulated carriers, in an order from high to low bits, as vector data to be multiplied;
• and
• in the carrier collection and readout regions, the photo-generated carriers representing the matrix data and the carriers in the carrier control regions representing the binarized data at different bits of the vector data affect the carriers in the readout regions in the corresponding photo-generated carrier collection and readout regions, and the affected carriers are output in form of current under the drive of a constant voltage, the current is then converged by column and shifted and accumulated to obtain a final multiplication result vector;
• or
• the photoelectric computation method is used to perform a matrix vector multiplication operation, wherein:
• photo-generated carriers generated by photons emitted by the plurality of light emitting units are used as optical input terminal data of a matrix vector multiplier, i.e., matrix data;
• the plurality of photoelectric computation units are divided into multiple groups, and the units in each group are arranged in a unit array having the same number of rows and columns as the matrix to be multiplied, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein output terminals of the photo-generated carrier collection and readout regions of all units in each column of the unit array are connected to each other, and the carrier control regions of all units in each row of the unit array are connected to each other;
• each element of the vector data is converted into binarized data, and data representing different bits of different elements are injected into different rows of different groups serially from the connected carrier control regions in the same row in the corresponding group in form of modulated carriers in parallel, as vector data;
• and
• in the plurality of carrier collection and readout regions, the photo-generated carriers representing the matrix data and the carriers in the carrier control regions representing the binarized data at different bits of the vector data affect the carriers in the readout regions in the plurality of corresponding photo-generated carrier collection and readout regions, and the affected carriers are output in form of current under the drive of a constant voltage, the current is then converged by column and shifted and accumulated to obtain a final result vector,
• the photoelectrical computation method is used to perform an average pooling operation, to perform an average pooling operation on a matrix having at least 2 elements, wherein:
• a matrix to be pooled is split into independent elements, and all the split elements are recombined to obtain a vector having a dimension the same as the number of elements in the matrix to be pooled;
• the matrix vector multiplier is suitable for performing a matrix vector operation on a matrix to be multiplied having a same number of as the number of elements in the matrix to be pooled and one column;
• the vector recombined after splitting the matrix to be pooled is input into the matrix vector multiplier as a vector input terminal data, i.e., electrical input terminal data of the matrix vector multiplier;
• the matrix, in which the dimension is the same as the used matrix vector multiplier and each element is the reciprocal of the number of elements in the matrix to be pooled, is input into the matrix vector multiplier as the matrix input terminal data, i.e., optical input terminal data of the matrix vector multiplier, as the average denominator in the average pooling operation;
• and
• the output result from the result output terminal is a result of dividing each element in the matrix to be pooled by the number of elements and then adding, i.e., a result of average pooling of the matrix to be pooled, and
• the photoelectrical computation method is used to perform a convolution operation, wherein:
• photo-generated carriers generated by photons emitted by the light emitting units are used as the optical input terminal data of a convolver, i.e., convolution kernel data;
• the photoelectric computation units are arranged in a unit array having the same number of rows and columns as the convolution kernel for the convolution operation, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein the output terminals of the readout regions in all the photo-generated carrier collection and readout regions of the photoelectric computation units in the unit array are connected to form an output terminal;
• a small matrix having a dimension the same as the size of the convolution kernel is split from the matrix to be convoluted after zero padding according to the current position of the convolution kernel, and each element in the small matrix is converted into binarized data and then serially input into each unit from the carrier control region in form of modulated carriers in an order from high to low bits to serve as data of the matrix to be convoluted;
• in the carrier collection and readout regions of the units in the array, the photo-generated carriers representing the corresponding bit data of the small matrix split from the matrix to be convoluted and the carriers in the carrier control regions representing the convolution kernel data affect carriers in the readout regions in the corresponding photo-generated carrier collection and readout regions, and the affected carriers are output in form of current under the drive of a constant voltage, the current is then converged, and shifted and accumulated to obtain a result of convolution operation corresponding to the current position of the convolution kernel;
• and
• the convolution kernel is moved according to the step size required by the convolution operation, data of the small matrix split from the data of the matrix to be convoluted corresponding to the current position of the convolution kernel is input again to obtain an output value, and the convolution kernel is continuously moved until the whole convolution operation is completed;
• and, all the output values are recombined to obtain a matrix having a corresponding dimension, i.e., a final result of convolution operation;
• or
• the photoelectric computation method is used to perform a convolution operation, wherein:
• photo-generated carriers generated by photons emitted by the light emitting units are used as optical input terminal data of a convolver, i.e., convolution kernel data;
• the photoelectric computation units are divided into multiple groups, and the photoelectric computation units in each group are arranged in an array having the same number of rows and columns as the convolution kernel, without changing the optical correspondence between the light emitting units and the photoelectric computation units, wherein the output terminals of the readout regions in all the photo-generated carrier collection and readout regions of the photoelectric computation units in the unit array of each group are connected to form an output terminal;
• a small matrix having a dimension the same as the size of the convolution kernel is split from the matrix to be convoluted after zero padding according to the current position of the convolution kernel, and each element in the small matrix is converted into binarized data, and the corresponding bit data is input, in parallel, into each unit from the carrier control region of the array of the corresponding group in form of modulated carriers to serve as data of the matrix to be convoluted;
• in the carrier collection and readout regions of the photoelectric computation units, the photo-generated carriers representing the corresponding bit data of the small matrix split from the matrix to be convoluted and the carriers in the carrier control regions representing the convolution kernel data affect carriers in the readout regions in the corresponding photo-generated carrier collection and readout regions, the affected carriers are output in form of current under the drive of a constant voltage, and all the output current of the array of each group of photoelectric computation units is converged, and shifted and accumulated to obtain a result of convolution operation corresponding to the current position of the convolution kernel;
• and
• the convolution kernel is moved according to a step size required by the convolution operation, data of the small matrix split from the data of the matrix to be convoluted corresponding to the current position of the convolution kernel is input again to obtain an output value, and the convolution kernel is continuously moved until the whole convolution operation is completed;
• and, all the output values are recombined to obtain a matrix having a corresponding dimension, i.e., a final result of convolution operation,
• wherein the method comprises steps of:
• inputting a network weight from a matrix input terminal of a related matrix vector multiplier, and inputting, from a vector input terminal of the matrix vector multiplier, output data or initial data of an upper-level network;
• inputting, from an average denominator input terminal of a related average pooling calculator, a reciprocal of the number of elements in a matrix to be pooled, and inputting data to be pooled from a to-be-pooled matrix input terminal of the average pooling calculator;
• inputting data in a convolution kernel from a convolution kernel input terminal of a related convolver, and inputting, from a to-be-convoluted matrix input terminal of the convolver, small matrix data in a matrix to be convoluted corresponding to the current position of the convolution kernel;
• using a nonlinear function module including electrical computation components to operate nonlinear functions;
• and
• using a general logic operation module, comprising an electrical operator and/or is based on the photoelectric computing unit.