Back to results
US 12,717,867 B2Granted
Method and apparatus with network interoperability processing
AssigneeSamsung Electronics Co., Ltd.
InventorsHyunsun Park, Yoojin Kim, Junwoo Jang
Life of this patent
Estimating the term…
Abstract
A method of processing data includes manipulating input data based on a configuration of the input data and a configuration of hardware for processing the input data to generate manipulated data; rearranging the manipulated data based on sparsity of the manipulated data to generate rearranged data; and processing the rearranged data to generate output data.
Claims22
A hardware-accelerated processor-implemented method of processing neural network data for performing voice recognition, image recognition, or image classification, the method comprising:
- receiving, by a processor including an operator implemented in hardware as a neural network hardware accelerator having a second number of channels, input data according to an input neural network task to be analyzed by the operator, the input data being represented as a matrix having a first number of input channels, the second number of channels being determined by a hardware configuration of the operator;
- performing a convolution operation on the input data to generate input feature map data;
- generating manipulated data from the input feature map data, based on a comparison result of the first number of input channels of the input feature map data with the second number of channels of the operator implemented in the hardware, the generating comprising:
- adding one or more channels configured by zeros to the input feature map data to increase the first number of input channels to match the second number of channels of the operator so that idle channels in the operator are avoided;
- shifting elements of one or more columns included in the input feature map data;
- and
- rearranging the manipulated data based on sparsity of the manipulated data to generate rearranged data to optimize operations of the operator, the rearranging comprising:
- shifting elements of the input feature map data within the manipulated data to be positioned within the added one or more channels;
- and
- shifting elements of one or more columns included in the manipulated data to reduce blanks represented by zeros;
- and
- performing, by the operator implemented in the hardware, convolution operations on the rearranged data using each channel of the second number of channels of the operator to obtain a resulting analysis of the input data while reducing unnecessary operations caused by mismatched channels and sparsity.
9 more elements
A hardware-accelerated apparatus for processing data for performing image recognition or image classification, the apparatus comprising:
- a memory in which at least one program is stored;
- and
- a processor configured, the processor including an operator implemented in hardware as a neural network hardware accelerator having a second number of channels, to execute the at least one program to:
- receiving input image data according to a recognition task to be analyzed by the operator implemented in the hardware, the input image data being represented as a matrix having a first number of input channels, the second number of channels being determined by a hardware configuration of the operator;
- performing a convolution operation on the input image data to generate input feature map data;
- generate manipulated data from the input feature map data, based on a comparison result of the first number of input channels of the input feature map data with the second number of channels of the operator implemented in the hardware, the generating comprising:
- adding one or more channels configured by zeros to the input feature map data to increase the first number of input channels to match the second number of channels of the operator so that idle channels in the operator are avoided;
- and
- shifting elements of one or more columns included in the input feature map data;
- and
- rearrange the manipulated data based on sparsity of the manipulated data to generate rearranged data to optimize operations of the operator, the rearranging comprising:
- shifting elements of the input feature map data within the manipulated data to be positioned within the added one or more channels;
- and
- shifting elements of one or more columns included in the manipulated data to reduce blanks represented by zeros;
- and
- perform, by the operator implemented in the hardware, convolution operations on the rearranged image data using each channel of the second number of channels of the operator to generate recognition results while reducing unnecessary operations caused by mismatched channels and sparsity.
13 more elements
A hardware-accelerated apparatus for performing voice recognition, comprising:
- a memory;
- and
- a processor, the processor including an operator implemented in hardware as a neural network hardware accelerator having a second number of channels, configured to execute at least one program stored in the memory to:
- obtaining, by the operator implemented in the hardware, audio data for a neural network trained to perform a voice recognition task on the audio data;
- perform a convolution operation on the audio data to generate input feature map data, the input feature map data being represented as a matrix having a first number of input channels;
- manipulate the input feature map data to generate first data based on a comparison result of the first number of input channels of the input feature map data with the second number of channels of the operator implemented in the hardware based on a value obtained by dividing the second number by the first number, the value indicating that the first number is not a multiple of the second number, the manipulating of the input feature map data comprising:
- adding at least one input channel configured by zeros to the input feature map data based on the comparison result indicating that an idle operator channel exists so that the idle operator channel in the operator is avoided;
- and
- shifting elements of one or more columns included in the input feature map data;
- generate second data by rearranging the first data based on a validity of elements included in the first data to reduce blanks represented by zeros;
- perform, by the operator implemented in the hardware, one or more convolution operations in the neural network on the second data using each channel of the second number of channels of the operator to generate recognition results for the audio data while reducing unnecessary operations caused by mismatched channels and sparsity;
- and
- perform the voice recognition task on the audio data based on the recognition result from the operator.
10 more elements
Dependencies are inferred from the claim text.