Three-dimensional convolution pipeline with memory organizer unit
AssigneeMeta Platforms, Inc.
InventorsDheevatsa Mudigere, Krishnakumar Nair, Abdulkadir Utku Diril
Life of this patent
Estimating the term…
Abstract
A processor system comprises a memory organizer unit and a matrix computing unit. The memory organizer unit is configured to receive a request for a three-dimensional data of a convolutional neural network layer. The requested three-dimensional data is obtained from a memory. The obtained three-dimensional data is rearranged in an optimized linear order and the rearranged data in the optimized linear order is provided to the matrix computing unit. The matrix computing unit is configured to perform at least a portion of a three-dimensional convolution using at least a portion of the provided rearranged data in the optimized linear order.
Claims20
A processor system, comprising:
- a memory organizer unit configured to:
- receive a request for a three-dimensional data of a data matrix for a convolutional neural network layer;
- obtain the requested three-dimensional data from a memory;
- rearrange the obtained three-dimensional data in an optimized linear order, including by being configured to align a slice of the obtained three-dimensional data of the data matrix different from a weight matrix with a linearized version of the weight matrix including by being configured to insert one or more zero-value elements between elements of one or more rows or columns of the slice of the obtained three-dimensional data to optimize data organization layout at a hardware component interfacing between the memory and a matrix computing unit;
- and
- provide to the matrix computing unit the rearranged data in the optimized linear order;
- and
- the matrix computing unit configured to perform at least a portion of a three-dimensional convolution using at least a portion of the provided rearranged data in the optimized linear order.
A method comprising:
- receiving a request for a three-dimensional data of a data matrix for a convolutional neural network layer;
- obtaining the requested three-dimensional data from a memory;
- rearranging the obtained three-dimensional data in an optimized linear order, including aligning a slice of the obtained three-dimensional data of the data matrix different from a weight matrix with a linearized version of the weight matrix including by inserting one or more zero-value elements between elements of one or more rows or columns of the slice of the obtained three-dimensional data to optimize data organization layout at a hardware component interfacing between the memory and a matrix computing unit;
- and
- providing to the matrix computing unit the rearranged data in the optimized linear order.
A method comprising:
- receiving a three-dimensional convolution operation instruction, wherein the three-dimensional convolution operation instruction specifies a three-dimensional data of a data matrix fora convolutional neural network layer;
- requesting the three-dimensional data of the convolutional neural network layer from a memory organizer unit;
- receiving a linearized version of a weight matrix of a convolutional filter;
- receiving in an optimized linear order the requested three-dimensional data from the memory organizer unit, wherein the optimized linear order aligns a slice of the three-dimensional data of the data matrix different from a weight matrix with a linearized version of the weight matrix by including one or more zero-value elements inserted between elements of one or more rows or columns of the slice of the three-dimensional data to optimize data organization layout at a hardware component interfacing between a memory and a matrix computing unit;
- and
- performing at least a portion of a three-dimensional convolution using at least a portion of the received requested three-dimensional data in the optimized linear order and at least a portion of the linearized version of the weight matrix.
Dependencies are inferred from the claim text.