Back to results
US 11,861,485 B2Granted
Data format transform method to improve AI engine MAC utilization
AssigneeBaidu USA LLC
InventorMin Guo
Life of this patent
Estimating the term…
Abstract
A data format converter rearranges data of an input image for input to a systolic array of multiply and accumulate processing elements. The image has a pixel height and a pixel width in a number of channels equal to a number of colors per pixel. The data format converter rearranges the data to a second, greater number of channels and inputs the second number of channels to one side of the systolic array. The second number of channels is less than or equal to the number of MAC PEs on the one side of the systolic array, and results in greater MAC PE utilization in the systolic array.
Claims17
An artificial intelligence (AI) engine, comprising:
- a systolic array of M by N MAC (multiply and accumulate) PEs (processing elements) having N MAC PEs on an input side of the systolic array;
- and
- a data format converter to rearrange data of an input image having a pixel height and a pixel width in a first number of channels corresponding to a number of colors per pixel to a second number of channels, wherein the data of the input image is rearranged based on a hardware design of the systolic array, wherein the second number of channels is greater than the first number of channels and is not greater than N, each having data of a lesser pixel height, a lesser pixel width, and one of the number of colors per pixel, wherein the second number of channels is obtained by arranging each of a plurality of pixel groups of the input image in each of the first number of channels, wherein when the first number of channels is three associated with red, green and blue channels respectively, the second number of channels can be twelve, obtained by arranging each of a plurality of four pixel by four pixel groups of the input image in each of red, green and blue channels into four two pixel by two pixel groups for each of red, green and blue, with each of the four two pixel by two pixel groups in each color presented in a corresponding channel for a total of twelve channels;
- wherein the systolic array is to receive and to process data of the second number of channels, such that more MAC PEs are utilized in a first and second convolution layer inference in the systolic array compared to inputting data of the first number of channels to the systolic array.
1 more element
A method of operating an artificial intelligence (AI) engine including a data format converter and a systolic array of M by N MAC (multiply and accumulate) PEs (processing elements) having N MAC PEs on an input side of the systolic array, and, comprising:
- arranging, based on a hardware design of the systolic array, data of an input image having a pixel height and a pixel width in a first number of channels corresponding to a number of colors per pixel to a second number of channels, wherein the second number of channels is greater than the first number of channels and is not greater than N, each having data of a lesser pixel height, a lesser pixel width, and one of the number of colors per pixel, wherein the second number of channels is obtained by arranging each of a plurality of pixel groups of the input image in each of the first number of channels, wherein
- when the first number of channels is three associated with red, green and blue channels respectively;
- the second number of channels can be twelve;
- and
- the arranging data of the input image comprises arranging each of a plurality of four pixel by four pixel groups in each of red, green and blue channels into four two pixel by two pixel groups for each of red, green and blue, and presenting each of the four two pixel by two pixel groups in each color in a corresponding channel for a total of twelve channels;
- and
- inputting data of the second number of channels to the systolic array, such that more MAC PEs are utilized in a first and second convolution layer inference in the systolic array compared to inputting data of the first number of channels to the systolic array.
4 more elements
A non-transitory computer-readable medium having instructions stored therein which, when executed by a processor, cause the processor to perform operations, the operations comprising:
- arranging data of an input image having a pixel height and a pixel width in a first number of channels corresponding to a number of colors per pixel to a second number of channels, wherein the data of the input image is rearranged based on a hardware design of a systolic array of M by N MAC (multiply and accumulate) PEs (processing elements) having N MAC PEs on an input side of the systolic array, wherein the second number of channels is greater than the first number of channels and is not greater than N, each having data of a lesser pixel height, a lesser pixel width, and one of the number of colors per pixel, wherein the second number of channels is obtained by arranging each of a plurality of pixel groups of the input image in each of the first number of channels, wherein
- when the first number of channels is three associated with red, green and blue channels respectively;
- the second number of channels can be twelve;
- and
- the arranging data of the input image comprises arranging each of a plurality of four pixel by four pixel groups in each of red, green and blue channels into four two pixel by two pixel groups for each of red, green and blue, and presenting each of the four two pixel by two pixel groups in each color in a corresponding channel for a total of twelve channels;
- and
- inputting data of the second number of channels to the systolic array, such that more MAC PEs are utilized in a first and second convolution layer inference in the systolic array compared to inputting data of the first number of channels to the systolic array.
4 more elements
Dependencies are inferred from the claim text.