US 2026/0087090 A1Published application

Convtranspose to conv transformation: mapping convtranspose on neural processing units accelerators for standard convolutions

AssigneeSTMicroelectronics International N.V.

InventorAlessandro BUSCHINI

Life of this patent

Estimating the term…

Loading patent…

Abstract

Disclosed herein are processing techniques that enable existing convolution hardware accelerators to efficiently handle transposed convolution (ConvTranspose) operations. The method addresses both the arithmetic differences between convolution and ConvTranspose operations and the impact of ConvTranspose-specific hyperparameters. The method includes obtaining original ConvTranspose hyperparameters, expanding input features when stride exceeds 1 with solutions that can be optimized by mapping tools and hardware DMA, rotating kernel weights, transposing weights for multi-channel operations, and computing new convolution hyperparameters. These steps transform the ConvTranspose operation into an equivalent standard convolution operation performable on existing hardware. The technique accounts for various hyperparameters including pads, strides, dilations, and groups, ensuring accurate replication of ConvTranspose behavior. By enabling ConvTranspose operations on standard convolution hardware, this approach enhances the versatility and efficiency of neural network accelerators without requiring specialized hardware.

Claims18

Dependent claims

A method for performing a transposed convolution (ConvTranspose) operation using a convolution hardware accelerator, the method comprising:

receiving input features for the ConvTranspose operation, wherein the input features are organized into one or more input channels, each channel representing a distinct feature map;
receiving at least one kernel comprising weights for the ConvTranspose operation, wherein the kernel is applied to the input features to produce output features;
obtaining original ConvTranspose hyperparameters, wherein the hyperparameters include:
pads defining additional zero-valued elements added to the borders of the input features, strides defining the step size for applying the kernel to the input features, dilations defining spacing between kernel elements, and groups defining how input channels and output channels are connected;
when a ConvTranspose stride is greater than 1, expanding the input features by interleaving zeros between each input feature value;
rotating weights of the kernel by 180 degrees;
when the ConvTranspose operation involves multiple input channels and multiple output channels, as determined by dimensions of the input features and the kernel, transposing weights based on the groups;
computing new convolution hyperparameters based on the original ConvTranspose hyperparameters;
when the ConvTranspose stride is greater than 1 and the ConvTranspose operation involves multiple input channels and multiple output channels, replacing the ConvTranspose operation with a standard convolution operation using the expanded input features, the rotated weights, the transposed weights, and the new convolution hyperparameters;
when the ConvTranspose stride is greater than 1 and the ConvTranspose operation does not involve multiple input channels and multiple output channels, replacing the ConvTranspose operation with a standard convolution operation using the expanded input features, the rotated weights, and the new convolution hyperparameters;
when the ConvTranspose stride is not greater than 1 and the ConvTranspose operation involves multiple input channels and multiple output channels, replacing the ConvTranspose operation with a standard convolution operation using the rotated weights, the transposed weights, and the new convolution hyperparameters;
and
when the ConvTranspose stride is not greater than 1 and the ConvTranspose operation does not involve multiple input channels and multiple output channels, replacing the ConvTranspose operation with a standard convolution operation using the rotated weights and the new convolution hyperparameters.

10 more elements

A hardware system for performing a transposed convolution (ConvTranspose) operation using standard convolution hardware, the system comprising:

a plurality of convolution accelerators (CAs) configured to perform standard convolution operations;
a stream switch configured to provide a reconfigurable interconnect framework for data flow between components;
a plurality of Direct Memory Access (DMA) controllers configured to manage data transfer between system memory and the CAs;
control registers configured to store configuration parameters for ConvTranspose operations;
a bus arbiter and system bus interface configured to manage communication between the hardware system and a System-on-Chip (SoC);
and
a processor configured to:
receive input features for the ConvTranspose operation;
receive at least one kernel comprising weights for the ConvTranspose operation;
obtain original ConvTranspose hyperparameters;
when a ConvTranspose stride is greater than 1, instruct the DMA controllers to expand the input features by interleaving zeros between each input feature value;
rotate weights of the kernel by 180 degrees;
when the ConvTranspose operation involves multiple input channels and multiple output channels, transpose weights based on a groups parameter;
compute new convolution hyperparameters based on the original ConvTranspose hyperparameters;
and
configure the CAs to perform a standard convolution operation using the expanded input features (when applicable), the rotated weights, the transposed weights (when applicable), and the new convolution hyperparameters.

13 more elements

Dependencies are inferred from the claim text.