Image processing apparatus, image processing method, and non-transitory computer-readable storage medium

An image processing method, comprising:

• obtaining pixel information from a pixel storage at a specific position connected to a multiplier the pixel storage being a part of a plurality of pixel storage units arranged in a two-dimensional array, each pixel storage unit comprising a multiplexer and a memory which stores pixel information of each pixel of a feature image;
• obtaining a coefficient from a coefficient filter;
• obtaining a dilation rate for a dilated convolution operation;
• performing the dilated convolution operation using the multiplier and an adder, based on the pixel information obtained, the coefficient obtained and the dilation rate obtained, wherein one common coefficient is input to a plurality of the multipliers,
• wherein
• the pixel storage is connected to a pixel storage which store an adjacent pixel in the feature image and a pixel storage which store a non-adjacent pixel in the feature image, and the multiplexer can shift pixel information among pixel storage by selecting a connection to a pixel storage;
• in a case where performing the dilated convolution with the dilation rate of two or more, the multiplexer, by selecting a connection to a pixel storage that stores pixel information in the feature image that are non-adjacent to the pixel information already obtained, shifts non-adjacent pixel information stored in the selected pixel storage to the pixel storage at the specific position, and
• in a case where performing the dilated convolution with the dilation rate of one, the multiplexer, by selecting a connection to a pixel storage that stores pixel information in the feature image that are adjacent to the pixel information already obtained, shifts adjacent pixel information stored in the selected pixel storage to the pixel storage at the specific position.

A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as execute an image processing method, the method comprising:

• obtaining pixel information, from a pixel storage at a specific position connected to a multiplier the pixel storage being a part of a plurality of pixel storage units arranged in a two-dimensional array, each pixel storage unit comprising a multiplexer and a memory which stores pixel information of each pixel of a feature image;
• obtaining a coefficient from a coefficient filter;
• obtaining a dilation rate for a dilated convolution operation;
• performing the dilated convolution operation using the multiplier and an adder, based on the pixel information obtained, the coefficient obtained and the dilation rate obtained, wherein one common coefficient is input to a plurality of the multipliers,
• wherein
• the pixel storage is connected to a pixel storage which store an adjacent pixel in the feature image and a pixel storage which store a non-adjacent pixel in the feature image, and the multiplexer can shift pixel information among pixel storage by selecting a connection to a pixel storage;
• in a case where performing the dilated convolution with the dilation rate of two or more, the multiplexer, by selecting a connection to a pixel storage that stores pixel information in the feature image that are non-adjacent to the pixel information already obtained, shifts non-adjacent pixel information stored in the selected pixel storage to the pixel storage at the specific position, and
• in a case where performing the dilated convolution with the dilation rate of one, the multiplexer, by selecting a connection to a pixel storage that stores pixel information in the feature image that are adjacent to the pixel information already obtained, shifts adjacent pixel information stored in the selected pixel storage to the pixel storage at the specific position.

An image processing apparatus performing convolution operation on a feature image using a filter comprising:

• one or more circuitry which function as:
• a pixel holding unit comprising a plurality of pixel storage units arranged in a two-dimensional array, each pixel storage unit comprising a multiplexer and a memory that stores pixel information of each pixel of a feature image, wherein the multiplexer connects the memory to a memory that stores adjacent pixel information and to a memory that stores non-adjacent pixel information;
• a filter coefficient holding unit configured to hold filter coefficients;
• a operation unit configured to obtain N pixel information from pixel storage units at specified positions among the plurality of pixel storage units, to multiply each of the N pixel information by a common coefficient retrieved from the filter coefficient holding unit using N multipliers, and to accumulate the N multiplication results;
• a dilation rate acquiring unit configured to acquire a dilation rate;
• and
• a controller configured to control the multiplexer based on the acquired dilation rate,
• wherein shift amounts of pixel information stored in the pixel holding unit, as performed by the controlled multiplexer, depend on the acquired dilation rate, and
• wherein in a case where the filter is dilated, the controller controls the multiplexer to connect to the memory that stores non-adjacent pixel information corresponding to the dilation rate.