Stream processing-based non-blocking ORB feature extraction accelerator implemented by FPGA

A stream processing-based non-blocking oriented FAST and rotated BRIEF (ORB) feature extraction accelerator implemented by a field programmable gate array (FPGA), comprising:

• a downsampling module configured to adjust an input image to a required size to obtain an original image;
• a Gaussian filtering module configured to blur the original image to obtain a blurred image;
• a corner detection module configured to determine a position of a feature point in the blurred image and output a mask of the feature point;
• a non-maximum suppression (NMS) module configured to perform NMS on the mask of the feature point, wherein the stream processing-based non-blocking ORB feature extraction accelerator further comprises:
• a rotated binary robust independent elementary feature (rBRIEF) computation module configured to perform window column selection, rBRIEF computation, and window column First-In First-Out (FIFO) in parallel by using a cache management algorithm, wherein
• a working region of a computation module of an rBRIEF descriptor is a 37×37 sliding window, and the 37×37 sliding window is updated in each clock cycle;
• whenever the 37×37 sliding window is updated, an intermediate pixel of a leftmost column of pixels, namely left pixel, and an intermediate pixel of a rightmost column of pixels, namely right pixel, in the mask of the feature point are detected;
• when the right pixel is a feature point, an intermediate column of the blurred image is considered as a first column of a window, the intermediate column of the blurred image is written into a window column FIFO queue, a counter is reset, and next 36 intermediate columns are further written into the window column FIFO queue;
• and when the left pixel is a feature point, the intermediate column of the blurred image is considered as a last column of a window, a label is attached when each intermediate column is written into the window column FIFO queue, to indicate whether the intermediate column is a start column or an end column, and the rBRIEF computation module determines whether to perform the rBRIEF computation based on the label;
• and
• the rBRIEF computation module reads data from the window column FIFO queue and concatenates the read data into another window matrix;
• a new column is inserted at an end of the window matrix in each clock cycle, and other columns of the window matrix are sequentially shifted leftwards;
• window concatenation operates at a throughput of one column per cycle until a predetermined column is an end column;
• the reading of the window column FIFO queue is stopped, and the rBRIEF computation is performed;
• and when the rBRIEF computation is performed, the original image is configured to compute a direction of a centroid, and a BRIEF coordinate is rotated based on an angle of the direction to compute the rBRIEF descriptor;
• and
• a sorting module configured to perform hardware implemented count sorting.