# US 2015/0146915 A1 · Hardware convolution pre-filter to accelerate object detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0146915 A1
- Document ID: US20150146915A1
- Status: Published application
- Kind: A1
- Application number: 13/977,061
- Filed: 2012-12-18
- Published: 2015-05-28
- Earliest claimed: 2012-12-18
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Ya-Ti Peng; Yi-Jen Chiu; Jianguo Li
- Web page: https://patentgrep.com/patent/US20150146915A1

## CPC

- G06V 10/443: {by matching or filtering}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/2451: linear, e.g. hyperplane
- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06V 10/955: {using specific electronic processors}
- G06T 2207/20024: Filtering details
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems, apparatus, articles, and methods are described related to a hardware-based convolution pre-filter to accelerate object detection.

## Claims

### Claim at position 1

1-31. (canceled)

### Claim 32

A computer-implemented method for object detection comprising:
performing, via hardware, a convolution of an input image and a convolution kernel to generate a filtered image;
applying a threshold to the filtered image to generate a masked image having a plurality of masked pixels and a plurality of unmasked pixels; and
applying a cascade filter to individual pixels of the plurality of unmasked pixels of the masked image to determine one or more object detection regions related to one or more pixels passing the cascade filter.

### Claim 33 (depends on 32)

The method of claim 32, further comprising:
performing object recognition on the one or more object detection regions, wherein performing the object recognition comprises identifying an object in a first object detection region.

### Claim 34 (depends on 32)

The method of claim 32, further comprising:
receiving the input image;
merging a first object detection region and a second object detection region of the one or more object detection regions to form a merged object detection region; and
performing object recognition on the merged object detection region, wherein performing the object recognition comprises identifying an object in the merged object detection region.

### Claim 35 (depends on 32)

The method of claim 32, wherein the hardware comprises at least one of a digital signal processor, a graphics processing unit, or a field programmable gate array.

### Claim 36 (depends on 32)

The method of claim 32, wherein the cascade filter comprises at least one of a Viola-Jones cascade filter or a boosted cascade filter.

### Claim 37 (depends on 32)

The method of claim 32, wherein applying the cascade filter comprises determining, for a first stage of the cascade filter, whether a first unmasked pixel of the plurality of unmasked pixels passes the first stage; and
if the first unmasked pixel passes the first stage, transferring the first unmasked pixel to a second stage of the cascade filter; or
if the first unmasked pixel fails the first stage, rejecting the first unmasked pixel.

### Claim 38 (depends on 32)

The method of claim 32, wherein the convolution kernel comprises a pre-trained convolution kernel.

### Claim 39 (depends on 32)

The method of claim 32, wherein applying the threshold to the filtered image comprises at least one of applying a pre-trained threshold or applying a convolution results pre-trained threshold.

### Claim 40 (depends on 32)

The method of claim 32, wherein performing the convolution comprises performing the convolution of the input image on a pixel-by-pixel basis, and wherein applying the threshold to the filtered image comprises applying the threshold to the filtered image on a pixel-by-pixel basis.

### Claim 41 (depends on 32)

The method of claim 32, further comprising:
receiving the input image;
performing object recognition on the one or more object detection regions, wherein performing the object recognition comprises identifying an object in a first object detection region of the one or more object detection regions and wherein the object comprises at least one of a face, an eye, a landmark, a written character, a human, or an automobile; and
merging a second object detection region and a third object detection region of the one or more object detection regions to form a merged object detection region, and
wherein the input image comprises at least one of an image file or a video frame,
wherein the hardware comprises at least one of a digital signal processor, a graphics processing unit, or a field programmable gate array,
wherein the convolution kernel comprises a pre-trained convolution kernel, and wherein the pre-trained convolution kernel comprises at least one of a linear classifier trained by a large-scale training set, wherein the large scale training set comprises a plurality of images containing the object and a plurality of images not containing the object, or a designed convolution kernel,
wherein applying the threshold to the filtered image comprises at least one of applying a pre-trained threshold or applying a convolution results pre-trained threshold,
wherein performing the convolution comprises performing the convolution of the input image on a pixel-by-pixel basis,
wherein applying the threshold to the filtered image comprises applying the threshold to the filtered image on a pixel-by-pixel basis,
wherein the input image, the filtered image, and the masked image are a same size,
wherein the plurality of masked pixels are identified by a value of 1 and the unmasked pixels are identified by a value of 0,
wherein applying the cascade filter comprises applying a seven-stage cascade filter,
wherein applying the cascade filter comprises determining, for a first stage of the cascade filter, whether a first unmasked pixel of the plurality of unmasked pixels passes the first stage; and
if the first unmasked pixel passes the first stage, transferring the first unmasked pixel to a second stage of the cascade filter; or
if the first unmasked pixel fails the first stage, rejecting the first unmasked pixel,
wherein a first object detection region of the one or more object detection regions is related to a first passing pixel of the one or more pixels passing the cascade filter, and wherein the first passing pixel is in the center of the first object detection region,
wherein the cascade filter comprises at least one of a Viola-Jones cascade filter or a boosted cascade filter,
wherein applying the threshold to the filtered image comprises applying the threshold to the filtered image via at least one of the digital signal processor, the graphics processing unit, or the field programmable gate array,
wherein applying the cascade filter comprises applying the cascade filter via at least one of the graphics processing unit or a central processing unit,
wherein performing object recognition on the one or more object detection regions comprises performing object recognition on the one or more object detection regions via the central processing unit, and
wherein merging the first object detection region and the second object detection region comprises merging the first object detection region and the second object detection region via the central processing unit.

### Claim 42

A system for object detection on a computer comprising:
a hardware module configured to perform a convolution of an input image and a convolution kernel to generate a filtered image;
one or more processors communicatively coupled to the hardware module;
one or more memory stores communicatively coupled to the one or more processors;
a threshold module configured to apply a threshold to the filtered image to generate a masked image having a plurality of masked pixels and a plurality of unmasked pixels; and
a cascade filter module configured to perform, for individual pixels of the plurality of unmasked pixels, a cascade filter to determine one or more object detection regions related to one or more pixels passing the cascade filter.

### Claim 43 (depends on 42)

The system of claim 42, wherein the hardware module comprises a convolution module of dedicated hardware configured to perform the convolution of the input image and the convolution kernel.

### Claim 44 (depends on 42)

The system of claim 42, wherein the hardware module comprises at least one of a digital signal processor or a field programmable gate array, and wherein the hardware module comprises a convolution module of dedicated hardware configured to perform the convolution of the input image and the convolution kernel.

### Claim 45 (depends on 42)

The system of claim 42, further comprising:
a graphics processing unit communicatively coupled to the one or more processors, wherein the threshold module and the cascade filter module are implemented via the graphics processing unit.

### Claim 46 (depends on 42)

The system of claim 42, further comprising:
a graphics processing unit communicatively coupled to the one or more processors, wherein the threshold module is implemented via the graphics processing unit and the cascade filter module is implemented via the one or more processors.

### Claim 47 (depends on 42)

The system of claim 42, further comprising:
a merge module implemented via the one or more processors and configured to merge a first object detection region and a second object detection region of the one or more object detection regions to form a merged object detection region; and
an object recognition module implemented via the one or more processors and configured to perform object recognition on the one or more object detection regions.

### Claim 48 (depends on 42)

The system of claim 42, wherein the hardware module comprises a graphics processing unit, wherein the graphics processing unit comprises a convolution module of dedicated hardware configured to perform the convolution of the input image and the convolution kernel, and wherein the threshold module and the cascade filter module are implemented via the graphics processing unit.

### Claim 49 (depends on 42)

The system of claim 42, further comprising:
a merge module implemented via the one or more processors and configured to merge a first object detection region and a second object detection region of the one or more object detection regions to form a merged object detection region; and
an object recognition module implemented via the one or more processors and configured to perform object recognition on the one or more object detection regions, and
wherein the hardware module comprises a graphics processing unit, wherein the graphics processing unit comprises a convolution module of dedicated hardware configured to perform the convolution of the input image and the convolution kernel, and wherein the threshold module and the cascade filter module are implemented via the graphics processing unit.

### Claim 50 (depends on 42)

The system of claim 42, further comprising:
a merge module configured to merge a first object detection region and a second object detection region of the one or more object detection regions to form a merged object detection region; and
an object recognition module configured to perform object recognition on the one or more object detection regions, and
wherein the hardware module comprises a graphics processing unit, wherein the graphics processing unit comprises a convolution module of dedicated hardware configured to perform the convolution of the input image and the convolution kernel, and wherein the threshold module, the cascade filter module, the merge module, and the object recognition module are implemented via the graphics processing unit.

### Claim 51 (depends on 42)

The system of claim 42, further comprising:
an object recognition module configured to perform object recognition on the one or more object detection regions to identify an object in a first object detection region of the one or more object detection regions, wherein the object comprises at least one of a face, an eye, a landmark, a written character, a human, or an automobile;
a merge module configured to merge a second object detection region and a third object detection region of the one or more object detection regions to form a merged object detection region; and
wherein the input image comprises at least one of an image file or a video frame,
wherein the hardware module comprises at least one of a digital signal processor, a graphics processing unit, or a field programmable gate array,
wherein the convolution kernel comprises a pre-trained convolution kernel, and wherein the pre-trained convolution kernel comprises at least one of a linear classifier trained by a large-scale training set, wherein the large scale training set comprises a plurality of images containing the object and a plurality of images not containing the object, or a designed convolution kernel,
wherein the threshold module is further configured to apply at least one of a pre-trained threshold or a convolution results pre-trained threshold,
wherein the hardware module is further configured to perform the convolution of the input image on a pixel-by-pixel basis,
wherein the threshold module is further configured to apply the threshold to the filtered image on a pixel-by-pixel basis,
wherein the input image, the filtered image, and the masked image are a same size,
wherein the plurality of masked pixels are identified by a value of 1 and the plurality of unmasked pixels are identified by a value of 0,
wherein the cascade filter module comprises a seven-stage cascade filter,
wherein the cascade filter module is further configured to perform the cascade filter by determining, for a first stage of the cascade filter, whether a first unmasked pixel of the plurality of unmasked pixels passes the first stage; and
if the first unmasked pixel passes the first stage, transferring the first unmasked pixel to a second stage of the cascade filter; or
if the first unmasked pixel fails the first stage, rejecting the first unmasked pixel,
wherein a first object detection region of the one or more object detection regions is related to a first passing pixel of the one or more pixels passing the cascade filter, wherein the first object detection region has a size of 32 pixels by 32 pixels, and wherein the first passing pixel is in the center of the first object detection region, and
wherein the cascade filter comprises at least one of a Viola-Jones cascade filter or a boosted cascade filter.

### Claim 52

At least one machine readable medium comprising a plurality of instructions that in response to being executed on a computing device, cause the computing device to perform object detection by:
performing, via hardware, a convolution of an input image and a convolution kernel to generate a filtered image;
applying a threshold to the filtered image to generate a masked image having a plurality of masked pixels and a plurality of unmasked pixels; and
applying a cascade filter to individual pixels of the plurality of unmasked pixels of the masked image to determine one or more object detection regions related to one or more pixels passing the cascade filter.

### Claim 53 (depends on 52)

The machine readable medium of claim 52, further comprising instructions that in response to being executed on the computing device, cause the computing device to perform object detection by:
performing object recognition on the one or more object detection regions, wherein performing the object recognition comprises identifying an object in a first object detection region.

### Claim 54 (depends on 52)

The machine readable medium of claim 52, wherein the cascade filter comprises at least one of a Viola-Jones cascade filter or a boosted cascade filter.

### Claim 55 (depends on 52)

The machine readable medium of claim 52, wherein performing the convolution comprises performing the convolution of the input image on a pixel-by-pixel basis, and wherein applying the threshold to the filtered image comprises applying the threshold to the filtered image on a pixel-by-pixel basis.

### Claim 56 (depends on 52)

The machine readable medium of claim 52, further comprising instructions that in response to being executed on the computing device, cause the computing device to perform object detection by:
receiving the input image;
performing object recognition on the one or more object detection regions, wherein performing the object recognition comprises identifying an object in a first object detection region of the one or more object detection regions and wherein the object comprises at least one of a face, an eye, a landmark, a written character, a human, or an automobile; and
merging a second object detection region and a third object detection region of the one or more object detection regions to form a merged object detection region, and
wherein the input image comprises at least one of an image file or a video frame,
wherein the hardware comprises at least one of a digital signal processor, a graphics processing unit, or a field programmable gate array,
wherein the convolution kernel comprises a pre-trained convolution kernel, and wherein the pre-trained convolution kernel comprises at least one of a linear classifier trained by a large-scale training set, wherein the large scale training set comprises a plurality of images containing the object and a plurality of images not containing the object, or a designed convolution kernel,
wherein applying the threshold to the filtered image comprises at least one of applying a pre-trained threshold or applying a convolution results pre-trained threshold,
wherein performing the convolution comprises performing the convolution of the input image on a pixel-by-pixel basis,
wherein applying the threshold to the filtered image comprises applying the threshold to the filtered image on a pixel-by-pixel basis,
wherein the input image, the filtered image, and the masked image are a same size,
wherein the plurality of masked pixels are identified by a value of 1 and the unmasked pixels are identified by a value of 0,
wherein applying the cascade filter comprises applying a seven-stage cascade filter,
wherein applying the cascade filter comprises determining, for a first stage of the cascade filter, whether a first unmasked pixel of the plurality of unmasked pixels passes the first stage; and
if the first unmasked pixel passes the first stage, transferring the first unmasked pixel to a second stage of the cascade filter; or
if the first unmasked pixel fails the first stage, rejecting the first unmasked pixel,
wherein a first object detection region of the one or more object detection regions is related to a first passing pixel of the one or more pixels passing the cascade filter, and
wherein the first passing pixel is in the center of the first object detection region, wherein the cascade filter comprises at least one of a Viola-Jones cascade filter or a boosted cascade filter,
wherein applying the threshold to the filtered image comprises applying the threshold to the filtered image via at least one of the digital signal processor, the graphics processing unit, or the field programmable gate array,
wherein applying the cascade filter comprises applying the cascade filter via at least one of the graphics processing unit or a central processing unit,
wherein performing object recognition on the one or more object detection regions comprises performing object recognition on the one or more object detection regions via the central processing unit, and
wherein merging the first object detection region and the second object detection region comprises merging the first object detection region and the second object detection region via the central processing unit.
