Method, systems, and apparatuses for heatmap regression in a neural network
AssigneeSTMicroelectronics International N.V.
InventorsDavide Denaro, Claudio Domenico Marchisio
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatuses for heatmap regression for neural networks are provided, including for systems with restricted computational resources, such as embedded systems. An exemplary method may include utilizing a trained neural network having a plurality of pooling layers, a plurality of convolutional layers, and an output layer to generate a heatmap. The plurality of pooling layers may reduce the spatial resolution of an input image, though the resolution may not be reduced below the spatial resolution of the output layer. The plurality of convolutional layers may utilize a plurality of depthwise convolutions and pointwise convolutions that will be stacked to provide a plurality of bottlenecks.
Claims20
A method for heatmap regression with a neural network comprising:
- receiving an image;
- generating a heatmap from the image with the neural network by:
- reducing a spatial resolution of the image with one or more pooling layers, wherein a reduction in the spatial resolution is to a spatial resolution of an output layer;
- performing a plurality of convolution operations with a plurality of convolution layers, wherein the plurality of convolution layers each have the same spatial resolution, and wherein the neural network includes a plurality of depthwise convolution and pointwise convolution bottlenecks varying a plurality depth channels associated with the plurality of convolution layers;
- generating the output layer with a depth channel of a first depth resolution;
- and
- generating a heatmap with a plurality of key points, wherein a number of the plurality of key points equals the first depth resolution.
A system for heatmap regression with a neural network comprising:
- at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to:
- receive an image;
- generate a heatmap from the image with the neural network by:
- reduce a spatial resolution of the image with one or more pooling layers, wherein a reduction in the spatial resolution is to a spatial resolution of an output layer;
- perform a plurality of convolution operations with a plurality of convolution layers, wherein the plurality of convolution layers each have the same spatial resolution, and wherein the neural network includes a plurality of depthwise convolution and pointwise convolution bottlenecks varying a plurality depth channels associated with the plurality of convolution layers;
- generate the output layer with a depth channel of a first depth resolution;
- and
- generate a heatmap with a plurality of key points, wherein a number of the plurality of key points equals the first depth resolution.
A computer program product for heatmap regression with a neural network comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising an executable portion configured to:
- receive an image;
- generate a heatmap from the image with the neural network by:
- reduce a spatial resolution of the image with one or more pooling layers, wherein a reduction in the spatial resolution is to a spatial resolution of an output layer;
- perform a plurality of convolution operations with a plurality of convolution layers, wherein the plurality of convolution layers each have the same spatial resolution, and wherein the neural network includes a plurality of depthwise convolution and pointwise convolution bottlenecks varying a plurality depth channels associated with the plurality of convolution layers;
- generate the output layer with a depth channel of a first depth resolution;
- and
- generate a heatmap with a plurality of key points, wherein a number of the plurality of key points equals the first depth resolution.
Dependencies are inferred from the claim text.