Methods for generating a deep neural net and for localising an object in an input image, deep neural net, computer program product, and computer-readable storage medium

A method for localizing any object of a predetermined type in an input image, the method comprising:

• providing a deep neural net that combines parts of:
• a discriminative counting model trained to classify images according to a number of objects of the predetermined type depicted in each of the images, wherein the discriminative counting model is trained for at least two different classes corresponding to different numbers of objects of the predetermined type;
• and
• a segmentation model trained to segment images by classification of each pixel of a respective image according to what part of the respective image the respective pixel belongs to,
• wherein the discriminative counting model and the segmentation model are trained in combination with each other, the training comprising:
• arranging a counting model head and a segmentation model head in parallel with each other downstream of a shared feature extractor for the discriminative counting model and the segmentation model, wherein the counting model head comprises at least one fully connected layer, the segmentation model head comprises at least one transposed convolutional layer, at least one convolutional layer, or the at least one transposed convolutional layer and the at least one convolutional layer, and the shared feature extractor comprises multiple convolutional layers;
• and
• feeding training images through the shared feature extractor to each of the model heads;
• providing the input image as an input to the deep neural net;
• capturing a map generated by the deep neural net as an output for the input image by processing the input image through the deep neural net, wherein any objects of the predetermined type depicted in the input image are indicated in the captured map;
• and
• providing an upsampling module after a final convolutional layer of the deep neural net, wherein the upsampling module comprises an image processing algorithm based neither on machine learning nor on a neural net architecture, wherein the upsampling module is configured to take the generated map as an input and to generate, from the generated map, an upsampled map.

A controller comprising:

• a processor configured to run a deep neural net for localizing objects of a predetermined type in an input image,
• wherein the deep neural net is a combination of parts of a discriminative deep counting model and a segmentation model, wherein the deep neural net is structured, such that by process of the respective input image, a respective corresponding map indicating locations of any objects of the predetermined type for each processed input image is generated,
• wherein the discriminative deep counting model is trained to classify images according to a number of objects of the predetermined type depicted in each of the images, wherein the discriminative deep counting model is trained for at least two different classes corresponding to different numbers of objects of the predetermined type,
• wherein the deep segmentation model is trained to segment images by classification of each pixel of a respective image according to what part of the respective image the respective pixel belongs to, and
• wherein an upsampling module is added after a final convolutional layer of the deep neural net, wherein the upsampling module comprises an image processing algorithm based neither on machine learning nor on a neural net architecture, wherein the upsampling module is configured to take the generated map as an input and to generate, from the generated map, an upsampled map.

In a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to generate a deep neural net for localizing objects of a predetermined type in an input image, the instructions comprising:

• training a discriminative deep counting model to classify images according to a number of objects of the predetermined type depicted in each of the images, wherein the discriminative deep counting model is trained for at least two different classes corresponding to different numbers of objects of the predetermined type;
• training a deep segmentation model to segment images, the training of the deep segmentation model comprising classifying each pixel of a respective image according to what part of the respective image the respective pixel belongs to;
• forming the deep neural net, the forming of the deep neural net comprising combining parts of the discriminative deep counting model and the segmentation model to form the deep neural net, wherein the deep neural net is structured, such that by processing the respective input image, a respective corresponding map indicating locations of any objects of the predetermined type for each processed input image is generated;
• and
• adding an upsampling module after a final convolutional layer of the deep neural net, wherein the upsampling module comprises an image processing algorithm based neither on machine learning nor on a neural net architecture, wherein the upsampling module is configured to take the generated map as an input and to generate, from the generated map, an upsampled map.