US 11,538,170 B2 · Integrated interactive image segmentation

Abstract

Methods and systems are provided for optimal segmentation of an image based on multiple segmentations. In particular, multiple segmentation methods can be combined by taking into account previous segmentations. For instance, an optimal segmentation can be generated by iteratively integrating a previous segmentation (e.g., using an image segmentation method) with a current segmentation (e.g., using the same or different image segmentation method). To allow for optimal segmentation of an image based on multiple segmentations, one or more neural networks can be used. For instance, a convolutional RNN can be used to maintain information related to one or more previous segmentations when transitioning from one segmentation method to the next. The convolutional RNN can combine the previous segmentation(s) with the current segmentation without requiring any information about the image segmentation method(s) used to generate the segmentations.

Claims20

Dependent claims

A computer-implemented method, the method comprising:

receiving a first interaction with an image;
receiving a second interaction with the image subsequent to the first interaction;
based on the second interaction with the image, performing a segmentation of the image using a first segmentation method to generate a current probability distribution map;
combining a feature map of the image with the current probability distribution map and combining the feature map with a previous probability distribution map, the previous probability distribution map generated based on a previous segmentation of the image, the previous segmentation based on the first interaction with the image and performed using a second segmentation method different from the first segmentation method;
and
generating a segmentation mask based on a concatenation of the feature map combined with the current probability distribution map and the feature map combined with the previous probability distribution map.

3 more elements

One or more non-transitory computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

receiving a first interaction with an image;
based on the first interaction with the image, performing a first segmentation of the image using a first segmentation method to generate a previous probability distribution map;
receiving a second interaction with the image subsequent to the first interaction;
based on the second interaction with the image, performing a second segmentation of the image using a second segmentation method different from the first segmentation method to generate a current probability distribution map;
generating, using a neural network, a feature map of the image;
combining, using the neural network, the feature map with the current probability distribution map to generate a first feature and the feature map with the previous probability distribution map to generate a second feature, wherein the previous probability distribution map is represented using a hidden state of the neural network, wherein the neural network is trained to perform the combining of the feature map of the image with the current probability distribution map, and perform the combining the feature map with the previous probability distribution map, without knowledge of which segmentation method was used to generate the current probability distribution map and the previous probability distribution map;
and
generating a segmentation mask based on a concatenation of the first feature and the second feature.

5 more elements

Dependencies are inferred from the claim text.

Integrated interactive image segmentation

Life of this patent

Integrated interactive image segmentation

Life of this patent

Abstract

Claims20