Learning apparatus, learning method, and learning program, region-of-interest extraction apparatus, region-of-interest extraction method, and region-of-interest extraction program, and learned extraction model

A learning apparatus of an extraction model that extracts a region of interest from images having a plurality of different representation formats comprising at least one processor,

• wherein the extraction model has
• an encoder that extracts a feature amount of a first image of a first representation format to derive a feature map of the first image,
• a first decoder that derives a second virtual image of a second representation format different from the representation format of the first image on the basis of the feature map,
• a first discriminator that discriminates a representation format of an input image and whether the input image is a real image or a virtual image generated by the first decoder, and outputs a first discrimination result,
• a second decoder that extracts a region of interest of the first image on the basis of the feature map, and
• a third discriminator that discriminates a representation format of the first image of the first representation format that is input to the encoder on the basis of the feature map, and outputs a third discrimination result,
• wherein the extraction model further comprises a second instance of the first decoder configured to derive a third virtual image of a third representation format which is different from the first representation format of the first image and the second representation format of the second virtual image, and
• wherein the processor is configured to perform learning of the encoder, the first decoder, and the first discriminator on the basis of the first discrimination result, and that performs learning of the encoder and the third discriminator on the basis of the third discrimination result, wherein the learning of the third discriminator is performed by alternately inputting the first image with a ground-truth mask and the first image without the ground-truth mask.

A learning method of an extraction model that extracts a region of interest from images having a plurality of different representation formats,

• wherein the extraction model has
• an encoder that extracts a feature amount of a first image of a first representation format to derive a feature map of the first image,
• a first decoder that derives a second virtual image of a second representation format different from the first representation format of the first image on the basis of the feature map,
• a first discriminator that discriminates a representation format of an input image and whether the input image is a real image or a virtual image generated by the first decoder, and outputs a first discrimination result,
• a second decoder that extracts a region of interest of the first image on the basis of the feature map, and
• a second discriminator that discriminates whether an extraction result of the region of interest by the second decoder is an extraction result of a first image with a ground-truth mask or an extraction result of a first image without the ground-truth mask, and outputs a second discrimination result,
• wherein the extraction model further comprises a second instance of the first decoder configured to derive a third virtual image of a third representation format which is different from the first representation format of the first image and the second representation format of the second virtual image, and
• the learning method comprises:
• performing learning of the encoder, the first decoder, and the first discriminator on the basis of the first discrimination result;
• and
• performing learning of the encoder, the second decoder, and the second discriminator on the basis of the second discrimination result, wherein the learning of the second discriminator is performed by alternately inputting the first image with the ground-truth mask and the first image without the ground-truth mask.

A learning method of an extraction model that extracts a region of interest from images having a plurality of different representation formats,

• wherein the extraction model has
• an encoder that extracts a feature amount of a first image of a first representation format to derive a feature map of the first image,
• a first decoder that derives a second virtual image of a second representation format different from the first representation format of the first image on the basis of the feature map,
• a first discriminator that discriminates a representation format of an input image and whether the input image is a real image or a virtual image generated by the first decoder, and outputs a first discrimination result,
• a second decoder that extracts a region of interest of the first image on the basis of the feature map, and
• a third discriminator that discriminates a representation format of the first image of the first representation format that is input to the encoder on the basis of the feature map, and outputs a third discrimination result,
• wherein the extraction model further comprises a second instance of the first decoder configured to derive a third virtual image of a third representation format which is different from the first representation format of the first image and the second representation format of the second virtual image, and
• the learning method comprises:
• performing learning of the encoder, the first decoder, and the first discriminator on the basis of the first discrimination result;
• and
• performing learning of the encoder and the third discriminator on the basis of the third discrimination result, wherein the learning of the third discriminator is performed by alternately inputting the first image with a ground-truth mask and the first image without the ground-truth mask.

A non-transitory computer-readable storage medium that stores a learning program causing a computer to execute a learning method of an extraction model that extracts a region of interest from images having a plurality of different representation formats,

• wherein the extraction model has
• an encoder that extracts a feature amount of a first image of a first representation format to derive a feature map of the first image,
• a first decoder that derives a second virtual image of a second representation format different from the first representation format of the first image on the basis of the feature map,
• a first discriminator that discriminates a representation format of an input image and whether the input image is a real image or a virtual image generated by the first decoder, and outputs a first discrimination result,
• a second decoder that extracts a region of interest of the first image on the basis of the feature map, and
• a second discriminator that discriminates whether an extraction result of the region of interest by the second decoder is an extraction result of a first image with a ground-truth mask or an extraction result of a first image without the ground-truth mask, and outputs a second discrimination result,
• wherein the extraction model further comprises a second instance of the first decoder configured to derive a third virtual image of a third representation format which is different from the first representation format of the first image and the second representation format of the second virtual image, and
• the learning program causes the computer to execute a process comprising:
• performing learning of the encoder, the first decoder, and the first discriminator on the basis of the first discrimination result;
• and
• performing learning of the encoder, the second decoder, and the second discriminator on the basis of the second discrimination result, wherein the learning of the second discriminator is performed by alternately inputting the first image with the ground-truth mask and the first image without the ground-truth mask.

A non-transitory computer-readable storage medium that stores a learning program causing a computer to execute a learning method of an extraction model that extracts a region of interest from images having a plurality of different representation formats,

• wherein the extraction model has
• an encoder that extracts a feature amount of a first image of a first representation format to derive a feature map of the first image,
• a first decoder that derives a second virtual image of a second representation format different from the first representation format of the first image on the basis of the feature map,
• a first discriminator that discriminates a representation format of an input image and whether the input image is a real image or a virtual image generated by the first decoder, and outputs a first discrimination result,
• a second decoder that extracts a region of interest of the first image on the basis of the feature map, and
• a third discriminator that discriminates a representation format of the first image of the first representation format that is input to the encoder on the basis of the feature map, and outputs a third discrimination result,
• wherein the extraction model further comprises a second instance of the first decoder configured to derive a third virtual image of a third representation format which is different from the first representation format of the first image and the second representation format of the second virtual image, and
• the learning program causes the computer to execute a process comprising:
• performing learning of the encoder, the first decoder, and the first discriminator on the basis of the first discrimination result;
• and
• performing learning of the encoder and the third discriminator on the basis of the third discrimination result, wherein the learning of the third discriminator is performed by alternately inputting the first image with a ground-truth mask and the first image without the ground-truth mask.