Constructing trained models to associate object in image with description in sentence where feature amount for sentence is derived from structured information

A learning method comprising:

• deriving a first feature amount for an object included in an image by a first neural network;
• structuring a sentence including description of the object included in the image to derive structured information for the sentence;
• deriving a second feature amount for the sentence from the structured information by a second neural network;
• and
• constructing a first derivation model that derives a feature amount for the object included in the image and a second derivation model that derives a feature amount for the sentence including the description of the object by training the first neural network and the second neural network such that, in a feature space to which the first feature amount and the second feature amount belong, a distance between the derived first feature amount and second feature amount is smaller in a case in which the object included in the image and the object described in the sentence correspond to each other than a case in which the object included in the image and the object described in the sentence do not correspond to each other.

A non-transitory computer-readable storage medium that stores a learning program causing a computer to execute:

• a procedure of deriving a first feature amount for an object included in an image by a first neural network;
• a procedure of structuring a sentence including description of the object included in the image to derive structured information for the sentence;
• a procedure of deriving a second feature amount for the sentence from the structured information by a second neural network;
• and
• a procedure of constructing a first derivation model that derives a feature amount for the object included in the image and a second derivation model that derives a feature amount for the sentence including the description of the object by training the first neural network and the second neural network such that, in a feature space to which the first feature amount and the second feature amount belong, a distance between the derived first feature amount and second feature amount is smaller in a case in which the object included in the image and the object described in the sentence correspond to each other than a case in which the object included in the image and the object described in the sentence do not correspond to each other.