Mixup image captioning

A computer program product comprising:

• one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the stored program instructions comprising:
• program instructions to sample a ratio lambda from a probability distribution based on a pair of datapoints contained in a dataset, wherein each datapoint in the pair of datapoints comprises an image and an associated caption;
• program instructions to extend the dataset by generating one or more new datapoints based on the sampled ratio lambda for each pair of datapoints in the dataset, wherein the sampled ratio lambda incorporates an interpolation of features associated with the pair of datapoints into the generated one or more new datapoints;
• program instructions to identify one or more objects contained within a subsequent image utilizing an image model trained utilizing the extended dataset;
• and
• program instructions to generate a subsequent caption for one or more identified objects contained within the subsequent image utilizing a language generating model trained utilizing the extended dataset.

A computer system comprising:

• one or more computer processors;
• one or more computer readable storage media;
• and
• program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the stored program instructions comprising:
• program instructions to sample a ratio lambda from a probability distribution based on a pair of datapoints contained in a dataset, wherein each datapoint in the pair of datapoints comprises an image and an associated caption;
• program instructions to extend the dataset by generating one or more new datapoints based on the sampled ratio lambda for each pair of datapoints in the dataset, wherein the sampled ratio lambda incorporates an interpolation of features associated with the pair of datapoints into the generated one or more new datapoints;
• program instructions to identify one or more objects contained within a subsequent image utilizing an image model trained utilizing the extended dataset;
• and
• program instructions to generate a subsequent caption for one or more identified objects contained within the subsequent image utilizing a language generating model trained utilizing the extended dataset.