Data augmentation including background modification for robust prediction using neural networks

A method comprising:

• receiving images of one or more objects having a plurality of backgrounds;
• generating a set of inference scores that correspond to one or more predictions of a prediction task performed on the one or more objects using the images;
• selecting a background based at least on one or more inference scores of the set of inference scores;
• generating an image based at least on integrating an object with the background based at least on the selecting of the background;
• and
• applying the image during training of at least one neural network to perform the prediction task.

A system comprising:

• one or more processors;
• and
• one or more memory devices storing instructions that, when executed by the one or more processors, cause the one or more processors to execute operations comprising:
• obtaining at least one neural network trained to perform a predictive task on images using inputs generated from masks that correspond to objects in the images based at least on modifying backgrounds of the objects using the masks;
• generating a mask that corresponds to an object in an image, wherein the object has a background in the image;
• generating, using the mask, inputs to the at least one neural network, the inputs capturing the object with at least a portion of the background;
• and
• generating at least one prediction of the predictive task based at least on applying the inputs to the at least one neural network.