Real time salient object detection in images and videos

An apparatus, comprising:

• one or more processors;
• and
• one or more storage devices storing a machine learning model having processing operations that are performed by the one or more processors, the machine learning model comprising:
• a first backbone stage receiving one or more images and generating a first feature map, the first backbone stage comprising a first spatial difference layer to perform convolution using one or more first parameters trained based on different spatial convolution types;
• a second backbone stage receiving the first feature map from the first backbone stage and generating a second feature map based on the first feature map;
• a first branch receiving the second feature map from the second backbone stage and generating a third feature map based on the second feature map;
• and
• a second branch receiving the first feature map from the first backbone stage and the third feature map from the first branch, combining the first feature map with the third feature map to generate a first combined feature map, and generating one or more predictions about the one or more images based on the first combined feature map.

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to:

• input one or more images into a first backbone stage of a machine learning model, the first backbone stage comprising a first spatial difference layer to perform convolution using one or more first parameters trained based on different spatial convolution types;
• generate, by the first backbone stage, a first feature map based on the one or more images;
• input the first feature map into a second backbone stage of the machine learning model;
• generate, by the second backbone stage, a second feature map based on the first feature map;
• input the second feature map into a first branch of the machine learning model;
• generate, by the first branch, a third feature map based on the second feature map;
• input the first feature map and the third feature map into a second branch of the machine learning model;
• combine, by the second branch, the first feature map with the third feature map to generate a first combined feature map;
• and
• generate, by the second branch, one or more predictions about the one or more images based on the first combined feature map.