Generating stereo-based dense depth images

A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform respective operations, the operations comprising:

• obtaining data representing a first image of a particular scene generated by a first sensor and a second image of the particular scene generated by a second sensor, wherein the particular scene includes one or more objects, wherein each of the first and second images includes a plurality of pixels;
• determining that a subset of the plurality of pixels in the first image are boundary pixels associated with a boundary of an object that is in the particular scene and represented in the first image;
• determining, from a plurality of candidate penalty values and for each boundary pixel of the subset of the plurality of pixels in the first image, an optimized penalty value for the boundary pixel, wherein the optimized penalty value is determined based on saliency information associated with the boundary pixel and is used for penalizing a matching cost of the boundary pixel and a corresponding pixel in the second image;
• generating an optimized cost function for the first image based on the optimized penalty values for the subset of the plurality of pixels in the first image;
• and
• generating a depth image for the first image based on the optimized cost function.

One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform respective operations, the respective operations comprising:

• obtaining data representing a first image of a particular scene generated by a first sensor and a second image of the particular scene generated by a second sensor, wherein the particular scene includes one or more objects, wherein each of the first and second images includes a plurality of pixels;
• determining that a subset of the plurality of pixels in the first image are boundary pixels associated with a boundary of an object that is in the particular scene and represented in the first image;
• determining, from a plurality of candidate penalty values and for each boundary pixel of the subset of the plurality of pixels in the first image, an optimized penalty value for the boundary pixel, wherein the optimized penalty value is determined based on saliency information associated with the boundary pixel and is used for penalizing a matching cost of the boundary pixel and a corresponding pixel in the second image;
• generating an optimized cost function for the first image based on the optimized penalty values for the subset of the plurality of pixels in the first image;
• and
• generating a depth image for the first image based on the optimized cost function.