Image depth prediction neural networks

A system comprising:

• an image depth prediction neural network implemented by one or more computers, wherein the image depth prediction neural network is a recurrent neural network that is configured to receive a sequence of images and, for each image in the sequence:
• process the image, which is a current image at a first time step in the sequence, in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a current depth map that characterizes a current depth of the image in the sequence;
• and
• an image generation subsystem configured to, for each image in the sequence:
• receive the current depth map that characterizes the current depth of the image,
• construct, based on the current depth map and the image, a plurality of three-dimensional (3D) points, each of the plurality of 3D points corresponding to a different pixel in the image, and
• generate a depth output that characterizes a predicted depth of a future image in the sequence by applying one or more transformation layers to the plurality of 3D points, wherein the depth output comprises a set of values defining the topology of a scene represented by the future image in a third, depth dimension.

A method comprising:

• receiving a sequence of images;
• and
• for each image in the sequence:
• processing the image using an image depth prediction neural network,
• wherein the image depth prediction neural network is a recurrent neural network that is configured to, for each image in the sequence:
• process the image, which is a current image at a first time step in the sequence, in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a current depth map that characterizes a current depth of the image in the sequence;
• receiving the current depth map that characterizes the current depth of the image,
• constructing, based on the current depth map and the image, a plurality of three-dimensional (3D) points, each of the plurality of 3D points corresponding to a different pixel in the image, and
• generating a depth output that characterizes a predicted depth of a future image in the sequence by applying one or more transformation layers to the plurality of 3D points, wherein the depth output comprises a set of values defining the topology of a scene represented by the future image in a third, depth dimension.

One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• receiving a sequence of images;
• and
• for each image in the sequence:
• processing the image using an image depth prediction neural network,
• wherein the image depth prediction neural network is a recurrent neural network that is configured to, for each image in the sequence:
• process the image, which is a current image at a first time step in the sequence, in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a current depth map that characterizes a current depth of the image in the sequence;
• receiving the current depth map that characterizes the current depth of the image,
• constructing, based on the current depth map and the image, a plurality of three-dimensional (3D) points, each of the plurality of 3D points corresponding to a different pixel in the image, and
• generating a depth output that characterizes a predicted depth of a future image in the sequence by applying one or more transformation layers to the plurality of 3D points, wherein the depth output comprises a set of values defining the topology of a scene represented by the future image in a third, depth dimension.