Parametric top-view representation of complex road scenes

A processing system for producing a road layout model, comprising:

• one or more processor devices;
• a memory in communication with at least one of the one or more processor devices;
• and
• a display screen;
• wherein the processing system includes a graphical modeler configured to receive as input a plurality of sequential digital images, wherein the digital images are of a perspective view, convert each of the plurality of sequential digital images into a top-down view image using a processor through semantic segmentation, wherein each object in the sequential digital images is segmented by clustering pixels of the object into ground truth classes, and assigning a depth value to each pixel, convey a top-down view image of time t to a neural network that performs a feature transform on the top-down view image of time t to form a feature map of time t, and convey a top-down view image of time t+1 to the neural network that performs a feature transform on the top-down view image of time t+1 to form a feature map of time t+1;
• a feature transform module configured to receive the feature map of the top-down view image of time t and warp the feature map of time t to a time t+1;
• a combiner configured to combine the warped feature map of time t with the feature map of time t+1 to form a combined feature map;
• a long short-term memory (LSTM) module configured to receive the combined feature map and to generate the road layout model;
• and
• the processor device is configured to transmit the road layout model to the display screen for presentation to a user.

A non-transitory computer readable storage medium comprising a computer readable program for producing a road layout model, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:

• capturing a plurality of sequential digital images using a video camera, wherein the digital images are of a perspective view;
• converting each of the plurality of sequential digital images to a top-down view image using a processor through semantic segmentation, wherein each object in the sequential digital images is segmented by clustering pixels of the object into ground truth classes, and assigning a depth value to each pixel;
• conveying a top-down view image of time t to a neural network that performs a feature transform on the top-down view image of time t to form a feature map of time t;
• transferring the feature map of the top-down view image of time t into a feature transform module configured to warp the feature map of time t to a time t+1;
• conveying a top-down view image of time t+1 to the neural network that performs a feature transform on the top-down view image of time t+1 to form a feature map of time t+1;
• combining the warped feature map of time t with the feature map of time t+1 to form a combined feature map;
• transferring the combined feature map to a long short-term memory (LSTM) module to generate the road layout model;
• and
• displaying the road layout model to a user.