Methods and systems for joint pose and shape estimation of objects from sensor data

A method to jointly estimate a pose and a shape of an object perceived by an autonomous vehicle, the method comprising:

• generating a neural network by:
• receiving training data comprising partial point cloud data and ground truth data corresponding to one or more objects;
• generating a trained shared encoder and a trained shape decoder by using the training data to collectively train a shared encoder neural network and a shape decoder neural network, wherein the trained shared encoder and the trained shared decoder are configured to estimate the shape of each of the plurality of objects;
• generating a trained pose decoder by using one or more codes generated by the trained shared encoder to train a pose decoder neural network while freezing one or more parameters associated with the trained shared encoder;
• and
• outputting the neural network comprising the trained shared encoder, the trained pose decoder, and the trained shape decoder;
• receiving point cloud data corresponding to the object, wherein the point cloud data is an incomplete point cloud representation of the object;
• inputting the point cloud data into the trained shared encoder;
• outputting, by the trained shared encoder, a code representative of the point cloud data;
• generating, by the trained pose decoder, an estimated pose of the object based on the code, the estimated pose comprising at least a heading or a translation of the object;
• and
• generating, by the trained shape decoder, an estimated shape of the object based on the code, the estimated shape comprising an estimated point cloud representation of the object that is denser than the incomplete point cloud representation of the object.

A system to jointly estimate a pose and a shape of an object perceived by an autonomous vehicle comprising:

• a processor;
• and
• a non-transitory computer readable medium comprising:
• data and program code collectively defining a neural network which has been trained to jointly estimate, for each of a plurality of objects, a pose and a shape from incomplete point cloud data associated with that object, the neural network comprising a trained shared encoder shared by a trained pose decoder and a trained shape decoder, the trained shared encoder having one or more parameters unable to be modified during training of the trained pose detector;
• and
• one or more programming instructions that when executed by the processor, cause the processor to:
• receive point cloud data corresponding to the object, wherein the point cloud data is an incomplete point cloud representation of the object,
• inputting the point cloud data into the trained shared encoder,
• outputting, by the trained shared encoder, a code representative of the point cloud data,
• generating, by the trained pose decoder, an estimated pose of the object based on the code, the estimated pose comprising at least a heading or a translation of the object, and
• generating, by the trained shape decoder, an estimated shape of the object based on the code, the estimated shape comprising an estimated point cloud representation of the object that is denser than the incomplete point cloud representation of the object.