Probabilistic state simulation for end-to-end drive stack learning for autonomous and semi-autonomous machines and applications

Abstract

In various examples, perception encoder uses one or more neural networks implemented using a transformer architecture, a sensor perspective encoding, a planned navigation route, and/or detected ego-motion to extract a scene embedding representing one or more aspects of an observed scene, such as visual information, motion information, ego-state of an ego-machine, a planned navigation route, and/or other types of information. The perception encoder may be used in a probabilistic state simulation stack, and/or may be used to extract and apply a scene embedding as an input for 3D perception or reconstruction tasks such as object detection and classification, semantic segmentation, depth map extraction, trajectory prediction, path planning, navigation control, and/or localization or mapping, to name a few example tasks.

Dependencies are inferred from the claim text.