Method for representing an environment of a mobile platform

A non-transitory computer-readable storage medium on which is stored a computer program for representing an environment of a first mobile platform, the computer program, when executed by a computer, causing the computer to perform the following steps:

• capturing features of the environment by discrete time sequences of sensor-data originating from at least two sensors and respective time markers;
• determining distances of the first mobile platform to the features of the environment;
• estimating semantic information of the features of the environment, using the sequences of data of each of the sensors characterizing the environment;
• transforming the semantic information of the features of the environment into a moving spatial reference system using spatial reference information and temporal reference information sensor-feature-wise, and model-based into a 2- or 3-dimensional grid representation, wherein a position of the first mobile platform is at a constant site, using the respective determined distances and respective time markers;
• creating an input tensor, using sequences of the transformed semantic representation of the features of the environment, corresponding to the sequences of the sensor data of the at least two sensors by stacking 2-dimensional grids, generated by the transformation of spatial coordinates of the sensors to the moving spatial reference system, including semantic representation of the data from a plurality of different sensors;
• and
• generating an output tensor that represents the environment using a deep neural network at a requested point in time and the input tensor to fuse the semantic grids of all individual ones of the sensors included in the input tensor and yield a predictive semantic grid output tensor at the requested time, wherein the deep neural network is trained with training input tensors and true information training output tensors as to accurately reproduce arbitrary environment representations at the requested point in time.

An at least partially autonomous system, comprising:

• a controller configured to represent an environment of a first mobile platform, the controller configured to:
• capturing features of the environment by discrete time sequences of sensor-data originating from at least two sensors and respective time markers;
• determining distances of the first mobile platform to the features of the environment;
• estimating semantic information of the features of the environment, using the sequences of data of each of the sensors characterizing the environment;
• transforming the semantic information of the features of the environment into a moving spatial reference system using spatial reference information and temporal reference information sensor-feature-wise, and model-based into a 2- or 3-dimensional grid representation, wherein a position of the first mobile platform is at a constant site, using the respective determined distances and respective time markers;
• creating an input tensor, using sequences of the transformed semantic representation of the features of the environment, corresponding to the sequences of the sensor data of the at least two sensors by stacking 2-dimensional grids, generated by the transformation of spatial coordinates of the sensors to the moving spatial reference system, including semantic representation of the data from a plurality of different sensors;
• and
• generating an output tensor that represents the environment using a deep neural network at a requested point in time and the input tensor to fuse the semantic grids of all individual ones of the sensors included in the input tensor and yield a predictive semantic grid output tensor at the requested time, wherein the deep neural network is trained with training input tensors and true information training output tensors as to accurately reproduce arbitrary environment representations at the requested point in time;
• wherein the controller is configured to execute or initiate a motion action of the at least partially autonomous system depending on content of the output tensor.