Method for determining spatial-temporal patterns related to the environment of a vehicle

A computer system comprising:

• one or more processors;
• and
• memory including instructions that, when executed by the one or more processors, perform a method including:
• determining respective sets of characteristics detected by a perception system of a host vehicle in the environment of the host vehicle for a current point in time and for a predefined number of previous points in time,
• wherein the characteristics include a current position, a current velocity and an object class for a plurality of objects, and
• generating a set of current input data associated with the set of characteristics for the current point in time,
• generating a set of memory data by aggregating the sets of characteristics for the previous points in time,
• applying an attention algorithm to the set of current input data and to the set of memory data in order to generate a joined spatial-temporal data set,
• wherein the attention algorithm includes weights which are defined by relating elements of the set of current input data and assigned elements of the set of memory data, and
• wherein the weights of the attention algorithm are applied to values generated by employing a union of the elements of the set of current input data and the assigned elements of the set of memory data in order to provide the joined spatial-temporal data set as an output of the attention algorithm,
• determining at least one pattern for the environment of the host vehicle from the joined spatial-temporal data set,
• wherein the pattern is provided as an abstract feature map which is stored in a grid map, and
• wherein tasks implemented as machine learning algorithms are applied to the abstract feature map in order to derive information regarding a position and/or dynamics of a respective object in the environment of the host vehicle, and/or in order to perform a grid segmentation of the environment of the host vehicle, and
• controlling the host vehicle based on the at least one pattern related to the environment of the host vehicle from the joined spatial-temporal data set,
• wherein:
• for generating the joined spatial-temporal data set, the output of the attention algorithm is further combined with the set of memory data, and
• a gating procedure is configured to define weights for combining respective channels within the set of memory data with the output of the attention algorithm.

A non-transitory computer-readable medium comprising instructions for carrying out a method comprising:

• determining respective sets of characteristics detected by a perception system of a host vehicle in the environment of the host vehicle for a current point in time and for a predefined number of previous points in time,
• wherein the characteristics include a current position, a current velocity and an object class for a plurality of objects, and
• generating a set of current input data associated with the set of characteristics for the current point in time,
• generating a set of memory data by aggregating the sets of characteristics for the previous points in time,
• applying an attention algorithm to the set of current input data and to the set of memory data in order to generate a joined spatial-temporal data set,
• wherein the attention algorithm includes weights which are defined by relating elements of the set of current input data and assigned elements of the set of memory data, and
• wherein the weights of the attention algorithm are applied to values generated by employing a union of the elements of the set of current input data and the assigned elements of the set of memory data in order to provide the joined spatial-temporal data set as an output of the attention algorithm,
• determining at least one pattern for the environment of the host vehicle from the joined spatial-temporal data set,
• wherein the pattern is provided as an abstract feature map which is stored in a grid map, and
• wherein tasks implemented as machine learning algorithms are applied to the abstract feature map in order to derive information regarding a position and/or dynamics of a respective object in the environment of the host vehicle, and/or in order to perform a grid segmentation of the environment of the host vehicle, and
• controlling the host vehicle based on the at least one pattern related to the environment of the host vehicle from the joined spatial-temporal data set,
• wherein:
• for generating the joined spatial-temporal data set, the output of the attention algorithm is further combined with the set of memory data, and
• a gating procedure is configured to define weights for combining respective channels within the set of memory data with the output of the attention algorithm.