Methods and systems for predicting dynamic object behavior

A computing system for predicting behavior of a dynamic object of interest in an environment of a vehicle, the computing system comprising:

• a processor system configured to execute instructions to cause an object behavior prediction subsystem of the computing system to:
• receive a plurality of time series of feature data, each time series of feature data representing a respective feature of a plurality of objects in the environment over a plurality of time steps, the plurality of objects including a dynamic object of interest;
• categorize each time series of feature data into one of a plurality of defined object categories, each defined object category being defined by one or more shared characteristics of objects belonging to that defined object category, to obtain a categorized set of data for each defined object category, each categorized set of data containing one or more time series of feature data representing one or more objects belonging to the respective defined object category;
• for each given categorized set of data belonging to a given defined object category, encode, using a spatiotemporal encoder, the one or more time series of feature data belonging to the given categorized set of data into a single categorical representation, the single categorical representation being a time series of feature vectors representing temporal change of features within the given defined object category over the plurality of time steps, wherein a respective plurality of categorical representations is obtained for the plurality of defined object categories;
• combine the plurality of categorical representations into a single shared representation that is a single time series of tensors that represents spatiotemporal changes in each of the plurality of defined object categories over the plurality of time steps;
• generate a categorical interaction representation based on the single shared representation, the categorical interaction representation being a weighted vector representing contributions of temporal change in each defined object category to a final time step of the shared representation;
• and
• generate predicted data based at least on the categorical interaction representation, data representing dynamics of the objects, and data representing a state of the vehicle, the predicted data representing a predicted future behavior of the dynamic object of interest.

A non-transitory computer readable medium comprising computer-executable instructions to implement an object behavior prediction subsystem to predict behavior of a dynamic object of interest in a sensed environment of a vehicle, wherein the instructions, when executed by a processing system of a computing system, cause the computing system to:

• receive a plurality of time series of feature data, each time series of feature data representing a respective feature of a plurality of objects in an environment over a plurality of time steps, the plurality of objects including a dynamic object of interest;
• categorize each time series of feature data into one of a plurality of defined object categories, each defined object category being defined by one or more shared characteristics of objects belonging to that defined object category, to obtain a categorized set of data for each defined object category, each categorized set of data containing one or more time series of feature data representing one or more objects belonging to the respective defined object category;
• for each given categorized set of data belonging to a given defined object category, encode, using a spatiotemporal encoder, the one or more time series of feature data belonging to the given categorized set of data into a single categorical representation, the single categorical representation being a time series of feature vectors representing temporal change of features within the given defined object category over the plurality of time steps, wherein a respective plurality of categorical representations is obtained for the plurality of defined object categories;
• combine the categorical representations into a single shared representation that is a single time series of tensors that represents spatiotemporal changes in each of the plurality of defined object categories over the plurality of time steps;
• generate a categorical interaction representation based on the single shared representation, the categorical interaction representation being a weighted vector representing contributions of temporal change in each defined object category to a final time step of the shared representation;
• and
• generate predicted data based on the categorical interaction representation, data representing dynamics of the plurality of objects, and data representing a state of the vehicle, the predicted data representing a predicted future behavior of the dynamic object of interest.