Generating trajectory labels from short-term intention and long-term result

A system comprising:

• one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
• obtaining a training input, the training input comprising (i) data characterizing an agent in an environment as of a first time and (ii) data characterizing a candidate trajectory of the agent in the environment for a first time period that is after the first time;
• determining a long-term label for the candidate trajectory that indicates whether the agent actually followed the candidate trajectory for the first time period after the first time;
• determining a short-term label for the candidate trajectory that indicates whether the agent intended to follow the candidate trajectory;
• determining, based on the long-term label and the short-term label for the candidate trajectory, a ground-truth probability for the candidate trajectory, wherein the ground-truth probability for the candidate trajectory is a probability that should be assigned to the candidate trajectory by a machine learning model that performs trajectory prediction;
• associating the training input with the ground-truth probability for the candidate trajectory in training data;
• and
• training the machine learning model on the training data.

A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• obtaining a training input, the training input comprising (i) data characterizing an agent in an environment as of a first time and (ii) data characterizing a candidate trajectory of the agent in the environment for a first time period that is after the first time;
• determining a long-term label for the candidate trajectory that indicates whether the agent actually followed the candidate trajectory for the first time period after the first time;
• determining a short-term label for the candidate trajectory that indicates whether the agent intended to follow the candidate trajectory;
• determining, based on the long-term label and the short-term label for the candidate trajectory, a ground-truth probability for the candidate trajectory, wherein the ground-truth probability for the candidate trajectory is a probability that should be assigned to the candidate trajectory by a machine learning model that performs trajectory prediction;
• associating the training input with the ground-truth probability for the candidate trajectory in training data;
• and
• training the machine learning model on the training data.