Systems and methods for deriving an agent trajectory based on tracking points within images

A non-transitory computer-readable medium comprising program instructions stored thereon that, when executed by at least one processor of a computing system, cause the computing system to perform functions comprising:

• receiving a sequence of images captured by a camera associated with a vehicle, wherein each image was captured at a respective capture time during a period of operation of the vehicle;
• for each respective image in at least a subset of images in the sequence in which a given agent is detected:
• generating a respective pixel mask that identifies a boundary of the given agent within the respective image and distinguishes the given agent from any other agent appearing within the respective image;
• identifying, as a respective tracking point for the given agent within the respective image, at least one given pixel within the respective pixel mask that is representative of an estimated intersection point between the given agent and a ground plane within the vehicle's surrounding environment;
• and
• based on the respective tracking point for the given agent within the respective image and information regarding the ground plane within the vehicle's surrounding environment, determining a position of the given agent at the respective capture time of the respective image;
• and
• determining a trajectory for the given agent based on the determined positions of the given agent.

A computing system comprising:

• at least one processor;
• a non-transitory computer-readable medium;
• and
• program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is configured to:
• receive a sequence of images captured by a camera associated with a vehicle, wherein each image was captured at a respective capture time during a period of operation of the vehicle;
• for each respective image in at least a subset of images in the sequence in which a given agent is detected:
• generate a respective pixel mask that identifies a boundary of the given agent within the respective image and distinguishes the given agent from any other agent appearing within the respective image;
• identify, as a respective tracking point for the given agent within the respective image, at least one given pixel within the respective pixel mask that is representative of an estimated intersection point between the given agent and a ground plane within the vehicle's surrounding environment;
• and
• based on the respective tracking point for the given agent within the respective image and information regarding the ground plane within the vehicle's surrounding environment, determine a position of the given agent at the respective capture time of the respective image;
• and
• determine a trajectory for the given agent based on the determined positions of the given agent.