Systems and methods for real-time state estimation of fast-moving objects

A system for predicting a location of a fast-moving object, the system comprising:

• an event camera;
• an event-based neural network;
• a catching device;
• a memory storing instructions;
• and
• a processor configured to execute the instructions to:
• receive event information from the event camera, the event information corresponding to events detected by the event camera;
• generate a Binary Event History Image (BEHI) based on the event information;
• provide the BEHI as an input to the event-based neural network;
• in response to inputting the BEHI, obtain, as an output of the event-based neural network, a first predicted location of the fast-moving object, a normal distribution indicating prediction uncertainty of the predicted location, and a predicted time-to-collision (TTC);
• estimate a second predicted location of the fast-moving object based on the first predicted location, the normal distribution, and the predicted TTC output by the event-based neural network;
• and
• actuate the mechanical catching device to be at the second predicted location.

A non-transitory computer readable medium for storing computer readable program code or instructions which are executable by a processor to perform operations for predicting a location of a fast-moving object, the operations comprising:

• receiving event information from an event camera, the event information corresponding to events detected by the event camera;
• generating a Binary Event History Image (BEHI) based on the event information;
• providing the BEHI as an input to an event-based neural network;
• obtaining, as an output of the event-based neural network in response to inputting the BEHI, a first predicted location of the fast-moving object, a normal distribution indicating prediction uncertainty of the predicted location, and a predicted time-to-collision (TTC);
• estimating a second predicted location of the fast-moving object based on the first predicted location, the normal distribution, and the predicted TTC output by the event-based neural network;
• and
• actuating a mechanical catching device to be at the second predicted location.