Jointly learning visual motion and confidence from local patches in event cameras

A device comprising:

• a memory configured to store instructions;
• and
• a processor configured to execute the instructions to:
• obtain a set of events, of a set of pixels of a dynamic vision sensor, associated with an object;
• determine, for each pixel of the set of pixels, a set of voltages corresponding to a set of time constants, based on the set of events;
• group the set of voltages of the set of pixels into a set of voltage groups, based on the set of time constants and respective polarities of the set of pixels;
• generate, for each voltage group in the set of voltage groups, an image based on voltages corresponding to that voltage group, to result in a set of images;
• input the set of images into a first neural network configured to output a visual motion estimation of the object;
• input the set of images into a second neural network configured to output a confidence score of the visual motion estimation output by the first neural network;
• obtain the visual motion estimation of the object and the confidence score of the visual motion estimation of the object, based on inputting the set of images into the first neural network and the second neural network;
• and
• provide the visual motion estimation of the object and the confidence score.

A non-transitory computer-readable medium storing instructions, the instructions comprising:

• one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to:
• obtain a set of events, of a set of pixels of a dynamic vision sensor, associated with an object;
• determine, for each pixel of the set of pixels, a set of voltages corresponding to a set of time constants, based on the set of events;
• group the set of voltages of the set of pixels into a set of voltage groups, based on the set of time constants and respective polarities of the set of pixels;
• generate, for each voltage group in the set of voltage groups, an image based on voltages corresponding to that voltage group, to result in a set of images, based on the set of voltages of the set of pixels;
• input the set of images into a first neural network configured to output a visual motion estimation of the object;
• input the set of images into a second neural network configured to output a confidence score of the visual motion estimation output by the first neural network;
• obtain the visual motion estimation of the object and the confidence score of the visual motion estimation of the object, based on inputting the set of images into the first neural network and the second neural network;
• and
• provide the visual motion estimation of the object and the confidence score.