Hybrid image and event sensing with rolling shutter compensation

A sensor device comprising:

• a plurality of pixels each configured to receive light and perform photoelectric conversion to generate an electrical signal;
• event detection circuitry that is configured to generate event data by detecting as events intensity changes above a predetermined threshold of the light received by each of event detecting pixels that form a first subset of the pixels;
• pixel signal generating circuitry that is configured to generate pixel signals indicating intensity values of the light received by each of intensity detecting pixels that form a second subset of the pixels, the intensity detecting pixels being arranged in a two dimensional array comprising a plurality of pixel rows;
• and
• a control unit that is configured
• to associate with each other event detecting pixels and intensity detecting pixels that have a corresponding field of view in order to assign to each detected event a corresponding pixel row number;
• to read out the pixel signals by applying a rolling shutter that starts readout of the pixel signals of adjacent pixel rows at times separated by a predetermined time period and
• to assign time stamps to each pixel signal that indicate the time of readout of the corresponding pixel row,
• to assign to the events event time stamps based on the time at which the events were detected,
• to convert by a conversion function that depends on the pixel row number the time stamps of all pixel signals to a single frame time, where the pixel signals were obtained during an intensity frame acquisition period that is defined as the time required for a single readout of all pixel rows, and
• to convert the event time stamps by the same conversion function to obtain converted event time stamps for detected events.