Processing of signals using a recurrent state estimator
AssigneeApple Inc.
InventorWalter Nistico
Life of this patent
Estimating the term…
Abstract
In one implementation, a method includes receiving pixel events output by an event sensor that correspond to a feature disposed within a field of view of the event sensor. Each respective pixel event is generated in response to a specific pixel within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold. A characteristic of the feature is determined at a first time based on the pixel events and a previous characteristic of the feature at a second time that precedes the first time. Movement of the feature relative to the event sensor is tracked over time based on the characteristic and the previous characteristic.
Claims22
A method comprising:
- receiving pixel events output by an event sensor, each respective pixel event generated in response to a specific pixel within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold, the pixel events corresponding to a feature disposed within a field of view of the event sensor;
- determining a characteristic of the feature at a first time based on the pixel events and a previous characteristic of the feature at a second time that precedes the first time, wherein the previous characteristic of the feature at the second time was a result of a previous characteristic determination based on previous pixel events, wherein the characteristic is determined by inputting the pixel events output by the event sensor to a neural network that produces output corresponding to the characteristic based on the pixel events and state information corresponding to the previous characteristic, the neural network comprising a filter configured to persist the state information from earlier input data for use in generating subsequent outputs;
- and
- tracking movement of the feature relative to the event sensor over time based on the characteristic and the previous characteristic.
A system comprising:
- a processor;
- an image pipeline;
- and
- a computer-readable storage medium comprising instructions that upon execution by the processor cause the system to perform operations, the operations comprising:
- receiving, by the image pipeline, pixel events output by an event sensor comprising a plurality of pixels positioned to receive light from a surface of an eye, each respective pixel event generated in response to a specific pixel within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold;
- determining a gaze characteristic at a first time based on the pixel events and a previous gaze characteristic at a second time that precedes the first time, wherein the previous gaze characteristic of the feature at the second time was a result of a previous gaze characteristic determination based on previous pixel events, wherein the gaze characteristic is determined by inputting the pixel events output by the event sensor to a neural network that produces output corresponding to the gaze characteristic based on the pixel events and state information corresponding to the previous gaze characteristic, the neural network comprising a filter configured to persist the state information from earlier input data for use in generating subsequent outputs;
- and
- tracking a gaze of the eye based on the gaze characteristic.
A non-transitory computer-readable storage medium storing program instructions computer-executable on a computer to perform operations comprising:
- receiving pixel events output by an event sensor comprising a plurality of pixels positioned to receive light from a scene disposed within a field of view of the event sensor, each respective pixel event generated in response to a specific pixel within the plurality of pixels detecting a change in light intensity that exceeds a comparator threshold;
- determining, with a recurrent estimation process, a characteristic of a feature within the field of view at a first time based on the pixel events and a previous characteristic of the feature at a second time that precedes the first time, wherein the previous characteristic of the feature at the second time was a result of a previous characteristic determination based on previous pixel events, wherein the characteristic is determined by inputting the pixel events output by the event sensor to a neural network that produces output corresponding to the characteristic based on the pixel events and state information corresponding to the previous characteristic, the neural network comprising a filter configured to persist the state information from earlier input data for use in generating subsequent outputs;
- and
- tracking movement of the feature within the field of view using the characteristic.
The method of 1, wherein the filter is a 2D recurrent filter.
The method of 1, wherein the filter is a 3D recurrent filter.
Dependencies are inferred from the claim text.