Back to results
US 9,288,395 B2Granted
Super-resolution based on optical image stabilization
AssigneeApple Inc.
InventorsRichard L. Baer, Damien J. Thivent
Life of this patent
Estimating the term…
Abstract
A system and method for creating a super-resolution image using an image capturing device. In one embodiment, an electronic image sensor captures a reference optical sample through an optical path. Thereafter, an optical image stabilization (OIS) processor to adjusts the optical path to the electronic image sensor by a known amount. A second optical sample is then captured along the adjusted optical path, such that the second optical sample is offset from the first optical sample by no more than a sub-pixel offset. The OIS processor may reiterate this process to capture a plurality of optical samples at a plurality of offsets. The optical samples may be combined to create a super-resolution image.
Claims29
An image capturing device comprising:
- an electronic image sensor operable to capture a plurality of optical samples through an optical path;
- a lens through which the optical path travels;
- an actuator that is operable to move the lens;
- an accelerometer;
- an optical image stabilization (OIS) processor to signal the actuator to move the lens so as to stabilize an optical image that is being projected onto the sensor through the optical path, before the sensor converts the optical image into digital form being said captured plurality of optical samples, in response to using the accelerometer to detect movement of the device due to involuntary shaking by a user who is holding the device;
- the OIS processor to command a plurality of super-resolution shifts of the optical path by signaling the actuator to move the lens while the plurality of optical samples are being captured, wherein each super-resolution shift corresponds to one of a plurality of sub-pixel offsets;
- a super-resolution engine to create a super-resolution image based on the plurality of captured optical samples;
- the OIS processor to command a plurality of increase color sampling density shifts of the optical path by signaling the actuator to move the lens while the plurality of optical samples are being captured, wherein each increase color sampling density shift corresponds to one of a plurality of one-pixel offsets to provide full color sampling at each pixel location of each color the image sensor is capable of capturing;
- and
- the super-resolution engine to create an increase color sampling density image based on the plurality of captured optical samples.
7 more elements
An article of manufacture comprising a non-transitory machine readable storage medium in which are stored executable program instructions that program an image capturing device to:
- capture, using an image sensor, a plurality of optical samples along a plurality of optical paths obtained by signaling an actuator to move a lens located in front of the image sensor so as to stabilize an optical image that is being projected onto the sensor through an optical path, before the sensor converts the optical image into digital form being said captured plurality of optical samples, in response to using an accelerometer to detect movement of the device due to involuntary shaking by a user who is holding the device;
- command, using an optical image stabilization (OIS) processor, a plurality of super resolution shifts of the optical path, wherein the super resolution shifts correspond to sub-pixel offsets, and wherein the command of the super resolution shifts comprises signaling the actuator to move the lens while the plurality of optical samples are being captured;
- combine the plurality of optical samples to create a super-resolution images;
- command, using an optical image stabilization (OIS) processor, a plurality of increase color sampling density shifts of the optical path, wherein the increase color sampling density shifts correspond to one-pixel offsets to provide full color sampling at each pixel location of each color the image sensor is capable of capturing, and wherein the command of the increase color sampling density shifts comprises signaling the actuator to move the lens while the plurality of optical samples are being captured;
- combine the plurality of increase color sampling density samples to create a increase color sampling density image.
2 more elements
A method for capturing a super-resolution image by an image capturing device, comprising:
- capturing, using an image sensor, a plurality of optical samples along a plurality of optical paths obtained by signaling an actuator to move a lens located in front of the image sensor so as to stabilize an optical image that is being projected onto the sensor through an optical path, before the sensor converts the optical image into digital form being said captured plurality of optical samples, in response to using an accelerometer to detect movement of the device due to involuntary shaking by a user who is holding the device;
- commanding a plurality of calibrated super resolution shifts of the optical path to the image sensor by an optical image stabilization (OIS) processor, wherein the plurality of calibrated super resolution shifts are associated with sub-pixel offsets, and wherein the command of the plurality of calibrated super resolution shifts comprises signaling the actuator to move the lens while the plurality of optical samples are being captured;
- combining the plurality of optical samples to create a densely sampled image;
- commanding a plurality of calibrated increase color sampling density shifts of the optical path to the image sensor by an optical image stabilization (OIS) processor, wherein the plurality of calibrated increase color sampling density shifts are associated with one-pixel offsets to provide full color sampling at each pixel location of each color the image sensor is capable of capturing, and wherein the command of the plurality of calibrated increase color sampling density shifts comprises signaling the actuator to move the lens while the plurality of optical samples are being captured;
- and
- combining the plurality of optical samples to create a increase color sampling densely sampled image.
3 more elements
Dependencies are inferred from the claim text.