Back to results
US 2019/0356842 A1Published application
Camera depth prediction using generative adversarial network
AssigneeSony Corporation
InventorJunji Shimada
Life of this patent
Estimating the term…
Abstract
An apparatus and method for improving camera depth estimations for a moving subject in the camera field of view. The camera utilizes a generative adversarial network (GAN) to predict future frames from a current frame(s). These predicted future frames are then utilized as a target for determining a depth map from which lens focusing direction and distance are determined for moving the camera into proper focus and thus reducing focusing errors on captured frames as target and/or camera motion progresses.
Claims11
An apparatus for adjusting depth of focus in response to predicting future frames, comprising:
- (a) an image sensor configured for capturing digital images;
- (b) a focusing device coupled to said image sensor for controlling focal length of a digital image being captured;
- (c) a processor configured for performing image processing on images captured by said image sensor, and for outputting a signal for controlling focal length set by said focusing device;
- and
- (d) a memory storing programming executable by said processor for estimating depth of focus based on blur differences between images in a blur matching process;
- (e) wherein said programming when executed by said processor performs enhanced passive depth estimation by steps comprising:
- (i) selecting a moving focus target region, which is being tracked by a recognition process, at the current frame captured by said image sensor;
- (ii) generating estimations of at least one future frame utilizing a generative adversarial network (GAN);
- (iii) generating a depth map between said current frame and said at least one future frame in predicting future depth information while taking into account the period of time required for moving the focusing device to the moving focus target;
- (iv) determining direction and distance for adjusting said focusing device coupled to said image sensor to auto focus on the moving focus target region;
- and
- (v) moving said focusing device toward achieving a proper focus at said at least one future frame to update focus position of the selected moving focus target region.
9 more elements
An apparatus for adjusting depth of focus in response to predicting future frames, comprising:
- (a) an image sensor of an image capture device, said image sensor configured for capturing digital images;
- (b) wherein said image capture device is selected from a group of image capture devices consisting of camera systems, camera-enabled cell phones, and other image-capture enabled electronic devices;
- (b) a focusing device coupled to said image sensor for controlling focal length of a digital image being captured;
- (c) a processor configured for performing image processing on images captured by said image sensor, and for outputting a signal for controlling focal length set by said focusing device;
- and
- (d) a memory storing programming executable by said processor for estimating depth of focus based on blur differences between images in a blur matching process;
- (e) wherein said programming when executed by said processor performs enhanced passive depth estimation by steps comprising:
- (i) selecting a moving focus target region at the current frame captured by said image sensor which is being tracked by a recognition process;
- (ii) generating estimations of at least one future frame utilizing a generative adversarial network (GAN);
- (iii) generating a depth map between said current frame and said at least one future frame in predicting future depth information while taking into account a period of time required for moving the focusing device to the moving focus target;
- (iv) determining direction and distance for adjusting said focusing device coupled to said image sensor to auto focus on the moving focus target region;
- and
- (v) moving said focusing device toward achieving a proper focus at said at least one future frame to update focus position of the selected moving focus target region.
10 more elements
A method for adjusting depth of focus in response to predicting future frames, comprising:
- (a) capturing digital images utilizing an autofocus process based on estimating depth of focus based on blur differences between images in a blur matching process;
- (b) selecting a moving focus target region, which is tracked by a recognition process, at the current frame captured by an image sensor within a device configured for capturing digital images utilizing an autofocus enhanced passive depth estimation process;
- (c) generating estimations of at least one future frame utilizing a generative adversarial network (GAN) executing on a processor which is executing programming from a memory;
- (d) generating a depth map between said current frame and said at least one future frame in predicting future depth information while taking into account a period of time required for moving the focusing device to the moving focus target;
- (e) determining direction and distance for adjusting a focusing device coupled to the image sensor to auto focus on the moving focus target region;
- and
- (f) moving said focusing device toward achieving a proper focus at said at least one future frame to update focus position of the selected moving focus target region.
4 more elements
Dependencies are inferred from the claim text.