Back to results
US 2019/0379819 A1Published application
Detection of main object for camera auto focus
AssigneeSony Corporation
InventorJunji Shimada
Life of this patent
Estimating the term…
Abstract
A camera apparatus and method which selects a main object for camera autofocus control. Captured images are input to a convolution neural network (CNN) which is configured for generating pose information. The pose information is utilized in a process of tracking and determining trajectory similarities between the camera trajectory and the trajectory of each of said multiple objects. A main object of focus is then selected as the main object based on which objects maintains the smallest difference in trajectory between camera and object. The autofocus operation of the camera is based on position and trajectory of this main object.
Claims19
A camera apparatus, comprising:
- (a) an image sensor configured for capturing digital images;
- (b) a focusing device coupled to said image sensor for controlling focal length of a digital image being captured;
- (c) a processor configured for performing image processing on images captured by said image sensor, and for outputting a signal for controlling focal length set by said focusing device;
- and
- (d) a memory storing programming executable by said processor for estimating depth of focus based on blur differences between images;
- (e) said programming when executed performing steps comprising:
- (i) inputting an image captured by the camera image sensor into a multiple-branch, multiple-stage convolution neural network (CNN) which is configured for predicting anatomical relationships and generating pose information;
- (ii) tracking bounding boxes of multiple objects using a recursive state-space model in combination with a matching algorithm to estimate intersections over union distances (IoU) between the multiple objects;
- (iii) determining trajectory similarities between the camera trajectory and the trajectory of each of said multiple objects by obtaining a camera trajectory and trajectories of each of said multiple objects, followed by a dynamic time warping process to estimate trajectory differences across frames;
- (iv) selecting a main object of focus as the object from said multiple objects which maintains the smallest difference in trajectory between camera and object;
- and
- (v) performing camera autofocusing based on the position and trajectory of said main object.
9 more elements
A camera apparatus, comprising:
- (a) an image sensor configured for capturing digital images;
- (b) a focusing device coupled to said image sensor for controlling focal length of a digital image being captured;
- (c) a processor configured for performing image processing on images captured by said image sensor, and for outputting a signal for controlling focal length set by said focusing device;
- and
- (d) a memory storing programming executable by said processor for estimating depth of focus based on blur differences between images;
- (e) said programming when executed performing steps comprising:
- (i) inputting an image captured by the camera image sensor into a multiple-branch, multiple-stage convolution neural network (CNN), having at least a first branch configured for predicting confidence maps of body parts for each person object detected within the image, and at least a second branch for predicting part affinity fields (PAFs) for each person object detected within the image, with said CNN configured for predicting anatomical relationships and generating pose information;
- (ii) tracking bounding boxes of multiple objects using a recursive state-space model in combination with a matching algorithm to estimate intersections over union distances (IoU) between the multiple objects;
- (iii) determining trajectory similarities between the camera trajectory and the trajectory of each of said multiple objects by obtaining a camera trajectory and trajectories of each of said multiple objects, followed by a dynamic time warping process to estimate trajectory differences across frames;
- (iv) selecting a main object of focus as the object from said multiple objects which maintains the smallest difference in trajectory between camera and object;
- and
- (v) performing camera autofocusing based on the position and trajectory of said main object.
9 more elements
A method for selecting a main object within the field of view of a camera apparatus, comprising:
- (a) inputting an image captured by an image sensor of a camera into a multiple-branch, multiple-stage convolution neural network (CNN) which is configured for predicting anatomical relationships and generating pose information;
- (b) tracking bounding boxes of multiple objects within an image using a recursive state-space model in combination with a matching algorithm to estimate intersections over union distances (IoU) between the multiple objects;
- (c) determining trajectory similarities between a physical trajectory of the camera and the trajectory of each of said multiple objects by obtaining a camera trajectory and trajectories of each of said multiple objects, followed by a dynamic time warping process to estimate trajectory differences across frames;
- (d) selecting a main object of focus as the object from said multiple objects which maintains the smallest difference in trajectory between the camera and object;
- and
- (e) performing camera autofocusing based on the position and trajectory of said main object.
3 more elements
Dependencies are inferred from the claim text.