# US 2021/0344844 A1 · Photographing method for compensating dynamic scenes and a camera device using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0344844 A1
- Document ID: US20210344844A1
- Status: Published application
- Kind: A1
- Application number: 17/034,280
- Filed: 2020-09-28
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: Hongfujin Precision Electronics (Zhengzhou) Co., Ltd.; Hon Hai Precision Industry Co., Ltd.
- Applicants: Hongfujin Precision Electronics (Zhengzhou) Co., Ltd.; Hon Hai Precision Industry Co., Ltd.
- Inventors: YU ZHANG
- Web page: https://patentgrep.com/patent/US20210344844A1

## CPC

- G06T 5/73: Deblurring; Sharpening
- H04N 23/632: {for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06T 5/20: using local operators
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/74: {involving reference images or patches}
- H04N 23/61: based on recognised objects
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- H04N 23/80: Camera processing pipelines; Components thereof
- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20081: Training; Learning
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In capturing images, a method applied in a camera device taking images of a dynamic scenes which provides compensation for blurring and lack of clarity caused by movement of objects within the scene. From images captured in preview mode, determining whether a target object is a dynamic object. A preset number of the preview images are stored as first images of a target object determined as being dynamic, and a second image is also taken, the second image being analyzed for an area of fuzziness. The camera device takes feature information from each first image and the feature information corresponding to the area of fuzziness of the second image and combines feature information having a highest similarity with the fuzzy area to generate a final target image with sharpness and full clarity, and outputting the target image.

## Claims

### Claim 1

A camera device, comprising:
a camera;
a preview analysis unit;
an image optimization unit;
a processor configured to the camera, the preview analysis unit, and the image optimization unit;
a non-transitory storage medium coupled to the processor and configured to store a plurality of instructions, which causes the processor to:
activate the camera and activate a preview mode of the camera;
obtain preview images of the preview mode from the camera;
identify a target object in the preview images by the preview analysis unit, and determine whether the target object is a dynamic object;
store a preset number of the preview images as first images when the target object is a dynamic object, and store a second image taken by the camera when the camera receiving an image-capturing instruction;
divide each of the first images to obtain a plurality of first sub-images and divide the second image to obtain a plurality of second sub-images by the image optimization unit;
compare the plurality of first sub-images in each of the first images with the plurality of second sub-images of the second image by an edge sharpness and a clarity calculation to determine whether the second image has a fuzzy area range;
compare the plurality of first sub-images in each of the first images with the plurality of the second sub-images in the second image to determine feature information in the first images and the feature information corresponds to the fuzzy area range of the second image;
determine the feature information having a highest similarity with the fuzzy region range as the target feature information, and combines the target feature information with the second image to generate the target image;
output the target image.

### Claim 2 (depends on 1)

The camera device according to claim 1, wherein the plurality of instructions is further configured to cause the processor to:
when the second image doesn't have a fuzzy area range, output the second image.

### Claim 3 (depends on 1)

The camera device according to claim 1, wherein the plurality of instructions is further configured to cause the processor to:
analyze each position of the target object in the preview images;
determine whether the target object is a dynamic object according to each position of the target object in the preview images;
when the position of the target object in the preview images changes, determine the target object as a dynamic object; and
when the position of the target object in the preview images doesn't change, determine the target object to be a static object.

### Claim 4 (depends on 1)

The camera device according to claim 1, wherein the plurality of instructions is further configured to cause the processor to:
compare the plurality of the first sub-images in each of the first images with the plurality of the second sub-images in the second image to determine a target location area of the target object corresponding to the fuzzy area of the second image; and
extract feature points of the target location area of each of the first images and characterize a feature description of the target location area of each of the first images to obtain the feature information of the first images.

### Claim 5 (depends on 4)

The camera device according to claim 4, wherein the plurality of instructions is further configured to cause the processor to:
extract the feature points of the target location area of each of the first images to obtain the feature information of the first images by an Oriented FAST and Rotated BRIEF algorithm.

### Claim 6 (depends on 5)

The camera device according to claim 5, wherein the plurality of instructions is further configured to cause the processor to:
take any point in the target location area of each of the first images as the target point, draw a circle at a center of the target point to generate a feature point circumference including the feature points, and filter out the feature points from the feature point circumference;
train a decision tree by an ID3 algorithm, and filter out an optimal FAST feature point from the feature points according to the decision tree;
remove multiple feature points near the FAST feature points by a non-significant value suppression algorithm;
set a scale factor and the number of layers of an image pyramid, zoom out each of the first images into a plurality of sub-images according to the scale factor, extract a total of the feature points from the plurality of the sub-images in different proportions as an oFAST feature point of each first image;
obtain details of changes of the feature points of each of the first images by describing the oFAST feature point, and obtain a pixel position having a largest grayscale value from the oFAST feature point, and calculate the feature information of the feature point by using the edge sharpness and the clarity calculation according to the pixel position having the largest grayscale value.

### Claim 7 (depends on 1)

The camera device according to claim 1, wherein the plurality of instructions is further configured to cause the processor to:
combine the target feature information with the second image by a Laplace image fusion algorithm to generate the target image.

### Claim 8

A photographing method for compensating dynamic scenes, the method comprising:
activating a camera and activating a preview mode of the camera;
obtaining preview images of the preview mode from the camera;
identifying a target object in the preview images by a preview analysis unit, and determining whether the target object is a dynamic object;
storing a preset number of the preview images as first images when the target object is a dynamic object, and storing a second image taken by the camera when the camera receiving an image-capturing instruction;
dividing each of the first images to obtain a plurality of first sub-images and dividing the second image to obtain a plurality of second sub-images by an image optimization unit;
comparing the plurality of first sub-images in each of the first images with the plurality of second sub-images of the second image by an edge sharpness and a clarity calculation to determine whether the second image has a fuzzy area range;
comparing the plurality of first sub-images in each of the first images with the plurality of the second sub-images in the second image to determine feature information in the first images and the feature information corresponding to the fuzzy area range of the second image;
determining the feature information having a highest similarity with the fuzzy region range as the target feature information, and combining the target feature information with the second image to generate the target image;
outputting the target image.

### Claim 9 (depends on 8)

The photographing method for compensating dynamic scenes according to claim 8, further comprising:
outputting the second image when the second image doesn't have a fuzzy area range.

### Claim 10 (depends on 8)

The photographing method for compensating dynamic scenes according to claim 8, further comprising:
analyzing each position of the target object in the preview images;
determining whether the target object is a dynamic object according to each position of the target object in the preview images;
determining the target object as a dynamic object when the position of the target object in the preview images changes; and
determining the target object to be a static object when the position of the target object in the preview images doesn't change.

### Claim 11 (depends on 8)

The photographing method for compensating dynamic scenes according to claim 8, further comprising:
comparing the plurality of the first sub-images in each of the first images with the plurality of the second sub-images in the second image to determine a target location area of the target object corresponding to the fuzzy area of the second image; and
extracting feature points of the target location area of each of the first images and characterizing a feature description of the target location area of each of the first images to obtain the feature information of the first images.

### Claim 12 (depends on 11)

The photographing method for compensating dynamic scenes according to claim 11, further comprising:
extracting the feature points of the target location area of each of the first images to obtain the feature information of the first images by an Oriented FAST and Rotated BRIEF algorithm.

### Claim 13 (depends on 12)

The photographing method for compensating dynamic scenes according to claim 12, further comprising:
taking any point in the target location area of each of the first images as the target point, drawing a circle at a center of the target point to generate a feature point circumference including the feature points, and filtering out the feature points from the feature point circumference;
training a decision tree by an ID3 algorithm, and filtering out an optimal FAST feature point from the feature points according to the decision tree;
removing multiple feature points near the FAST feature points by a non-significant value suppression algorithm;
setting a scale factor and the number of layers of an image pyramid, zooming out each of the first images into a plurality of sub-images according to the scale factor, extracting a total of the feature points from the plurality of the sub-images in different proportions as an oFAST feature point of each first image;
obtaining details of changes of the feature points of each of the first images by describing the oFAST feature point, and obtaining a pixel position having a largest grayscale value from the oFAST feature point, and calculating the feature information of the feature point by using the edge sharpness and the clarity calculation according to the pixel position having the largest grayscale value.

### Claim 14 (depends on 12)

The photographing method for compensating dynamic scenes according to claim 12, further comprising:
combining the target feature information with the second image by a Laplace image fusion algorithm to generate the target image.

### Claim 15

A non-transitory storage medium having stored thereon instructions that, when executed by at least one processor of a camera device, causes the least one processor to execute instructions of a photographing method for compensating dynamic scenes, the photographing method for compensating dynamic scenes comprising:
activating a camera and activating a preview mode of the camera;
obtaining preview images of the preview mode from the camera;
identifying a target object in the preview images by a preview analysis unit, and determining whether the target object is a dynamic object;
storing a preset number of the preview images as first images when the target object is a dynamic object, and storing a second image taken by the camera when the camera receiving an image-capturing instruction;
dividing each of the first images to obtain a plurality of first sub-images and dividing the second image to obtain a plurality of second sub-images by an image optimization unit;
comparing the plurality of first sub-images in each of the first images with the plurality of second sub-images of the second image by an edge sharpness and a clarity calculation to determine whether the second image has a fuzzy area range;
comparing the plurality of first sub-images in each of the first images with the plurality of the second sub-images in the second image to determine feature information in the first images and each characteristic information corresponding to the fuzzy area range of the second image;
determining the feature information having a highest similarity with the fuzzy region range as the target feature information, and combining the target feature information with the second image to generate the target image;
outputting the target image.

### Claim 16 (depends on 15)

The non-transitory storage medium according to claim 15, wherein the photographing method for compensating dynamic scenes further comprising:
outputting the second image when the second image doesn't have a fuzzy area range.

### Claim 17 (depends on 15)

The non-transitory storage medium according to claim 15, wherein the photographing method for compensating dynamic scenes further comprising:
analyzing each position of the target object in the preview images;
determining whether the target object is a dynamic object according to each position of the target object in the preview images;
determining the target object as a dynamic object when the position of the target object in the preview images changes; and
determining the target object to be a static object when the position of the target object in the preview images doesn't change.

### Claim 18 (depends on 15)

The non-transitory storage medium according to claim 15, wherein the photographing method for compensating dynamic scenes further comprising:
comparing the plurality of the first sub-images in each of the first images with the plurality of the second sub-images in the second image to determine a target location area of the target object corresponding to the fuzzy area of the second image; and
extracting feature points of the target location area of each of the first images and characterizing a feature description of the target location area of each of the first images to obtain the feature information of the first images.

### Claim 19 (depends on 18)

The non-transitory storage medium according to claim 18, wherein the photographing method for compensating dynamic scenes further comprising:
extracting the feature points of the target location area of each of the first images to obtain the feature information of the first images by an Oriented FAST and Rotated BRIEF algorithm.

### Claim 20 (depends on 15)

The non-transitory storage medium according to claim 15, wherein the photographing method for compensating dynamic scenes further comprising:
combining the target feature information with the second image by a Laplace image fusion algorithm to generate the target image.