# US 10,311,560 B2 · Method and system for estimating blur kernel size

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,560 B2
- Document ID: US10311560B2
- Status: Granted
- Kind: B2
- Application number: 15/709,454
- Filed: 2017-09-19
- Published: 2018-03-08
- Granted: 2019-06-04
- Earliest claimed: 2016-09-07
- Assignees: Huazhong University of Science and Technology
- Applicants: Huazhong University of Science and Technology
- Inventors: Nong Sang; Lerenhan Li; Luxin Yan; Changxin Gao; Yuanjie Shao; Juncai Peng; Shiwei Zhang; Jin Wang
- Web page: https://patentgrep.com/patent/US10311560B2

## CPC

- G06T 5/73: Deblurring; Sharpening
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 3/02: Neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 10/30: Noise filtering
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/82: using neural networks
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for estimating a blur kernel size, the method including: (1) pre-processing a blurred image, to obtain an image, so that a size of the image satisfies an image input size of a multi-class convolutional neural network (CNN); (2) inputting the image into a multi-class CNN with completed training, to obtain a blur-kernel-size probability distribution vector; and (3) comparing each element in the blur-kernel-size probability distribution vector, so that an estimated blur kernel size of the blurred image is the blur kernel size corresponding to a largest element. The invention also provides a system for estimating a blur kernel size. The system includes an image pre-processing module, a training-set synthesizing module, a multi-class CNN module, and a blur kernel size estimation module.

## Claims

### Claim 1

A method for estimating a blur kernel size, the method comprising:
(1) pre-processing a blurred image B, to obtain an image B0, so that a size of the image B0 satisfies an image input size of a multi-class convolutional neural network (CNN);
(2) inputting the image B0 into a multi-class CNN with completed training, to obtain a blur-kernel-size probability distribution vector; and
(3) comparing each element in the blur-kernel-size probability distribution vector, so that an estimated blur kernel size of the blurred image B is the blur kernel size corresponding to a largest element.

### Claim 2 (depends on 1)

The method of claim 1, wherein the image B0 is input into a multi-class CNN, to obtain the blur-kernel-size probability distribution vector as follows:
P
⁡
(
B
size
=
(
s
1
,
…
⁢
,
s
m
)
|
B
0
)
=
e
(
ω
(
1
,
⁢
…
⁢
,
⁢
m
)
soft
-
max
)
T
⁢
x
full
-
connect
⁢
⁢
(
B
0
)
∑
t
=
1
m
⁢
⁢
e
(
ω
m
soft
-
max
)
T
⁢
x
full
-
connect
⁢
⁢
(
B
0
)
=
(
p
1
,
…
⁢
,
p
m
)
where, ωisoft-max represents weight of a soft-max layer; xfull-connect( ) represents an output of a full-connect layer; T represents matrix transpose; m represents the number of blur kernel size classes; each element in the obtained blur-kernel-size probability distribution vector represents the probability that the blur kernel size of the blurred image B is the blur kernel size corresponding to that element.

### Claim 3 (depends on 1)

The method of claim 1, wherein the training of the multi-class CNN comprises:
(31) selecting a training set for the multi-class CNN
T={t1, . . . , tn}, n=u×v×m,
where, T represents a two-dimensional convolution operation of randomly collected natural scene images in number of u and randomly generated blur kernels in number of v×m; u represents the number of images; m represents size classes of blur kernels; v represents the number of blur kernels in each size class; blurred images generated by the blur kernels in the same size class is labeled as an image category; then, pre-processing the training images obtained by the two-dimensional convolution operation, so that the sizes of the training images satisfy the image input size of the multi-class CNN;
(32) inputting one training image t1 labeled in sl blur kernel size class into the multi-class CNN, to obtain a probability distribution sequence vector of each blur kernel size class: P(Bsize=(s1, . . . , sl, . . . , sm)|t1), and adjusting the weight of each layer of the network, so that the element corresponding to the blur kernel size class sl of the training image t1 is the largest element in the distribution sequence vector; and
(33) repeating the step (32) to sequentially input all images in the training set T={t1, . . . , tn}, n=u×v×m, to complete the training of the multi-class CNN.

### Claim 4 (depends on 3)

The method of claim 3, wherein the image pre-processing is using a down-sampling approach to obtain an input image, when the size of the blurred image to be processed is smaller than N times of the image input size M of the multi-class CNN; otherwise using a size interception approach to intercept the image containing blurred information to get an input image.

### Claim 5 (depends on 4)

The method of claim 4, wherein the down-sampling approach is a bilinear interpolation down-sampling approach.

### Claim 6 (depends on 1)

The method of claim 1, wherein the image pre-processing is using a down-sampling approach to obtain an input image, when the size of the blurred image to be processed is smaller than N times of the image input size M of the multi-class CNN; otherwise using a size interception approach to intercept the image containing blurred information to get an input image.

### Claim 7 (depends on 6)

The method of claim 6, wherein the down-sampling approach is a bilinear interpolation down-sampling approach.

### Claim 8

A system for estimating a blur kernel size, the system comprising:
an image pre-processing module, which is used for pre-processing a blurred image, by using a down-sampling approach to obtain an input image, when the size of the blurred image is smaller than N times of the image input size M of a multi-class CNN, otherwise by using a size interception approach to intercept an image containing blurred information to get an input image;
a training-set synthesizing module, which is used for generating a training image set for training demands
T={t1, . . . , tn}, n=u×v×m,
where, T represents a two-dimensional convolution operation of randomly collected natural scene images in number of u and randomly generated blur kernels in number of v×m; u represents the number of images; m represents size classes of blur kernels; v represents the number of blur kernels in each size class; blurred images generated by the blur kernels in the same size class is labeled as an image category; and then inputting the training images obtained by the two-dimensional convolution operation into the image pre-processing module to obtain input images, so that the sizes of the input images satisfy the input image size of the multi-class CNN;
a multi-class CNN module, which is used in the training, for sequentially inputting the training image set T={t1, . . . , tn}, n=u×v×m into the multi-class CNN, to obtain a probability distribution sequence vector of each blur kernel size class: P(Bsize=(s1, . . . , sl, . . . , sm)|T), and adjusting the weight of each layer of the network, so that the element corresponding to the blur kernel size class sl in the training image set T is the largest element in the distribution sequence vector;
and after completion of the training, inputting an image B0 into the multi-class CNN, to obtain a probability distribution vector as follows:
P
⁡
(
B
size
=
(
s
1
,
…
⁢
,
s
m
)
|
B
0
)
=
e
(
ω
(
1
,
⁢
…
⁢
,
⁢
m
)
soft
-
max
)
T
⁢
x
full
-
connect
⁢
⁢
(
B
0
)
∑
t
=
1
m
⁢
⁢
e
(
ω
m
soft
-
max
)
T
⁢
x
full
-
connect
⁢
⁢
(
B
0
)
=
(
p
1
,
…
⁢
,
p
m
)
,
where, ωisoft-max represents weight of a soft-max layer; xfull-connect( ) represents an output of a full-connect layer; T represents matrix transpose; m represents the number of blur kernel size classes; each element in the obtained blur-kernel-size probability distribution vector represents the probability that the blur kernel size of the blurred image B is the blur kernel size corresponding to that element; and
a blur kernel size estimation module, which is used for blur kernel size estimation, and for comparing each element in the probability distribution vector, so that the estimated blur kernel size of the blurred image B is the blur kernel size corresponding to a largest element.

### Claim 9 (depends on 8)

The system of claim 8, wherein the down-sampling approach is a bilinear interpolation down-sampling approach.