# US 9,436,981 B2 · Dictionary creation device, image processing device, image processing system, dictionary creation method, image processing method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,981 B2 - Document ID: US9436981B2 - Status: Granted - Kind: B2 - Application number: 14/363,367 - Filed: 2012-12-10 - Published: 2014-10-30 - Granted: 2016-09-06 - Earliest claimed: 2011-12-12 - Assignees: NEC Corporation - Applicants: Takashi Shibata; Akihiko Iketani; Shuji Senda - Inventors: Takashi Shibata; Akihiko Iketani; Shuji Senda - Web page: https://patentgrep.com/patent/US9436981B2 ## CPC - G06T 5/73: Deblurring; Sharpening - G06N 20/00: Machine learning - G06T 5/60: using machine learning, e.g. neural networks - G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole} - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A dictionary creation device including a blurred image generation unit which outputs a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image, a patch pair generation unit which generates a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image, and a registration unit which associates the patch pair with a blur parameter corresponding to the blurred patch in the patch pair and registers them in a dictionary. ## Claims ### Claim 1 A dictionary creation device comprising: a processor configured to: output a blurred image to a learning image together with a blur parameter indicating a blurring state of the blurred image; generate a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image; and associate the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary. ### Claim 2 (depends on 1) The dictionary creation device according to claim 1 wherein the processor is further configured to generate the blurred image by performing at least one among a size reduction process, a lightness reducing process, a high frequency component eliminating process, and a posture variation process to the learning image and the blur parameter includes a value indicating at least one among a scaling factor, lightness, a degree of blur, and a posture slope. ### Claim 3 (depends on 1) The dictionary creation device according to claim 1 wherein the processor is further configured to generate the patch pair by using the restoration patch that is generated by shifting the patch by one pixel column in the learning image. ### Claim 4 An image processing device comprising: a processor configured to: estimate a value of a blur parameter indicating a blurring state with respect to an input image that is an image received from outside; divide the input image and generate a plurality of input patches; refer to a dictionary storing two or more sets of data in which a blurred patch generated from a learning image, a restoration patch, and the blur parameter of the blurred patch are associated with each other and select the restoration patch for each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary; and combine the restoration patches selected for each input patch and generate the restored image. ### Claim 5 (depends on 4) The image processing device according to claim 4 wherein the blur parameter includes values indicating a scaling factor, lightness, a degree of blur, and a posture slope. ### Claim 6 (depends on 5) The image processing device according to claim 5 wherein the processor is further configured to estimate the value of the blur parameter of the input image by calculating the scaling factor based on an aspect ratio of the input image and an aspect ratio of the learning image, calculating the lightness based on an average value of brightness values of the input image, calculating the degree of blur based on a magnitude of a high frequency component of the input image, or calculating the posture slope by estimating a direction. ### Claim 7 (depends on 5) The image processing device according to claim 5 wherein the processor is further configured to calculate the value indicating the similarity of the blur parameter by using a value weighted by a weighting factor set for each blur parameter. ### Claim 8 (depends on 7) The image processing device according to claim 7 wherein the processor is further configured to set a higher weighting factor for the scaling factor and the lightness than for the degree of blur and the posture slope. ### Claim 9 (depends on 7) The image processing device according to claim 7 wherein the processor is further configured to receive an input of the weighting factor set by a user. ### Claim 10 (depends on 4) The image processing device according to claim 4 wherein the processor is further configured to select the restoration patch in only a range in which it is shown that the blur parameter is similar that is determined by a comparison between a value indicating a similarity of the blur parameter and a predetermined threshold value. ### Claim 11 An image processing system including a dictionary creation device and an image processing device wherein the dictionary creation device configured to: output a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image, generate a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image, and associate the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary and the image processing device is configured to: estimate value of a blur parameter indicating the blurring state with respect to an input image that is an image received from outside, divide the input image and generate a plurality of input patches, refer to the dictionary created by the dictionary creation device and select the restoration patch for each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary, and combine the restoration patches selected for each input patch and generate the restored image. ### Claim 12 A method for creating a dictionary comprising: outputting a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image; generating a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image; and associating the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary. ### Claim 13 A method for processing an image comprising: estimating a value of a blur parameter indicating a blurring state with respect to an input image that is an image received from outside; dividing the input image and generating a plurality of input patches; referring to a dictionary storing two or more sets of data in which a blurred patch that is a patch of a blurred image generated from a learning image, a restoration patch, and the blur each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary; and combining the restoration patches selected for each input patch and generating a restored image. ### Claim 14 A non-transitory computer-readable storage medium storing a program for causing a computer to execute, comprising: outputting a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image; generating a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image; and associating the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary. ### Claim 15 Anon-transitory computer-readable storage medium storing a program for causing a computer to execute, comprising: estimating a value of a blur parameter indicating a blurring state with respect to an input image that is an image received from outside; dividing the input image and generating a plurality of input patches; referring to a dictionary storing two or more sets of data in which a blurred patch that is a patch of a blurred image generated from a learning image, a restoration patch that is a patch of the learning image, and the blur parameter of the blurred patch are associated with each other and selecting the restoration patch for each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary; and combining the restoration patches selected for each input patch and generating the restored image.