# US 2014/0153823 A1 · Method and apparatus for processing image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0153823 A1 - Document ID: US20140153823A1 - Status: Published application - Kind: A1 - Application number: 13/915,993 - Filed: 2013-06-12 - Published: 2014-06-05 - Earliest claimed: 2012-11-30 - Assignees: Industry-Academic Cooperation Foundation, Yonsei University; Samsung Techwin Co., Ltd. - Applicants: Industry-Academic Cooperation Foundation, Yonsei University; Samsung Techwin Co., Ltd. - Inventors: Jeong-Ahn LEE; Moon-gi KANG; Sang-wook PARK; Paul OH - Web page: https://patentgrep.com/patent/US20140153823A1 ## CPC - G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing apparatus includes an adaptive interpolation device which converts a MFA pattern image into a quincuncial pattern image based on difference values, and interpolates color channels and an NIR channel, based on difference values of the converted quincuncial pattern image in vertical and horizontal pixel directions; a frequency compensation device which obtains a high-resolution MFA image using high-frequency and medium-frequency components of a high-resolution base image, based on linear regression analysis and compared energy levels of MFA channel images to an energy level of a base image; and a channel interference suppression device which removes color distortion generated between each channel of the high-resolution MFA image, and another channel of the high-resolution MFA image and a base channel using a weighted average of pixel value differences between each channel of the high-resolution MFA image, and the other channel of the high-resolution MFA image and the base channel. ## Claims ### Claim 1 An image processing apparatus comprising: an adaptive interpolation device configured to convert a multispectral filter array (MFA) pattern image into a quincuncial pattern image based on difference values in a diagonal pixel direction, and interpolate color channels and a near infrared (NIR) channel of the MFA pattern image to a maximum resolution, based on difference values of the converted quincuncial pattern image in vertical pixel directions and horizontal pixel directions; a frequency compensation device configured to obtain a high-resolution MFA image using high-frequency components and medium-frequency components of a high-resolution base image corresponding to high-frequency components and medium-frequency components of a low-resolution MFA image interpolated by the adaptive interpolation device, based on linear regression analysis and a result of comparing energy levels of MFA channel images to an energy level of a base image; and a channel interference suppression device configured to remove color distortion generated between each channel of the high-resolution MFA image, and a first channel of the high-resolution MFA image and a base channel using a weighted average of pixel value differences between each channel of the high-resolution MFA image, and a second channel of the high-resolution MFA image and the base channel. ### Claim 2 (depends on 1) The image processing apparatus of claim 1, wherein the quincuncial pattern image is an image having a same geometric model as a Bayer pattern image, and having different channel values from the Bayer pattern image. ### Claim 3 (depends on 1) The image processing apparatus of claim 1, wherein, when the MFA pattern image is converted into the quincuncial pattern image, the adaptive interpolation device uses a sum of a first weighted average of values of pixels spaced apart from a certain pixel (m,n) of the MFA pattern image by one pixel in a diagonal direction, and a second weighted average, obtained by calculating a weighted average of difference values between the certain pixel (m,n) of the MFA pattern image and pixels spaced apart from the certain pixel (m,n) of the MFA pattern image by two pixels in the diagonal direction. ### Claim 4 (depends on 3) The image processing apparatus of claim 3, wherein each weight used to calculate the first weighted average and the second weighted average is inversely proportional to a difference value in each pixel direction. ### Claim 5 (depends on 1) The image processing apparatus of claim 1, wherein the frequency compensation device generates the high-resolution MFA image by comparing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device to medium-frequency components and high-frequency components of the base image in each of vertical, horizontal, and diagonal directions, calculating a weighted sum of corresponding parts, and reconstructing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device. ### Claim 6 (depends on 1) The image processing apparatus of claim 1, wherein the frequency compensation device provides a weight to each of the medium-frequency components and the high-frequency components of the low-resolution MFA image and medium-frequency components and high-frequency components of the base image, and wherein the weight is set such that a large weight is provided to an edge of the low-resolution MFA image or an edge of the base image, whichever has a higher energy level. ### Claim 7 (depends on 1) The image processing apparatus of claim 1, wherein the frequency compensation device calculates a ratio between energy levels of medium-frequency components and high-frequency components of the base image and the MFA image, using the linear regression analysis, and provides a weight for correcting an energy level difference. ### Claim 8 (depends on 7) The image processing apparatus of claim 7, wherein the frequency compensation device removes color distortion between channels by strongly reconstructing medium-frequency components and high-frequency components of a channel having a high energy level of the high-frequency components, and weakly reconstructing medium-frequency components and high-frequency components of a channel having a low energy level of the high-frequency components using the weight for correcting the energy level difference, so as to allow edges of channels to be uniform in consideration of energy levels of red (R), green (G), blue (B), and NIR channels of the MFA image. ### Claim 9 An image processing method comprising: converting a multispectral filter array (MFA) pattern image into a quincuncial pattern image based on difference values in a diagonal pixel direction, and interpolating color channels and a near infrared (NIR) channel of the MFA pattern image to a maximum resolution, based on difference values of the converted quincuncial pattern image in vertical pixel directions and horizontal pixel directions, using an adaptive interpolation device; obtaining a high-resolution MFA image using high-frequency components and medium-frequency components of a high-resolution base image corresponding to high-frequency and medium-frequency components of a low-resolution MFA image interpolated by the adaptive interpolation device, based on linear regression analysis and a result of comparing energy levels of MFA channel images to an energy level of a base image, using a frequency compensation device; and removing color distortion generated between each channel of the high- resolution MFA image, and a first channel of the high-resolution MFA image and a base channel using a weighted average of pixel value differences between each channel of the high-resolution MFA image, and a second channel of the high-resolution MFA image and the base channel, using a channel interference suppression device. ### Claim 10 (depends on 9) The image processing method of claim 9, wherein the quincuncial pattern image is an image having a same geometric model as a Bayer pattern image, and having different channel values from the Bayer pattern image. ### Claim 11 (depends on 9) The image processing method of claim 9, wherein, when the MFA pattern image is converted into the quincuncial pattern image, a sum of a first weighted average of values of pixels spaced apart from a certain pixel (m,n) of the MFA pattern image by one pixel in a diagonal direction, and a second weighted average, obtained by calculating a weighted average of difference values between the certain pixel (m,n) of the MFA pattern image and pixels spaced apart from the certain pixel (m,n) of the MFA pattern image by two pixels in the diagonal direction, is used. ### Claim 12 (depends on 11) The image processing method of claim 11, wherein each weight used to calculate the first weighted average and the second weighted average is inversely proportional to a difference value in each pixel direction. ### Claim 13 (depends on 9) The image processing method of claim 9, wherein the obtaining of the high-resolution MFA image comprises generating the high-resolution MFA image by comparing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device to medium-frequency components and high-frequency components of the base image in each of vertical, horizontal, and diagonal directions, calculating a weighted sum of corresponding parts, and reconstructing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device. ### Claim 14 (depends on 9) The image processing method of claim 9, wherein the obtaining of the high-resolution MFA image comprises providing a weight to each of the medium-frequency components and the high-frequency components of the low-resolution MFA image and medium-frequency components and high-frequency components of the base image, and wherein the weight is set such that a large weight is provided to an edge of the low-resolution MFA image or an edge of the base image, whichever has a higher energy level. ### Claim 15 (depends on 9) The image processing method of claim 9, wherein the obtaining of the high-resolution MFA image comprises calculating a ratio between energy levels of medium-frequency components and high-frequency components of the base image and the MFA image, using the linear regression analysis, and providing a weight for correcting an energy level difference. ### Claim 16 (depends on 15) The image processing method of claim 15, wherein the obtaining of the high-resolution MFA image comprises removing color distortion between channels by strongly reconstructing medium-frequency components and high-frequency components of a channel having a high energy level of the high-frequency components, and weakly reconstructing medium-frequency components and high-frequency components of a channel having a low energy level of the high-frequency components using the weight for correcting the energy level difference, so as to allow edges of channels to be uniform in consideration of energy levels of red (R), green (G), blue (B), and NIR channels of the MFA image.