# US 2017/0286798 A1 · Objective assessment method for color image quality based on online manifold learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0286798 A1 - Document ID: US20170286798A1 - Status: Published application - Kind: A1 - Application number: 15/197,604 - Filed: 2016-06-29 - Published: 2017-10-05 - Earliest claimed: 2016-03-31 - Assignees: Ningbo University - Applicants: Ningbo University - Inventors: Gangyi Jiang; Meiling He; Fen Chen; Yang Song - Web page: https://patentgrep.com/patent/US20170286798A1 ## CPC - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06F 18/2135: based on approximation criteria, e.g. principal component analysis - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis - G06V 10/56: relating to colour - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods} - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06V 40/172: {Classification, e.g. identification} - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/20081: Training; Learning - G06T 2207/30168: Image quality inspection - G06V 10/513: {Sparse representations} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An objective assessment method for a color image quality based on online manifold learning considers a relationship between a saliency and an image quality objective assessment. Through a visual saliency detection algorithm, saliency maps of a reference image and a distorted image are obtained for further obtaining a maximum fusion saliency map. Based on maximum saliencies of image blocks in the maximum fusion saliency map, a saliency difference between each reference image block and a corresponding distorted image block is measured through an absolute difference, and thus reference visual important image blocks and distorted visual important image blocks are screened and extracted. Through manifold eigenvectors of the reference visual important image blocks and the distorted visual important image blocks, an objective quality assessment value of the distorted image is calculated. The method has an increased assessment effect and a higher correlation between an objective assessment result and a subjective perception. ## Claims ### Claim 1 An objective assessment method for a color image quality based on online manifold learning, comprising steps of: {circle around (1)} representing an undistorted reference image having a width W and a height H by IR; and representing a distorted image to be assessed, which is corresponding to the IR, by ID; {circle around (2)} through a visual saliency detection algorithm, respectively obtaining saliency maps of the IR and the ID, correspondingly denoted as MR and MD; then, according to the MR and the MD, calculating a maximum fusion saliency map, denoted as MF; and denoting a pixel value of a pixel having coordinates of (x, y) in the MF as MF(x, y), MF(x, y)=max (MR(x, y), MD(x, y)), wherein: 1≦x≦W, 1≦y≦H; the max( ) is a function to find a maximum; the MR(x, y) represents a pixel value of a pixel having coordinates of (x, y) in the MR; and the MD(x, y) represents a pixel value of a pixel having coordinates of (x, y) in the MD; {circle around (3)} respectively dividing the IR, the ID, the MR, the MD, and the MF into W × H 8 × 8 image blocks, which are not overlapping mutually and have a size of 8×8; vectorizing color values of R, G, and B channels of all pixels in each image block of the IR and the ID; denoting a color vector obtained through vectorizing the color values of the R, the G, and the B channels of all the pixels in a jth image block of the IR as XjR; denoting a color vector obtained through vectorizing the color values of the R, the G, and the B channels of all the pixels in a jth image block of the ID as XjD; wherein: the j has an initial value of 1, 1 ≤ j ≤ W × H 8 × 8 ; both of the XjR and the XjD have a dimensionality of 192×1; values of a 1st element to a 64th element in the XjR respectively correspond to the color value of the R channel of each pixel in the jth image block of the IR in a line-by-line scanning manner; values of a 65th element to a 128th element in the XjR respectively correspond to the color value of the G channel of each pixel in the jth image block of the IR in the line-by-line scanning manner; values of a 129th element to a 192nd element in the XjR respectively correspond to the color value of the B channel of each pixel in the jth image block of the IR in the line-by-line scanning manner; values of a 1st element to a 64th element in the XjD respectively correspond to the color value of the R channel of each pixel in the jth image block of the ID in the line-by-line scanning manner; values of a 65th element to a 128th element in the XjD respectively correspond to the color value of the G channel of each pixel in the jth image block of the ID in the line-by-line scanning manner; and values of a 129th element to a 192nd element in the XjD respectively correspond to the color value of the B channel of each pixel in the jth image block of the ID in the line-by-line scanning manner; and vectorizing pixel values of all pixels in each image block of the MR, the MD, and the MF; denoting a pixel value vector obtained through vectorizing the pixel values of all the pixels in a jth image block of the MR as SjR; denoting a pixel value vector obtained through vectorizing the pixel values of all the pixels in a jth image block of the MD as SjD; and denoting a pixel value vector obtained through vectorizing the pixel values of all the pixels in a jth image block of the MF as SjF; wherein: the SjR, the SjD, and the SjF all have a dimensionality of 64×1; values of a 1st element to a 64th element in the SjR respectively correspond to the pixel value of each pixel in the jth image block of the MR in the line-by-line scanning manner; values of a 1st element to a 64th element in the SjD respectively correspond to the pixel value of each pixel in the jth image block of the MD in the line-by-line scanning manner; and values of a 1st element to a 64th element in the SjF respectively correspond to the pixel value of each pixel in the jth image block of the MF in the line-by-line scanning manner; {circle around (4)} calculating a saliency of each image block in the MF; and denoting the saliency of the jth image block in the MF as dj, d j = ∑ i = 1 64  S j F  ( i ) , wherein: 1≦i≦64 and the SjF(i) represents a value of an ith element in the SjF; orderly arranging the saliencies of all the image blocks in the MF, from the biggest to the smallest; and, after arranging, determining sequence numbers of the image blocks corresponding to former t1 saliencies, wherein: t 1 = λ 1 × W × H 8 × 8 ; the λ1 represents an image block selection proportionality coefficient; and λ1ε(0,1]; and finding the image blocks in the IR, which are corresponding to the determined t1 sequence numbers, and defining as reference image blocks; finding the image blocks in the ID, which are corresponding to the determined t1 sequence numbers, and defining as distorted image blocks; finding the image blocks in the MR, which are corresponding to the determined t1 sequence numbers, and defining as reference saliency image blocks; finding the image blocks in the MD, which are corresponding to the determined t1 sequence numbers, and defining as distorted saliency image blocks; {circle around (5)} measuring a saliency difference between each reference image block in the IR and a corresponding distorted image block in the ID through an absolute difference; and denoting a saliency difference between a t′th reference image block in the IR and a t′th distorted image block in the ID as et′, e t ′ = 1 64  ∑ i = 1 64   S ~ t ′ R  ( i ) - S ~ t ′ D  ( i )  , wherein: the t′ has an initial value of 1, 1≦t′≦t1; the symbol “∥” is an absolute value symbol; the {tilde over (S)}t′R(i) represents a value of an ith element in a pixel value vector {tilde over (S)}t′R corresponding to a t′th reference saliency image block in the MR; and the {tilde over (S)}t′D(i) represents a value of an ith element in a pixel value vector {tilde over (S)}t′D corresponding to a t′th distorted saliency image block in the MD; and orderly arranging the measured t1 saliency differences, from the biggest to the smallest; after arranging, determining the reference image blocks and the distorted image blocks corresponding to former t2 saliency differences; defining the determined t2 reference image blocks as reference visual important image blocks, and adopting a matrix formed by color vectors corresponding to all the reference visual important image blocks as a reference visual important image block matrix, denoted as YR; defining the determined t2 distorted image blocks as distorted visual important image blocks, and adopting a matrix formed by color vectors corresponding to all the distorted visual important image blocks as a distorted visual important image block matrix, denoted as YD; wherein: t2=λ2×t1, the λ2 represents a selection proportionality coefficient of the reference image blocks and the distorted image blocks, and λ2ε(0,1]; the YR and the YD have a dimensionality of 192×t2; a t″th column vector in the YR is a color vector corresponding to a determined t″th reference visual important image block; a t″th column vector in the YD is a color vector corresponding to a determined t″th distorted visual important image block; and the t″ has an initial value of 1, 1≦t″≦t2; {circle around (6)} centralizing the YR through subtracting a mean value of values of all elements in each column vector from a value of each element in the same column vector of the YR; and denoting an obtained centralized matrix as Y, wherein the Y has a dimensionality of 192×t2; and processing the Y with dimensionality reduction and whitening through a principal component analysis; and, denoting an obtained matrix after the dimensionality reduction and the whitening as Yw, Yw=W×Y; wherein: the Yw has a dimensionality of M×t2; the W represents a whitened matrix and has a dimensionality of M×192, 1