# US 2013/0156304 A1 · Method for classification of videos > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2013/0156304 A1 - Document ID: US20130156304A1 - Status: Published application - Kind: A1 - Application number: 13/807,375 - Filed: 2011-06-29 - Published: 2013-06-20 - Earliest claimed: 2010-07-01 - Assignees: Telefonica, S.A. - Inventors: Anush Moorty; Pere Obrador; Nuria Oliver Ramirez - Web page: https://patentgrep.com/patent/US20130156304A1 ## CPC - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06F 18/2115: by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination - G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06F 2218/12: Classification; Matching - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for classifying a video regarding a subjective characteristic, the method comprising: measuring a plurality of basic features (11) per frame thus obtaining a plurality of basic features measurements; creating a plurality of second-level features by pooling (12) said basic features (11) measurements using a plurality of statistics of said basic features measurements in a determined period of time of footage; creating a plurality of video features by pooling (13) said plurality of second-level features using a plurality of statistics of said second level features along the duration of the video; choosing at least one video feature of said plurality of video features for classifying a video regarding a subjective characteristic. ## Claims ### Claim 1 A method for classifying a video regarding a subjective characteristic which is an aesthetic quality of the video perceived by human, the method comprising: measuring a plurality of basic features (11) per frame thus obtaining a plurality of basic features measurements; creating a plurality of second-level features by pooling (12) said basic features (11) measurements using a plurality of statistics of said basic features measurements in a determined period of time of footage; creating a plurality of video features by temporal pooling (13) said plurality of second-level features using a plurality of statistics of said second level features along the duration of the video; creating an additional video feature by measuring the frame rate of said video and using a structural similarity index (SSIM) algorithm as a measure of similarity between frames for computing said frame rate; choosing at least one video feature of said plurality of video features for classifying a video regarding the aesthetic quality, the step of choosing said at least one video feature comprising: comparing, for a determined combination of video features, an output value from a classifier, which is trained using the determined combination of features, with a training set of values predefined by human input. ### Claim 2 (depends on 1) The method of claim 1, wherein said plurality of basic features (11) measurements comprises at least one of the following features measurements: at least one of the following motion features measurements: motion ratio and size ratio, wherein said motion ratio is the ratio of motion magnitudes between the foreground and the background regions of the video and said size ratio is the ratio of sizes of said foreground and background regions; a focus of the region of interest measurement; a colourfulness measurement; a luminance measurement; a colour harmony measurement, wherein said measurement considers the effect that the combination of different colours has on the value of each frame; a blockiness quality measurement, wherein said blockiness is the consequence of portions of a frame breaking into little squares; a rule of thirds measurement, wherein said rule of thirds feature accounts for how important compositional elements of the image should be situated. ### Claim 3 (depends on 2) The method of claim 2, wherein said motion features are computed as follows: performing motion estimation, by low-pass filtering the video frame and down-sampling said video frame by a factor of 2; computing for each pixel location in said frame, the magnitude of the motion vector; running a k-means algorithm with 2 clusters to segregate the motion vectors into two classes; hiseogramming, within each one of said classes, the motion vector magnitudes and choosing the magnitude of the motion vector corresponding to the maximum of the histogram; motion ratio is computed as f2 =(mb+1)/(mf+1), wherein mf and mb denote the magnitude of the motion vectors for each of said classes; size ratio is computed as (sb+1)/(sf+1), wherein sf and sb denote the size (in pixels) of each of said foreground and background regions respectively. ### Claim 4 The method of 2, wherein said focus on the region of interest measurement is computed by extracting said region of interest and setting the median of the level of focus of said region of interest. ### Claim 5 (depends on 2) The method of claim 2, wherein the red, green and blue (RGB) values of the frame are used in the following expressions: α=R−B, β=0.5×(R+G)−B, ma=E[a], mb=E[b], s2a=E[(a−ma)2], and s2b=E[(b−mb)2] wherein, E is the expectation operator, μ is the mean and σ, the standard deviation, to compute said colourfulness feature measurement as: f5=√{square root over (s2a+s2b)}+0.3·√{square root over (m2a+m2b)}; ### Claim 6 (depends on 2) The method of claim 2, wherein said luminance measurement is computed as the mean value of the luminance within a frame. ### Claim 7 (depends on 2) The method of claim 2, wherein said colour harmony measurement is computed as follows: computing the normalized hue-histogram of each frame; performing a convolution of said hue-histogram with each one of seven harmonic templates over the hue channel in the hue, saturation, value (HSV) space; selecting the maximum of said convolution as a measure of similarity of the frame's histogram to one of said particular templates; selecting the maximum value of said measures of similarity as the colour harmony feature value. ### Claim 8 (depends on 2) The method of claim 2, wherein said blockiness quality measurement is computed by looking for blockiness artefacts. ### Claim 9 (depends on 2) The method of claim 2, wherein said rule of thirds measurement is computed as the minimum distance of the centroid of the region of interest to one of the four intersections of the lines that divide the image into nine equal rectangles. ### Claim 10 (depends on 1) The method of claim 1, wherein said plurality of statistics of basic features (11) measurements used to create said second-level features comprises at least one of the following: mean, median, minimum, maximum, first quartile and third quartile. ### Claim 11 (depends on 1) The method of claim 1, wherein said plurality of statistics of second-level features measurements used to create video features comprises average and standard deviation. ### Claim 12 (depends on 1) The method of claim 1, wherein the frame rate together with the following video features are selected to characterize a video as high/low appealing, said following video features being referred as statistic of second level feature-statistic of basic feature-basic Feature: Mean—third quartile—colourfulness Standard Deviation—median—rule of thirds Mean—first quartile—focus of the region of interest Mean—maximum—luminance Mean—first quartile—blockiness quality Standard Deviation—median—focus if the region of interest. ### Claim 13 A system comprising means adapted to perform the method according to any preceding claim. ### Claim 14 (depends on 12) A computer program comprising computer program code means adapted to perform the method according to claim 12 when said program is run on a computer, a digital signal processor, a field-programmable gate array, an application-specific integrated circuit, a micro-processor, a micro-controller, or any other form of programmable hardware. ### Claim 15 (depends on 3) The method of claim 3, wherein said plurality of statistics of basic features (11) measurements used to create said second-level features comprises at least one of the following: mean, median, minimum, first quartile and third guar ### Claim 16 (depends on 15) The method of claim 15, wherein said plurality of statistics of second-level features measurements used to create video features comprises average and standard deviation. ### Claim 17 (depends on 16) The method of claim 16, wherein the frame rate together with the following video features are selected to characterize a video as high/low appealing, said following video features being referred as statistic of second level feature-statistic of basic feature-basic Feature: Mean—third quartile—colourfulness Standard Deviation—median—rule of thirds Mean—first quartile—focus of the region of interest Mean—maximum—luminance Mean—first quartile—blockiness quality Standard Deviation—median—focus if the region of interest. ### Claim 18 (depends on 9) The method of claim 9, wherein said plurality of statistics of basic features (11) measurements used to create said second-level features comprises at least one of the following: mean, median, minimum, maximum, first quartile and third quartile. ### Claim 19 (depends on 18) The method of claim 18, wherein said plurality of statistics of second-level features measurements used to create video features comprises average and standard deviation. ### Claim 20 (depends on 19) The method of claim 19, wherein the frame rate together with the following video features are selected to characterize a video as high/low appealing, said following video features being referred as statistic of second level feature-statistic of basic feature-basic Feature: Mean—third quartile—colourfulness Standard Deviation—median—rule of thirds Mean—first quartile—focus of the region of interest Mean—maximum—luminance Mean—first quartile—blockiness quality Standard Deviation—median—focus if the region of interest.