# US 8,542,950 B2 · Finding iconic images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,950 B2 - Document ID: US8542950B2 - Status: Granted - Kind: B2 - Application number: 12/476,711 - Filed: 2009-06-02 - Published: 2010-12-02 - Granted: 2013-09-24 - Earliest claimed: 2009-06-02 - Assignees: Yahoo! Inc. - Applicants: Tamara L. Berg; Alexander Berg - Inventors: Tamara L. Berg; Alexander Berg - Web page: https://patentgrep.com/patent/US8542950B2 ## CPC - G06F 16/583: using metadata automatically derived from the content - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Iconic images for a given object or object category may be identified in a set of candidate images by using a learned probabilistic composition model to divide each candidate image into a most probable rectangular object region and a background region, ranking the candidate images according to the maximal composition score of each image, removing non-discriminative images from the candidate images, clustering highest-ranked candidate images to form clusters, wherein each cluster includes images having similar object regions according to a feature match score, selecting a representative image from each cluster as an iconic image of the object category, and causing display of the iconic image. The composition model may be a Naïve Bayes model that computes composition scores based on appearance cues such as hue, saturation, focus, and texture. Iconic images depict an object or category as a relatively large object centered on a clean or uncluttered contrasting background. ## Claims ### Claim 1 A method comprising: receiving, at a computing device, a plurality of candidate images; using, via the computing device, a learned probabilistic composition model to divide each candidate image in the plurality of candidate images into a most probable rectangular object region and a background region, wherein the most probable rectangular object region has a maximal composition score from possible composition scores computed according to the composition model for possible divisions of the candidate image into object and background regions, each possible composition score is based upon at least one image feature cue computed over the object and background regions, and the composition model is trained on a set of images independent of the plurality of candidate images; ranking, via the computing device, the plurality of candidate images according to the maximal composition score of the most probable rectangular object region of each image determined using the learned probabilistic composition model; removing, via the computing device, non-discriminative images from the plurality of candidate images; clustering, via the computing device, a plurality of highest-ranked images from the plurality of candidate images ranked according to the maximal composition score of the most probable rectangular object region of each image determined using the learned probabilistic composition model to form a plurality of clusters, wherein each cluster includes a plurality of images selected from the plurality of highest-ranked images and having similar object regions according to a feature match score; selecting, via the computing device, a representative image from each cluster as an iconic image representative of an object category; and causing, via the computing device, display of the iconic image. ### Claim 2 (depends on 1) The method of claim 1, wherein removing non-discriminative images from the plurality of candidate images comprises: comparing each candidate image in the plurality of candidate images to a plurality of highest-ranked images from the plurality of candidate images using a similarity measure based upon the feature match score, wherein the feature match score is spatially restricted to the object regions; and removing each candidate image for which more than a predetermined percentage of images most similar to the candidate image are in another category different from the object category. ### Claim 3 (depends on 1) The method of claim 1, wherein the composition model is a Naive Bayes model. ### Claim 4 (depends on 1) The method of claim 1, wherein the at least one image feature cue comprises hue, saturation, value, focus, texture, or a combination thereof. ### Claim 5 (depends on 4) The method of claim 4, wherein the hue, saturation, value, or a combination thereof is modeled as at least one histogram with a predetermined number of uniformly-spaced bins. ### Claim 6 (depends on 4) The method of claim 4, wherein the focus is computed as a ratio of high pass energy to low pass energy. ### Claim 7 (depends on 4) The method of claim 4, wherein the texture is modeled as a histogram of total response to a predetermined number of bar and spot features. ### Claim 8 (depends on 1) The method of claim 1, wherein the composition model is trained using a plurality of hand-labeled training images, wherein a correct layout and an incorrect layout are labeled for each training image. ### Claim 9 (depends on 1) The method of claim 1, wherein probability models for both good and bad layouts are trained for each image cue. ### Claim 10 (depends on 9) The method of claim 9, wherein the probability models represent, for both good and bad layouts, the distribution over contrast values computed using Chi-Squared distance between object and background histograms, the distribution over object histograms, the distribution over background histograms, or a combination thereof. ### Claim 11 (depends on 1) The method of claim 1, wherein the features for possible divisions are computed using summed area tables. ### Claim 12 (depends on 1) The method of claim 1, wherein clustering comprises using a k-medoids clustering method based upon geometric blur features computed within the object regions to find similar-looking object regions from the plurality of highest-ranked images, and selecting the representative image from each cluster comprises selecting the medoid of the cluster. ### Claim 13 (depends on 1) The method of claim 1, wherein the plurality of highest-ranked images comprises a predetermined number of the highest-ranked images. ### Claim 14 (depends on 1) The method of claim 1, wherein the plurality of highest-ranked images comprises images having a maximal composition score greater than a predetermined value. ### Claim 15 A computer system comprising: a memory for storing computer-executable instructions; and a processor for executing the instructions, the instructions for: receiving a plurality of candidate images; using a learned probabilistic composition model to divide each candidate image in the plurality of candidate images into a most probable rectangular object region and a background region, wherein the most probable rectangular object region has a maximal composition score from possible composition scores computed according to the composition model for possible divisions of the candidate image into object and background regions, each possible composition score is based upon at least one image feature cue computed over the object and background regions, and the composition model is trained on a set of images independent of the plurality of candidate images; ranking the plurality of candidate images according to the maximal composition score of the most probable rectangular object region of each image determined using the learned probabilistic composition model; removing non-discriminative images from the plurality of candidate images; clustering a plurality of highest-ranked images from the plurality of candidate images ranked according to the maximal composition score of the most probable rectangular object region of each image determined using the learned probabilistic composition model to form a plurality of clusters, wherein each cluster includes a plurality of images selected from the plurality of highest-ranked images and having similar object regions according to a feature match score; selecting a representative image from each cluster as an iconic image representative of an object category; and causing display of the iconic image. ### Claim 16 (depends on 15) The system of claim 15, wherein the instructions for removing non-discriminative images from the plurality of candidate images comprise: comparing each candidate image in the plurality of candidate images to a plurality of highest-ranked images from the plurality of candidate images using a similarity measure based upon the feature match score, wherein the feature match score is spatially restricted to the object regions; and removing each candidate image for which more than a predetermined percentage of images most similar to the candidate image are in another category different from the object category. ### Claim 17 (depends on 15) The system of claim 15, wherein the composition model is a Naive Bayes model. ### Claim 18 (depends on 15) The system of claim 15, wherein the at least one image feature cue comprises hue, saturation, value, focus, texture, or a combination thereof. ### Claim 19 (depends on 18) The system of claim 18, wherein the hue, saturation, value, or a combination thereof is modeled as at least one histogram with a predetermined number of uniformly-spaced bins. ### Claim 20 (depends on 18) The system of claim 18, wherein the focus is computed as a ratio of high pass energy to low pass energy. ### Claim 21 (depends on 18) The system of claim 18, wherein the texture is modeled as a histogram of total response to a predetermined number of bar and spot features. ### Claim 22 (depends on 15) The system of claim 15, wherein the composition model is trained using a plurality of hand-labeled training images, wherein a correct layout and an incorrect layout are labeled for each training image. ### Claim 23 (depends on 15) The system of claim 15, wherein probability models for both good and bad layouts are trained for each image cue. ### Claim 24 (depends on 23) The system of claim 23, wherein the probability models represent, for both good and bad layouts, the distribution over contrast values computed using Chi-Squared distance between object and background histograms, the distribution over object histograms, the distribution over background histograms, or a combination thereof. ### Claim 25 (depends on 15) The system of claim 15, wherein the features for possible divisions are computed using summed area tables. ### Claim 26 (depends on 15) The system of claim 15, wherein the instructions for clustering comprises using a k-medoids clustering method based upon geometric blur features computed within the object regions to find similar-looking object regions from the plurality of highest-ranked images, and selecting the representative image from each cluster comprises selecting the medoid of the cluster. ### Claim 27 (depends on 15) The system of claim 15, wherein the plurality of highest-ranked images comprises a predetermined number of the highest-ranked images. ### Claim 28 (depends on 15) The system of claim 15, wherein the plurality of highest-ranked images comprises images having a maximal composition score greater than a predetermined value. ### Claim 29 A computer readable storage medium comprising computer-executable instructions tangibly stored thereon, which when executed by a processor, perform a method comprising: receiving a plurality of candidate images; using a learned probabilistic composition model to divide each candidate image in the plurality of candidate images into a most probable rectangular object region and a background region, wherein the most probable rectangular object region has a maximal composition score from possible composition scores computed according to the composition model for possible divisions of the candidate image into object and background regions, each possible composition score is based upon at least one image feature cue computed over the object and background regions, and the composition model is trained on a set of images independent of the plurality of candidate images; ranking the plurality of candidate images according to the maximal composition score of the most probable rectangular object region of each image determined using the learned probabilistic composition model; removing non-discriminative images from the plurality of candidate images; clustering a plurality of highest-ranked images from the plurality of candidate images ranked according to the maximal composition score of the most probable rectangular object region of each image determined using the learned probabilistic composition model to form a plurality of clusters, wherein each cluster includes a plurality of images selected from the plurality of highest-ranked images and having similar object regions according to a feature match score; selecting a representative image from each cluster as an iconic image representative of the object category; and causing display of the iconic image. ### Claim 30 (depends on 29) The computer readable storage medium of claim 29, wherein removing non-discriminative images from the plurality of candidate images comprises: comparing each candidate image in the plurality of candidate images to a plurality of highest-ranked images from the plurality of candidate images using a similarity measure based upon the feature match score, wherein the feature match score is spatially restricted to the object regions; and removing each candidate image for which more than a predetermined percentage of images most similar to the candidate image are in another category different from the object category. ### Claim 31 (depends on 29) The computer readable storage medium of claim 29, wherein the composition model is a Naive Bayes model. ### Claim 32 (depends on 29) The computer readable storage medium of claim 29, wherein the at least one image feature cue comprises hue, saturation, value, focus, texture, or a combination thereof. ### Claim 33 (depends on 32) The computer readable storage medium of claim 32, wherein the hue, saturation, value, or a combination thereof is modeled as at least one histogram with a predetermined number of uniformly-spaced bins. ### Claim 34 (depends on 32) The computer readable storage medium of claim 32, wherein the focus is computed as a ratio of high pass energy to low pass energy. ### Claim 35 (depends on 32) The computer readable storage medium of claim 32, wherein the texture is modeled as a histogram of total response to a predetermined number of bar and spot features. ### Claim 36 (depends on 29) The computer readable storage medium of claim 29, wherein the composition model is trained using a plurality of hand-labeled training images, wherein a correct layout and an incorrect layout are labeled for each training image. ### Claim 37 (depends on 29) The computer readable storage medium of claim 29, wherein probability models for both good and bad layouts are trained for each image cue. ### Claim 38 (depends on 37) The computer readable storage medium of claim 37, wherein the probability models represent, for both good and bad layouts, the distribution over contrast values computed using Chi-Squared distance between object and background histograms, the distribution over object histograms, the distribution over background histograms, or a combination thereof. ### Claim 39 (depends on 29) The computer readable storage medium of claim 29, wherein the features for possible divisions are computed using summed area tables. ### Claim 40 (depends on 29) The computer readable storage medium of claim 29, wherein clustering comprises using a k-medoids clustering computer readable medium based upon geometric blur features computed within the object regions to find similar-looking object regions from the plurality of highest-ranked images, and selecting the representative image from each cluster comprises selecting the medoid of the cluster. ### Claim 41 (depends on 29) The computer readable storage medium of claim 29, wherein the plurality of highest-ranked images comprises a predetermined number of the highest-ranked images. ### Claim 42 (depends on 29) The computer readable storage medium of claim 29, wherein the plurality of highest-ranked images comprises images having a maximal composition score greater than a predetermined value.