# US 2014/0153829 A1 · Superpixel-based refinement of low-resolution foreground segmentation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0153829 A1 - Document ID: US20140153829A1 - Status: Published application - Kind: A1 - Application number: 14/089,536 - Filed: 2013-11-25 - Published: 2014-06-05 - Earliest claimed: 2012-11-30 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: QUANG TUAN PHAM; VEENA MURTHY SRINIVASA DODBALLAPUR - Web page: https://patentgrep.com/patent/US20140153829A1 ## CPC - G06V 10/28: Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns - G06F 18/24: Classification techniques - G06T 7/11: Region-based segmentation - G06T 7/187: involving region growing; involving region merging; involving connected component labelling - G06T 7/194: involving foreground-background segmentation - G06T 2207/10024: Color image - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A computer-implemented method performs foreground segmentation of an input image. The method receives a first foreground segmentation at a first resolution of the input image and determines a plurality of labelled seed points based on the first foreground segmentation of the input image. The method associates each of the plurality of pixels in the input image with one of the determined labelled seed points to obtain a second foreground/background segmentation of the input image, and performs foreground separation on the input image at a second resolution by classifying each of the segments of the second segmentation as one of foreground and background based on the label of the associated seed point. ## Claims ### Claim 1 A computer-implemented method of performing foreground/background segmentation of an input image having a plurality of pixels, said method comprising: receiving a first foreground segmentation at a first resolution of the input image; determining a plurality of seed points based on a location of the foreground segmentation of the input image; associating each of the plurality of pixels in the input image with one of the determined seed points to obtain a second foreground segmentation of the input image; and performing foreground separation on the input image at a second resolution by classifying each of the segments of the second segmentation as at least one of foreground and background based on a label of the associated seed point. ### Claim 2 (depends on 1) A method according to claim 1, wherein the first resolution is coarser than the second resolution. ### Claim 3 (depends on 1) A method according to claim 1, wherein the second resolution is the resolution of the input image. ### Claim 4 (depends on 1) A method according to claim 1, further comprising obtaining a source image and resizing the source image to form the input image. ### Claim 5 (depends on 4) A method according to claim 4, wherein the second resolution of the second segmentation is higher than the resolution of the source image. ### Claim 6 (depends on 4) A method according to claim 4, wherein the second resolution of the second segmentation is lower than the resolution of the source image. ### Claim 7 (depends on 1) A method according to claim 1, wherein the associating is performed at the second resolution. ### Claim 8 (depends on 2) A method according to claim 2, further comprising creating one seed point for each pixel of the first foreground segmentation, the seed point bearing the same label as the pixel. ### Claim 9 (depends on 8) A method according to claim 8, wherein the labelled seed point is located at the centre location of the corresponding pixel of the first foreground/background segmentation. ### Claim 10 (depends on 8) A method according to claim 8 wherein seed points corresponding to foreground pixels at a boundary of the first foreground segmentation are removed. ### Claim 11 (depends on 8) A method according to claim 8, further comprising generating a pair comprising a foreground-labelled seed point and a background-labelled seed point for each foreground pixel at a boundary of the first foreground segmentation. ### Claim 12 (depends on 11) A method according to claim 11, wherein the position of the generated foreground-labelled seed point is displaced from the centre location towards neighbouring foreground-classified pixels. ### Claim 13 (depends on 12) A method according to claim 12, wherein the position of the generated background-labelled seed point mirrors the location of the generated foreground-labelled seed point about the centre location of the corresponding pixel at the boundary of the first foreground segmentation. ### Claim 14 (depends on 11) A method according to claim 11, wherein the offsets of the generated foreground-labelled and background-labelled seed points from the centre location of the corresponding pixel at the boundary of the first foreground segmentation are learnt from training data. ### Claim 15 (depends on 14) A method according to claim 14, wherein a different set of offsets of the generated foreground-labelled and background-labelled seed points from the centre location of the corresponding pixel are learnt for each local 3×3 pixel pattern of the foreground segmentation of the input image at a first resolution. ### Claim 16 (depends on 1) A method according to claim 1, wherein the superpixel segmentation of the image at a second resolution using determined seed points is performed using geodesic distance transform. ### Claim 17 (depends on 1) A method according to claim 1, wherein the superpixel segmentation of the image at a second resolution using determined seed points is performed using five-dimensional clustering in the Labxy space. ### Claim 18 A non-transitory computer readable storage medium having a program recorded thereon, the program being executable by a computerised apparatus to perform foreground/background segmentation of an input image, said program comprising: code for receiving a first foreground segmentation at a first resolution of the input image; code for determining a plurality of seed points based on a location of the foreground segmentation of the input image; code for associating each of the plurality of pixels in the input image with one of the determined seed points to obtain a second foreground segmentation of the input image; and code for performing foreground separation on the input image at a second resolution by classifying each of the segments of the second segmentation as at least one of foreground and background based on a label of the associated seed point. ### Claim 19 A computerised device to perform foreground segmentation of an input image, said device comprising: a memory for storing a computer program; a processor coupled to the memory for executing the computer program, the computer program comprising instructions for: receiving a first foreground/background segmentation at a first resolution of the input image and for storing the input image in the memory; determining a plurality of seed points based on based on a location of the foreground segmentation of the input image; associating each of the plurality of pixels in the input image with one of the determined labelled seed points to obtain a second foreground segmentation of the input image; performing foreground separation on the input image at a second resolution by classifying each of the segments of the second segmentation as at least one of foreground and background based on a label of the associated seed point; and displaying at least one of the segmentations on the display. ### Claim 20 (depends on 19) A camera comprising the computerised device of claim 19.