# US 2010/0166262 A1 · Multi-modal object signature > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0166262 A1 - Document ID: US20100166262A1 - Status: Published application - Kind: A1 - Application number: 12/645,718 - Filed: 2009-12-23 - Published: 2010-07-01 - Earliest claimed: 2008-12-30 - Assignees: Canon Kabushiki Kaisha - Inventors: Daniel John Wedge - Web page: https://patentgrep.com/patent/US20100166262A1 ## CPC - G06T 7/248: {involving reference images or patches} - G06V 10/507: {Summing image-intensity values; Histogram projection analysis} - G06V 30/194: References adjustable by an adaptive method, e.g. learning - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Disclosed herein are a method and system for appearance-invariant tracking of an object in an image sequence. A track is associated with the image sequence, wherein the track has an associated track signature comprising at least one mode. The method detects the object in a frame of the image sequence (1020). A representative signature is associated with the detected object. The method determines a spatial difference measure between the track and the detected object, and determines, for each mode of the track signature, a visual difference (1410) between the mode of the track signature and the representative signature to obtain a lowest determined visual difference (1420). The method then utilises the spatial difference measure and the lowest determined visual difference to perform at least one of the following steps of: (i) associating the detected object with the track (1440), and (ii) adding a new mode to the track signature (1460), based on the representative signature. ## Claims ### Claim 1 A method for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising a plurality of modes, said method comprising the steps of: detecting said object in a frame of said image sequence; associating said detected object with a representative signature; determining a spatial difference between said track and said detected object; determining, for each mode of said track signature, a visual difference between said mode of said track signature and said representative signature to obtain a lowest determined visual difference; and utilizing said spatial difference and said lowest determined visual difference to perform at least one of the following steps of: (i) associating said detected object with said track, and (ii) adding a new mode, based on said representative signature, to said track signature. ### Claim 2 (depends on 1) The method according to claim 1, wherein the detected object comprises at least one detection. ### Claim 3 (depends on 1) The method according to claim 1, wherein said performing of at least one of the steps of associating said detected object with said track and adding a new mode, based on said representative signature, to said track signature, is performed based on one of: a sum of said spatial difference and said lowest determined visual difference being less than a combined difference threshold, and said spatial difference being less than a spatial difference threshold. ### Claim 4 (depends on 3) The method according to claim 3, comprising the further step of: deleting an existing mode of said track signature before said step of adding said new mode to said track signature. ### Claim 5 (depends on 4) The method according to claim 4, wherein said existing mode of said track signature is deleted based on temporal information selected from the group consisting of: a frame number of the frame in which said existing mode was added to said track signature; a number of frames in which said existing mode was used in obtaining said lowest computed difference; a frame number of the frame in which said existing mode was most recently used in obtaining said lowest computed difference; and a frame number of the current frame. ### Claim 6 (depends on 1) The method according to claim 1, wherein said track signature and said representative signature are derived from at least one histogram. ### Claim 7 (depends on 6) The method according to claim 6, wherein said visual difference is the Euclidean distance between vectors derived from the track signature and the representative signature. ### Claim 8 (depends on 6) The method according to claim 6, wherein said visual difference is the Bhattacharrya distance between the track signature and the representative signature. ### Claim 9 (depends on 1) The method according to claim 1, wherein said track signature is derived from at least one object detected in at least one frame acquired by a first image capturing device, and said frame in which said object is detected is acquired by a second image capturing device. ### Claim 10 A method for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising a plurality of modes, said method comprising the steps of: detecting said object in a frame of said image sequence; associating said detected object with a representative signature; determining a spatial difference between said track and said detected object; determining, for each mode of said track signature, a visual difference between said mode of said track signature and said representative signature to obtain a lowest determined visual difference; selecting a best mode of said track signature, said best mode being associated with producing said lowest determined visual difference; associating said detected object with said track and updating said best mode using said representative signature of said detected object, if a sum of said spatial difference and said lowest determined visual difference is less than a combined difference threshold; and associating said detected object with said track and adding a new mode to said track signature based on said representative signature, if: (i) a sum of said spatial difference and said lowest determined visual difference is greater than or equal to a combined difference threshold, and (ii) said spatial difference is less than a spatial difference threshold. ### Claim 11 A method for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising multiple regions, each of said regions comprising multiple modes, said method comprising the steps of: detecting said object in a frame of said image sequence; associating said detected object with a representative signature comprising multiple regions, wherein each of said multiple regions of said track signature corresponds to one of said multiple regions of said representative signature of said detected object; determining a spatial difference between said track and said detected object; determining, for each mode of each of said multiple regions of said track signature, a visual difference between said mode of the track signature and a corresponding region of said representative signature to obtain a lowest determined visual difference for each of said multiple regions; and if at least one of: (i) said spatial difference is less than a spatial difference threshold, and (ii) for each of said multiple regions, a sum of said spatial difference and said lowest visual difference of said each of said multiple regions is less than a combined difference threshold, performing the steps of: associating said detected object with said track; for each of said multiple regions, processing each region of said multiple regions in turn, where said processing comprises: selecting a best mode of said region of said track signature, the best mode being associated with producing said lowest determined visual difference for the region; updating a best mode of said region using said representative signature of said detected object, if a sum of said spatial difference and said lowest visual difference of said region is less than a combined difference threshold; and adding a new mode to said region of said track signature based on said representative signature, if a sum of said spatial difference and said lowest visual difference of said region is greater than or equal to a combined difference threshold. ### Claim 12 A method for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising multiple regions, each of said regions comprising multiple modes, the method comprising the steps of: detecting said object in a frame of said image sequence, a representative signature being associated with said detected object and comprising multiple regions, wherein each of said multiple regions of said track signature corresponds to one of said multiple regions of said representative signature of said detected object; determining a spatial difference between said track and said detected object, determining, for each mode of each of said multiple regions of said track signature, a visual difference between said mode of the track signature and a corresponding region of said representative signature to obtain a lowest determined visual difference for each of said multiple regions; and utilizing said spatial difference and said lowest determined visual difference for each of the multiple regions to perform at least one of: (i) associating said detected object with said track; and (ii) adding a new mode, based on said representative signature, to selected regions of said track signature. ### Claim 13 (depends on 12) The method according to claim 12, wherein associating said detected object with said track is based on at least one of: (i) a sum of said spatial difference and said lowest determined visual difference for each of the multiple regions being less than a combined difference threshold; and (ii) said spatial difference being less than a spatial difference threshold; and further wherein, for each region of said multiple regions, said region is added to said selected regions of said track signature if a sum of said spatial difference measure and said lowest determined visual difference for the region is less than a scaled combined difference threshold. ### Claim 14 A system for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising a plurality of modes, said system comprising: a storage device for storing a computer program; and a processor for executing the program, said program comprising: code for detecting said object in a frame of said image sequence, a representative signature being associated with said detected object, and code for determining a spatial difference between said track and said detected object; code for determining, for each mode of said track signature, a visual difference between said mode of said track signature and said representative signature to obtain a lowest determined visual difference; and code for utilizing said spatial difference and said lowest determined visual difference to perform at least one of: (i) associating said detected object with said track, and (ii) adding a new mode, based on said representative signature, to said track signature. ### Claim 15 A computer readable storage medium having recorded thereon a computer program for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising a plurality of modes, said computer program comprising code for performing the steps of: detecting said object in a frame of said image sequence; associating said detected object with a representative signature; determining a spatial difference between said track and said detected object; determining, for each mode of said track signature, a visual difference between said mode of said track signature and said representative signature to obtain a lowest determined visual difference; and utilizing said spatial difference and said lowest determined visual difference to perform at least one of the following steps of: (i) associating said detected object with said track, and (ii) adding a new mode, based on said representative signature, to said track signature. ### Claim 16 A computer readable storage medium having recorded thereon a computer program for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising a plurality of modes, said computer program comprising code for performing the steps of: detecting said object in a frame of said image sequence; associating said detected object with a representative signature; determining a spatial difference between said track and said detected object; determining, for each mode of said track signature, a visual difference between said mode of said track signature and said representative signature to obtain a lowest determined visual difference; selecting a best mode of said track signature, said best mode being associated with producing said lowest determined visual difference; associating said detected object with said track and updating said best mode using said representative signature of said detected object, if a sum of said spatial difference and said lowest determined visual difference is less than a combined difference threshold; and associating said detected object with said track and adding a new mode to said track signature based on said representative signature, if: (i) a sum of said spatial difference and said lowest determined visual difference is greater than or equal to a combined difference threshold, and (ii) said spatial difference is less than a spatial difference threshold. ### Claim 17 A computer readable storage medium having recorded thereon a computer program for tracking an object in an image sequence, wherein a track associated with said image sequence is associated with a track signature, said track signature comprising multiple regions, each of said regions comprising multiple modes, said computer program comprising code for performing the steps of: detecting said object in a frame of said image sequence; associating said detected object with a representative signature comprising multiple regions, wherein each of said multiple regions of said track signature corresponds to one of said multiple regions of said representative signature of said detected object; determining a spatial difference between said track and said detected object; determining, for each mode of each of said multiple regions of said track signature, a visual difference between said mode of the track signature and a corresponding region of said representative signature to obtain a lowest determined visual difference for each of said multiple regions; and if at least one of: (i) said spatial difference is less than a spatial difference threshold, and (ii) for each of said multiple regions, a sum of said spatial difference and said lowest determined visual difference of said each of said multiple regions is less than a combined difference threshold, performing the steps of: associating said detected object with said track; for each of said multiple regions, processing each region of said multiple regions in turn, where said processing comprises: selecting a best mode of said region of said track signature, which is associated with producing said lowest determined visual difference for the region; updating a best mode of said region using said representative signature of said detected object, if a sum of said spatial difference and said lowest determined visual difference of said region is less than a combined difference threshold; and adding a new mode to said region of said track signature based on said representative signature, if a sum of said spatial difference and said lowest determined visual difference of said region is greater than or equal to a combined difference threshold.