# US 7,609,293 B2 · Image stabilisation system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,609,293 B2 - Document ID: US7609293B2 - Status: Granted - Kind: B2 - Application number: 10/536,952 - Filed: 2003-12-04 - Published: 2006-03-23 - Granted: 2009-10-27 - Earliest claimed: 2002-12-13 - Assignees: Qinetiq Limited - Inventors: David Andrew Alexander Faulkner; Gary Shaun Price; Philip John Kent; Paul Fretwell - Web page: https://patentgrep.com/patent/US7609293B2 ## CPC - H04N 23/6815: {by distinguishing pan or tilt from motion} - H04N 23/68: for stable pick-up of the scene, e.g. compensating for camera body vibrations - H04N 23/6811: {based on the image signal} - H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory} - H04N 5/145: {Movement estimation (for video coding H04N19/51)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A system for stabilizing video signals generated by a video camera which may be mounted in an unstable manner includes a digital processing means for manipulation of each incoming image that attempts to overlay features of the current image onto similar features of a previous image. A mask is used that prevents parts of the image that are likely to cause errors in the overlaying process from being used in the calculation of the required movement to be applied to the image. The mask may include areas where small movements of the image have been detected, and many also include areas where image anomalies including excess noise have been detected. The invention also discloses means for dealing with wanted movements of the camera, such as pan or zoom, and also discloses means for dealing with the edges of video signals as processed by the system. A method is also disclosed. ## Claims ### Claim 1 A video image stabilisation system for correction of camera motion, that is arranged to receive one or more signals representative of a plurality of images from an image source wherein, for an image n following at least an image (n−1) and an image (n−2) the system is arranged to estimate a Global Motion Offset (GMO) value between image n and a previous image representative of the spatial separation between the scene imaged in image n and the previous image, and apply a corrective movement to the image n based upon this GMO, characterised in that: the system is arranged to estimate the GMO for the image n with reference to a mask that represents a region or regions of the image n not to be considered in the GMO estimation, the region(s) being region(s) estimated as likely to mislead the estimation of the GMO and wherein the system is arranged to examine one or more local regions of the image n and corresponding local regions of a previous image, and estimate a local motion offset (LMO) representative of spatial separation between like features in corresponding local regions of the current and previous images, and if the, or each, LMO is greater than a given threshold, to set area(s) of the mask that correspond to this local region or regions to indicate omission from the GMO estimation, wherein the LMO values of those parts of the image not masked are not used in the calculation of the GMO. ### Claim 2 (depends on 1) A stabilisation system as claimed in claim 1 wherein the local regions comprise an array of rectangular regions. ### Claim 3 (depends on 1) A stabilisation system as claimed in claim 1 wherein the system is arranged to estimate the GMO of an image representative of image n but having a spatial resolution lower than image n. ### Claim 4 (depends on 3) A stabilisation system as claimed in claim 3 wherein the system is arranged to iterate the estimation of the GMO on a plurality of images each representative of image n, where each of the plurality of images has a different spatial resolution. ### Claim 5 (depends on 1) A stabilisation system as claimed in claim 1 wherein the system is arranged to adjust the GMO if a stationary camera state is detected, this state being indicated by means of a plurality of contiguous GMOs including the current GMO all being below a given threshold. ### Claim 6 (depends on 1) A stabilisation system as claimed in claim 1 wherein the system is arranged to adjust the GMO if intentional adjustment of the image source viewing direction (pan) or field of view (zoom) is detected. ### Claim 7 (depends on 6) A stabilisation system as claimed in claim 6 wherein the system is arranged to detect a pan of the image source by means of low-pass filtering GMO values from at least a sequence of previous images at a cut-off frequency lower than that expected from unintentional camera movements. ### Claim 8 (depends on 6) A stabilisation system as claimed in claim 6 wherein a zoom is detected if a number x of LMOs examined for image n all show a direction of movement in towards a central region of the image n, or all show a direction of movement away from a central region of the image n, the number x being greater than some given threshold. ### Claim 9 (depends on 8) A stabilisation system as claimed in claim 8 wherein the threshold is 50% of those LMOs examined, and the number x is proportional to the absolute difference between the number of those LMOs examined showing a direction of movement in towards a central region of the image n, and those LMOs examined showing a direction of movement away from a central region of the image n. ### Claim 10 (depends on 8) A stabilisation system as claimed in claim 8 wherein the LMOs examined are taken from those local regions that are substantially adjacent the edge of image n. ### Claim 11 (depends on 1) A stabilisation system as claimed in claim 1 wherein the system is arranged to generate a border on at least one edge of the image n, the border being adjustable in size such that it covers any blank space between the edge of image n and the corresponding edge of a display area on which the image n is displayed. ### Claim 12 (depends on 11) A stabilisation system as claimed in claim 11 wherein the system is arranged to adjust the border size on at least one edge of the image n such that it also covers an area on image n corresponding to blank space present on one or more previous images. ### Claim 13 (depends on 11) A stabilisation system as claimed in claim 11 wherein the border generated by the system comprises of image data from one or more previous images. ### Claim 14 (depends on 1) A stabilisation system as claimed in claim 1 wherein the system is arranged to scale the image n, such that it covers any blank space between the edge of image n and the corresponding edge of a display area on which the image n is displayed. ### Claim 15 (depends on 1) A stabilisation system as claimed in claim 1 wherein anomalous pixels of the image n are used to set corresponding pixels of the mask such that they are excluded from the estimation of the GMO. ### Claim 16 (depends on 15) A stabilisation system as claimed in claim 15 wherein the pixels above a threshold in an image comprising the absolute difference between the image n and a previous image m, both images n and m having had corrective movements applied, are regarded as anomalous. ### Claim 17 (depends on 1) A stabilisation system as claimed in claim 1 wherein the system is arranged to multiply the calculated GMO, as adjusted in any other operation, by a decay constant factor lying between 0 and 1 before shifting the image n. ### Claim 18 A method of stabilising a present image relative to at least one previous image where both current and previous images are part of a sequence of video images represented by an electronic signal, comprising the steps of: i. estimating a global motion offset (GMO) between the current and previous image representative of the spatial separation between the scene imaged in the current image and that imaged in the previous image; and ii. applying a corrective movement to the current image based upon the GMO; characterised in that: a mask image is used in estimating the GMO, the mask image representing a region or regions of the current image not to be considered in the GMO estimation, the region(s) being region(s) being estimated as likely to mislead the estimation of the GMO, and wherein the method further includes the step of examining one or more local regions of the current image and corresponding local regions of a previous image, and estimating a local motion offset (LMO representing the spatial separation between the features in corresponding local regions of the current and previous images, and if the, or each, LMO is greater than a given threshold, setting area(s) of the mask that correspond to this local region or regions to indicate omission from the GMO estimation, wherein the LMO values of those parts of the image not masked are not used in the calculation of the GMO. ### Claim 19 A computer program designed to run on a computer, contained on a computer readable medium and arranged to implement a video image stabilisation system, the system being arranged to receive as an input a digital signal representative of a plurality of images from an image source wherein, for an image n following at least an image (n −1) and an image (n −2) the system is arranged to estimate a Global Motion Offset (GMO) value between image n and a previous image representative of the spatial separation between the scene imaged in image n and the previous image, and apply a corrective movement to the image n based upon this GMO, characterised in that: the system is arranged to estimate the GMO for the image n with reference to a mask that represents a region or regions of the image n not to be considered in the GMO estimation, the region(s) being region(s) estimated as likely to mislead the estimation of the GMO, and wherein the system is arranged to examine one or more local regions of the image n and corresponding local regions of a previous image, and estimate a local motion offset (LMO) representative of spatial separation between like features in corresponding local regions of the current and previous images, and if the, or each, LMO is greater than a given threshold, to set area(s) of the mask that correspond to this local region or regions to indicate omission from the GMO estimation, wherein the LMO values of those parts of the image not masked are not used in the calculation of the GMO. ### Claim 20 A video image stabilization system for correction of camera motion, that is arranged to receive one or more signals representative of a plurality of images from an image source wherein, for an image n following at least an image (n−1) and an image (n−2) the system is arranged to estimate a Global Motion Offset (GMO) value between image n and a previous image representative of the spatial separation between the scene imaged in image n and the previous image, and apply a corrective movement to the image n based upon this GMO, characterised in that: the system is arranged to estimate the GMO for the image n with reference to a mask that represents a region or regions of the image n not to be considered in the GMO estimation, the region(s) being region(s) estimated likely to mislead the estimation of the GMO and further wherein the system is arranged to generate a border on at least one edge of the image n, the border being adjustable in size such that it covers any blank space between the edge of image n and the corresponding edge of a display area on which the image n is displayed, the border also being arranged to cover an area on image n corresponding to blank space present on one or more previous images.