# US 2010/0166300 A1 · Method of generating motion vectors of images of a video sequence > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0166300 A1 - Document ID: US20100166300A1 - Status: Published application - Kind: A1 - Application number: 12/319,078 - Filed: 2008-12-31 - Published: 2010-07-01 - Earliest claimed: 2008-12-31 - Assignees: STMicroelectronics S.r.I. - Inventors: Giuseppe Spampinato; Arcangelo Ranieri Bruna; Alfio Castorina; Alessandro Capra - Web page: https://patentgrep.com/patent/US20100166300A1 ## CPC - H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)} - H04N 5/145: {Movement estimation (for video coding H04N19/51)} - H04N 23/10: for generating image signals from different wavelengths - H04N 23/68: for stable pick-up of the scene, e.g. compensating for camera body vibrations - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A digital image processing system and method for removing motion effects from images of a video sequence, and generating corresponding motion compensated images. ## Claims ### Claim 1 A method of generating signals representing absolute motion vectors of an image of a video sequence, the method comprising: defining horizontal and vertical weighing functions with values over pixels belonging to a first portion of the image being larger than values over pixels outside the first portion of the image; computing horizontal and vertical characteristic curves of the image and of a preceding image in the video sequence; determining a horizontal global displacement between the two images that minimizes a weighted average, calculated using the horizontal weighing function, of an absolute value of a difference between taps of the horizontal characteristic curve of the current image and of a translated replica of the horizontal characteristic curve of the preceding image in the sequence; determining a vertical global displacement between the two images that minimizes a weighted average, calculated using the vertical weighing function, of an absolute value of a difference between taps of the vertical characteristic curve of the current image and of a translated replica of the vertical characteristic curve of the preceding image in the sequence; aligning the image with the preceding image in the sequence and calculating relative motion vectors between the aligned images; and generating the signals representing absolute motion vectors between the images as a vector sum of the relative motion vectors and the horizontal and vertical global displacement. ### Claim 2 (depends on 1) The method of claim 1 wherein the images are YUV or RGB images and the characteristic curves are calculated considering only luminance values of pixels. ### Claim 3 (depends on 1) The method of claim 1 wherein the images are YUV or RGB images and the characteristic curves are calculated considering only green pixels. ### Claim 4 (depends on 1) The method of claim 1 wherein the images are Bayer images and the characteristic curves are calculated considering only green pixels. ### Claim 5 (depends on 1) The method of claim 1 wherein the horizontal and vertical weighing functions are null outside the first portion and, within the first portion, are respectively equal to a horizontal and vertical size of the current image. ### Claim 6 A method of filtering an image of an input video sequence, the method comprising: defining horizontal and vertical weighing functions with values over pixels belonging to a first portion of the image being larger than values over pixels outside the first portion of said image; computing horizontal and vertical characteristic curves of the image and of a preceding image in the sequence; determining a horizontal global displacement between the two images that minimizes a weighted average, calculated using the horizontal weighing function, of an absolute value of a difference between taps of the horizontal characteristic curve of the current image and of a translated replica of the horizontal characteristic curve of the preceding image in the sequence; determining a vertical global displacement between the two images that minimizes a weighted average, calculated using the vertical weighing function, of an absolute value of a difference between taps of the vertical characteristic curve of the current image and of a translated replica of the vertical characteristic curve of the preceding imagein the sequence; and generating a filtered output image by filtering out the horizontal and vertical global displacements from the current image. ### Claim 7 A method of filtering an image of an input video sequence, the method comprising: computing horizontal and vertical characteristic curves of the image and of a preceding image in the sequence, then generating corresponding high-pass replicas thereof using a high-pass filter; calculating a horizontal global displacement between the two images that minimizes an average of an absolute value of a difference between taps of the horizontal high-pass replica characteristic curve of the current image and of a translated replica of the high-pass replica horizontal characteristic curve of the preceding image in the sequence; calculating a vertical global displacement between the two images that minimizes an average of an absolute value of a difference between taps of the vertical high-pass replica characteristic curve of the current image and of a translated replica of the vertical high-pass replica characteristic curve of the preceding image in the sequence; and generating a filtered output image by filtering out the horizontal and vertical global displacements from the current image. ### Claim 8 (depends on 7) The method of claim 7, further comprising: high-pass filtering motion vectors with a second order Butterworth IIR high-pass filter, direct form II, with a cut-off frequency of about 0.40 Hz. ### Claim 9 (depends on 7) The method of claim 7, further comprising: high-pass filtering motion vectors with a second order Butterworth IIR high-pass filter, direct form II, with a cut-off frequency of about 0.40 Hz divided by half a frame rate of the video sequence. ### Claim 10 A method of filtering an image of an input video sequence, comprising: computing horizontal and vertical characteristic curves of the image and of a preceding image in the sequence, then generating corresponding band-pass replicas thereof using a Butterworth IIR band-pass filter; calculating a horizontal global displacement between the two images that minimizes an average of an absolute value of a difference between taps of the horizontal band-pass replica characteristic curve of the current image and of a translated replica of the horizontal band-pass replica characteristic curve of the proceding image in the sequence; calculating a vertical global displacement between the two images that minimizes an average of an absolute value of a difference between taps of the vertical band-pass replica characteristic curve of the current image and of a translated replica of the vertical band-pass replica characteristic curve of the preceding image in the sequence; and generating a filtered output image by filtering out the horizontal and vertical global displacements from the current image. ### Claim 11 (depends on 10) The method of claim 10 wherein the Butterworth IIR band-pass filter has cut-off frequencies of about 0.01 Hz and 0.20 Hz. ### Claim 12 A method of filtering an image of an input video sequence, comprising the steps of: computing horizontal and vertical characteristic curves of the image and of a preceding image in the sequence, then generating corresponding high-pass replicas thereof; calculating a horizontal global displacement between the two images that minimizes an average of an absolute value of a difference between taps of the horizontal high-pass replica characteristic curve of the current image and of a translated replica of the horizontal high-pass replica characteristic curve of the proceding image in the sequence; calculating a vertical global displacement between the two images that minimizes an average of an absolute value of a difference between taps of the vertical high-pass replica characteristic curve of the current image and of a translated replica of the vertical high-pass replica characteristic curve of the preceding image in the sequence; and generating a filtered output image by filtering out the horizontal and vertical global displacements from the current image. ### Claim 13 (depends on 12) The method of claim 12 wherein a FIR [−1 −8 −28 −55 −62 −28 28 62 55 28 8 1] is employed. ### Claim 14 (depends on 12) The method of claim 12 wherein the images are YUV or RGB images and the characteristic curves are calculated considering only luminance values of pixels. ### Claim 15 (depends on 12) The method of claim 12 wherein the images are YUV or RGB images and the characteristic curves are calculated considering only green pixels. ### Claim 16 (depends on 12) The method of claim 12 wherein the images are Bayer images and the characteristic curves are calculated considering only green pixels. ### Claim 17 A system to process digital images, comprising: a Bayer sensor configured to generate Bayer pattern images of captured scenes; a curve generator configured to generate characteristic curves of the Bayer pattern images; a memory storing the Bayer pattern images and the respective characteristic curves; and a digital signal processor configured to generate absolute global motion vectors based on the Bayer images and the respective characteristic curves. ### Claim 18 (depends on 17) The system of claim 17 wherein the digital signal processor is configured to: determine a horizontal global displacement between a current image and a preceding image that minimizes a weighted average, calculated using a horizontal weighing function, of an absolute value of a difference between taps of a horizontal characteristic curve of the current image and of a translated replica of a horizontal characteristic curve of the preceding image in the sequence; determine a vertical global displacement between the two images that minimizes a weighted average, calculated using a vertical weighing function, of an absolute value of a difference between taps of a vertical characteristic curve of the current image and of a translated replica of a vertical characteristic curve of the preceding image in the sequence; align the current image with the preceding image in the sequence and calculate relative motion vectors between the aligned images; and generate signals representing absolute motion vectors between the images as a vector sum of the relative motion vectors and the horizontal and vertical global displacement. ### Claim 19 (depends on 17) The system of claim 17 wherein the digital signal processor is configured to: determine a horizontal global displacement between a current image and a preceding image that minimizes a weighted average, calculated using a horizontal weighing function, of an absolute value of a difference between taps of a horizontal characteristic curve of the current image and of a translated replica of a horizontal characteristic curve of the preceding image in the sequence; determine a vertical global displacement between the two images that minimizes a weighted average, calculated using a vertical weighing function, of an absolute value of a difference between taps of a horizontal characteristic curve of the current image and of a translated replica of a characteristic curve of the preceding image in the sequence; align the current image with the preceding image in the sequence and calculate relative motion vectors between the aligned images; and generate signals representing absolute motion vectors between the images as a vector sum of the relative motion vectors and the horizontal and vertical global displacement. ### Claim 20 A computer readable memory medium comprising contents that cause a computing device to implement a method of processing a digital image, the method including: defining horizontal and vertical weighing functions with values over pixels belonging to a first portion of the image being larger than values over pixels outside the first portion of the image; computing horizontal and vertical characteristic curves of the image and of a preceding image in the video sequence; determining a horizontal global displacement between the two images that minimizes a weighted average, calculated using the horizontal weighing function, of an absolute value of a difference between taps of the horizontal characteristic curve of the current image and of a translated replica of the horizontal characteristic curve of the preceding image in the sequence; determining a vertical global displacement between the two images that minimizes a weighted average, calculated using the vertical weighing function, of an absolute value of a difference between taps of the vertical characteristic curve of the current image and of a translated replica of the vertical characteristic curve of the preceding image in the sequence; aligning the image with the preceding image in the sequence and calculating relative motion vectors between the aligned images; and generating the signals representing absolute motion vectors between the images as a vector sum of the relative motion vectors and the horizontal and vertical global displacement. ### Claim 21 A method of generating signals representing absolute motion vectors of an image of a video sequence, the method comprising: defining horizontal and vertical weighing functions with values over pixels belonging to a first portion of the image being larger than values over pixels outside the first portion of the image; computing horizontal and vertical characteristic curves of the image and of a preceding image in the video sequence; determining a horizontal global displacement between the two images that minimizes a weighted average, calculated using the horizontal weighing function, of an absolute value of a difference between taps of the horizontal characteristic curve of the current image and of a translated replica of the characteristic curve of the preceding image in the sequence; determining a vertical global displacement between the two images that minimizes a weighted average, calculated using the vertical weighing function, of an absolute value of a difference between taps of the horizontal characteristic curve of the current image and of a translated replica of the characteristic curve of the preceding image in the sequence; aligning the image with the preceding image in the sequence and calculating relative motion vectors between the aligned images; and generating the signals representing absolute motion vectors between the images as a vector sum of the relative motion vectors and the horizontal and vertical global displacement. ### Claim 22 (depends on 21) The method of claim 21 wherein the images are YUV or RGB images and the characteristic curves are calculated considering only luminance values of pixels. ### Claim 23 (depends on 21) The method of claim 21 wherein the images are Bayer images and the characteristic curves are calculated considering only green pixels. ### Claim 24 (depends on 21) The method of claim 21 wherein the horizontal and vertical weighing functions are null outside the first portion and, within the first portion, are respectively equal to a horizontal and vertical size of the current image.