# US 2002/0154695 A1 · Correcting motion vector maps for image processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2002/0154695 A1
- Document ID: US20020154695A1
- Status: Published application
- Kind: A1
- Application number: 09/838,868
- Filed: 2001-04-20
- Published: 2002-10-24
- Earliest claimed: 2001-04-20
- Assignees: Avid Technology, Inc.
- Inventors: Katherine H. Cornog; Randy M. Fayan; Garth Dickie
- Web page: https://patentgrep.com/patent/US20020154695A1

## CPC

- H04N 5/145: {Movement estimation (for video coding H04N19/51)}
- G06T 7/11: Region-based segmentation
- G06T 7/194: involving foreground-background segmentation
- H04N 7/014: {involving the use of motion vectors (motion estimation and compensation in video coding H04N19/51)}
- G06T 2207/20012: Locally adaptive
- G06T 2207/20092: Interactive image processing based on input by user
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Visible artifacts in images created using image processing based on motion vector maps may be reduced by providing one or more mechanisms for correcting the vector map. In general, the set of motion vectors is changed by selecting one or more portions of the image. The vectors corresponding to the selected one or more portions are modified. Various image processing operations, such as motion compensated interpolation, may be performed using the changed set of motion vectors. Various mechanisms for obtaining a changed set of motion vectors may be used separately or combination by a user. A region in an image may be defined. The region may be segmented into foreground and background regions. A tracker then may be used to track either the foreground region or the background region or both. A single motion vector or a parameterized motion model obtained from the tracker may be assigned to the tracked region. A combination map may be defined to control which pixels of the input images are used to contribute to each pixel of an output image based on how a motion vector transforms a pixel from the input image to the output image. The combination map is used with a specified region to which one or more motion vectors are assigned. Such combination maps may be used in combination with the tracker described above. A color image may be generated using the motion vectors. The color image then may be modified using conventional color modification tools. The modified color image then may be converted back to motion vectors. A user-specified transform between the two images may be used to define a set of vectors that correspond in time and resolution to the motion vectors estimated between the two images. The set of vectors may be combined with the estimated motion vectors to produce a set of vectors used for image processing operations.

## Claims

### Claim 1

A method for generating motion vectors, comprising:
determining a set of motion vectors that describes motion between the first and second images; and
changing the set of motion vectors by selecting one or more portions of the image and modifying the vectors corresponding to the selected one or more portions.

### Claim 2 (depends on 1)

The method of claim 1, further comprising performing image processing using the changed set of motion vectors.

### Claim 3 (depends on 1)

The method of claim 1, wherein changing the set of motion vectors comprises:
identifying a foreground region and a background region in the first and second images;
performing tracking on at least one of the foreground region and the background region to determine a motion model for the tracked region; and
changing the set of motion vectors corresponding to the tracked region according to the motion model for the tracked region.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
identifying a region in the first and second images; and
defining a combination map that limits how the vector map is applied to the identified region to transform a pixel from the input image to the output image.

### Claim 5 (depends on 1)

The method of claim 1, wherein changing the set of motion vectors comprises:
receiving an indication of a user specified transform between the first and second images;
computing vectors using the user specified transform and corresponding in time and in resolution with the motion vectors defined by motion estimation; and
combining the computed vectors with the set of motion vectors.

### Claim 6 (depends on 5)

The method of claim 5, wherein the user specified transform is defined by at least one point in a first image and at least one corresponding point in the second image, and further comprising:
determining a transform for warping the first image to the second image that maintains correspondence between the at least one point in the first image and the at least one point in the second image; and
wherein computing vectors comprises determining, for each pixel, a set of transform vectors that describe the spatial transform of the region of the first image to the corresponding region in the second image.

### Claim 7 (depends on 6)

The method of claim 6, wherein the user specified transform is defined by at least one line.

### Claim 8 (depends on 6)

The method of claim 6, wherein the user specified transform is defined by at least one region.

### Claim 9 (depends on 1)

The method of claim 1, wherein changing the set of motion vectors comprises:
displaying to the user a color image defined by the set of motion vectors;
allowing the user to modify the color image defined by the set of motion vectors; and
changing the set of motion vectors according to the modified color image.

### Claim 10

An apparatus for generating motion vectors, comprising:
means for determining a set of motion vectors that describes motion between the first and second images; and
means for changing the set of motion vectors by selecting one or more regions in the image and modifying the vectors corresponding to the selected one or more regions.

### Claim 11 (depends on 10)

The apparatus of claim 10, further comprising means for performing image processing using the changed set of motion vectors.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the means for changing the set of motion vectors comprises:
means for identifying a foreground region and a background region in the first and second images;
means for performing tracking on at least one of the foreground region and the background region to determine a motion model for the tracked region; and
means for changing the set of motion vectors corresponding to the tracked region according to the motion model for the tracked region.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the means for changing the set of motion vectors comprisies:
means for identifying a foreground region and a background region in the first and second images; and
means for defining a combination map that limits how the vector map is applied to transform a pixel from the input image to the output image.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the means for changing the set of motion vectors comprises:
means for receiving an indication of a user specified transform between the first and second images;
means for computing vectors using the user specified transform and corresponding in time and in resolution with the motion vectors defined by motion estimation; and
means for combining the computed vectors with the set of motion vectors.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein the user specified transform is defined by at least one point in a first image and a corresponding at least one point in the second image, and further comprising:
means for determining a transform for warping the first image to the second image that maintains correspondence between the at least one point in the first image and the at least one point in the second image; and
wherein the means for computing vectors comprises means for determining, for each pixel, a set of transform vectors that describe the spatial transform of the region of the first image to the corresponding region in the second image.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the user specified transform is defined by at least one line.

### Claim 17 (depends on 15)

The apparatus of claim 15, wherein the user specified transform is defined by at least one region.

### Claim 18 (depends on 10)

The apparatus of claim 10, wherein changing the set of motion vectors comprises:
displaying to the user a color image defined by the set of motion vectors;
allowing the user to modify the color image defined by the set of motion vectors; and
changing the set of motion vectors according to the modified color image.
