# US 2003/0206246 A1 · Motion estimator for reduced halos in MC up-conversion

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0206246 A1
- Document ID: US20030206246A1
- Status: Published application
- Kind: A1
- Application number: 09/855,628
- Filed: 2001-05-15
- Published: 2003-11-06
- Earliest claimed: 2000-05-18
- Assignees: Koninklijke Philips Electronics N.V.
- Inventors: Gerard De Haan; Mark Jozef Willem Mertens
- Web page: https://patentgrep.com/patent/US20030206246A1

## CPC

- H04N 19/553: Motion estimation dealing with occlusions
- G06T 7/238: using non-full search, e.g. three-step search
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- G06T 2207/10016: Video; Image sequence
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An apparatus for detecting motion at a temporal intermediate position between a previous image and a next image has optimizing means (1) for optimizing a criterion function for candidate vectors, whereby the criterion function depends on data from both the previous and the next image. The motion is detected at the temporal intermediate position in non-covering and in non-uncovering areas. The apparatus according to the invention has means for detecting covering and uncovering areas (2) and has its optimizing means being arranged to carry out the optimizing at the temporal position of the next image in covering areas and at the temporal position of the previous image in uncovering areas.

## Claims

### Claim 1

Method for detecting motion at a temporal intermediate position between previous and next images, in which a criterion function for candidate vectors is optimised, said function depending on data from both previous and next images and in which the optimising is carried out at the temporal intermediate position in non-covering and non-uncovering areas, characterised in that the optimising is carried out at the temporal position of the next image in covering areas and at the temporal position of the previous image in uncovering areas.

### Claim 2 (depends on 1)

Method according to claim 1, wherein the previous image is shifted over a fraction α times the candidate vector, the next image is shifted over 1−α times the candidate vector and the fraction a may change within the image period.

### Claim 3 (depends on 1)

Method according to claim 1, wherein the criterion function is a match error which is minimised.

### Claim 4 (depends on 2)

Method according to claim 2, wherein the fraction α is controlled by a covering/uncovering detector in the matching process.

### Claim 5 (depends on 4)

Method according to claim 4, wherein the fraction α is set to 1 in case of covering and set to 0 in case of uncovering.

### Claim 6 (depends on 4)

Method according to claim 4, wherein the covering/uncovering detector decides on data in a previous image to the fraction α in the current estimation.

### Claim 7 (depends on 1)

Method according to claim 1, wherein a velocity edge XE is determined, an occlusion area is marked around said edge and in said area foreground velocity is replaced by background velocity or reversibly dependent on the occlusion is a covering or uncovering area, the sign of the foreground velocity and on which side of the velocity edge XE the foreground is.

### Claim 8 (depends on 7)

Method according to claim 7, wherein at the position {right arrow over (x)}1 of a velocity edge
a first position {right arrow over (x)}α in the previous (covering) or next (uncovering) image is calculated by shifting {right arrow over (x)}1 over the first vector at one side of the edge
a second position {right arrow over (x)}b in the previous (covering) or next (uncovering) image is calculated by shifting {right arrow over (x)}1 over the second vector at the other side of the edge
and a third intermediate position between {right arrow over (x)}α and {right arrow over (x)}b is calculated
while finally, the vector fetched with v{right arrow over (α)}v at the third position in the previous (covering) or next (uncovering) image is filled in those regions of the image in the environment of the edge, to which no vector is projected, in case the background vector v{right arrow over (F)}G should be filled in and the vector chosen between
8 D → ⁡ ( x → - ( 1 0 ) , n )
and
9 ⁢ D → ⁡ ( x → + ( 1 0 ) , n )
which is most different from v{right arrow over (α)}v is filled in, in case a foreground vector v{right arrow over (F)}G should be filled in.

### Claim 9 (depends on 8)

Method according to claim 8, wherein the intermediate position is ({right arrow over (x)}α+{right arrow over (x)}b)/2.

### Claim 10 (depends on 7)

Method according to claim 7, wherein a background velocity is identified as a velocity which crosses the velocity discontinuity and projects to a foreground velocity in the previous picture, whereas a foreground velocity projects to itself.

### Claim 11 (depends on 7)

Method according to claim 7, wherein near edges it is tested whether the mentioned edge has moved over the first vector on one side of the edge, or over the second vector on the other side of the edge, in case the edge moves with the first (second) vector, the second (first) vector is filled in those regions of the projected vector field in the environment of the edge, to which no vector is projected, in case a background vector v{right arrow over (B)}G should be filled in, and the other vector is filled in, in case a foreground vector v{right arrow over (F)}G should be filled.

### Claim 12 (depends on 10)

Method according to claim 10, wherein the crossing from a background region to a foreground region in the previous image is verified by the match error of the vector in that block.

### Claim 13

Apparatus for detecting motion at a temporal intermediate position between previous and next images, comprising means (1) optimising a criterion function for candidate vectors, said function depending on data from both previous and next images in which the optimising is carried out at the temporal intermediate position in non-covering and non-uncovering areas, characterised in that means for detection covering or uncovering areas (2) are provided and that the optimising is carried out at the temporal position of the next image in covering areas and at the temporal position of the previous image in uncovering areas.

### Claim 14 (depends on 13)

Apparatus according to claim 13, wherein the previous image is shifted over a fraction α times the candidate vector, the next image is shifted over 1−α times the candidate vector and the fraction α may change within the image period.

### Claim 15 (depends on 13)

Apparatus according to claim 13, wherein the criterion function is a match error which is minimised.

### Claim 16 (depends on 14)

Apparatus according to claim 14, wherein the fraction α is controlled by a covering/uncovering detector (2) in the matching process.

### Claim 17 (depends on 16)

Apparatus according to claim 16, wherein the fraction α is set to 1 in case of covering and set to 0 in case of uncovering.

### Claim 18 (depends on 16)

Apparatus according to claim 16, wherein the covering/uncovering detector (2) decides on data in a previous image to the fraction α in the current estimation.

### Claim 19

Apparatus according to one of the preceding claims, wherein a velocity edge XE is determined, an occlusion area is marked around said edge and in said area foreground velocity is replaced by background velocity or reversibly dependent on the occlusion is a covering or uncovering area, the sign of the foreground velocity and on which side of the velocity edge XE the foreground is.

### Claim 20 (depends on 19)

Apparatus according to claim 19, wherein calculation means (5,6,8) are provided for, at the position {right arrow over (x)}1 of a velocity edge, calculating
a first position {right arrow over (x)}α in the previous (covering) or next (uncovering) image by shifting {right arrow over (x)}1 over the first vector at one side of the edge
a second position {right arrow over (x)}b in the previous (covering) or next (uncovering) image by shifting {right arrow over (x)}1 over the second vector at the other side of the edge
and a third intermediate position between {right arrow over (x)}α and {right arrow over (x)}b,
while finally, the vector fetched with v{right arrow over (α)}v at the third position in the previous (covering) or next (uncovering) image (9) is filled in those regions of the image in the environment of the edge, to which no vector is projected, in case the background vector v{right arrow over (F)}G should be filled in and the vector chosen between
10 D → ⁡ ( x → - ( 1 0 ) , n ) ⁢ ⁢ and ⁢ ⁢ D → ⁡ ( x → + ( 1 0 ) , n )
which is most different from v{right arrow over (α)}v is filled in, in case a foreground vector v{right arrow over (F)}G should be filled in.

### Claim 21 (depends on 20)

Apparatus according to claim 20, wherein the intermediate position is ({right arrow over (x)}α+{right arrow over (x)}b)/2.

### Claim 22 (depends on 19)

Apparatus according to claim 19, wherein means (10,11) are provided for projecting two positions on either side of the edge to the previous (covering) or next (uncovering) image, in which a background velocity is identified (14) as a velocity which crosses the velocity discontinuity and projects to a foreground velocity in the previous picture, whereas a foreground velocity projects to itself.

### Claim 23 (depends on 19)

Apparatus according to claim 19, wherein means (20) are provided for testing near edges whether the mentioned edge has moved over the first vector on one side of the edge, or over the second vector on the other side of the edge, in case the edge moves with the first (second) vector, the second (first) vector is filled in those regions of the projected vector field in the environment of the edge, to which no vector is projected, in case a background vector v{right arrow over (B)}G should be filled in, and the other vector is filled in, in case a foreground vector v{right arrow over (F)}G should be filled.

### Claim 24 (depends on 22)

Apparatus according to claim 22, wherein verification means are provided for verifying the crossing from a background region to a foreground region in the previous image by the match error of the vector in that block.

### Claim 25 (depends on 13)

Image display apparatus comprising detection apparatus (21) for detection a motion vector according to claim 13, means (22) for interpolating image parts connected to said detection apparatus (21) and a display device (23) connected to the interpolating means.
