# US 2006/0153287 A1 · Enhancing video images depending on prior image enhancements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2006/0153287 A1
- Document ID: US20060153287A1
- Status: Published application
- Kind: A1
- Application number: 10/537,889
- Filed: 2003-12-12
- Published: 2006-07-13
- Earliest claimed: 2002-12-19
- Assignees: Trident Microsystems (Far East) Ltd.
- Inventors: Richard Shen
- Web page: https://patentgrep.com/patent/US20060153287A1

## CPC

- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/172: the region being a picture, frame or field
- H04N 19/61: in combination with predictive coding
- G06T 2207/10016: Video; Image sequence
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video stream containing encoded frame-based video information includes a first frame and a second frame. The encoding of the second frame depends on the encoding of the first frame. The encoding includes motion vectors indicating differences in positions between regions of the second frame and corresponding regions of the first frame, the motion vectors defining the correspondence between regions of the second frame and regions of the first frame. The first frame is decoded and a re-mapping strategy for video enhancement of the decoded first frame is determined using a region-based analysis. Regions of the decoded first frame are re-mapped according to the determined video enhancement re-mapping strategy for the first frame so as to enhance the first frame. The motion vectors for the second frame are recovered from the video stream and the second frame is decoded.

## Claims

### Claim 1

A method, comprising
receiving a video stream containing encoded frame based video information including an encoded first frame and an encoded second frame, the encoding of the second frame depends on the encoding of the first frame, the encoding of the second frame includes motion vectors indicating differences in positions between regions of the second frame and corresponding regions of the first frame, the motion vectors define the correspondence between regions of the second frame and corresponding regions of the first frame;
decoding the first frame;
determining a re-mapping strategy for video enhancement of the decoded first frame using a region-based analysis;
re-mapping regions of the decoded first frame according to the determined video enhancement re-mapping strategy for the first frame so as to enhance the first frame;
recovering from the video stream, the motion vectors for the second frame;
decoding the second frame;
re-mapping regions of the second frame that correspond to regions of the first frame using the video enhancing re-mapping strategy for the first frame so as to enhance the second frame.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the first frame is an I-frame and the second frame is a subsequent non-I-frame.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the video stream is an MPEG stream of packets; and
the non-I-frame is a P-frame or a B-frame.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the video enhancement re-mapping strategy for the first frame includes re-mapping intensity values to adjust the contrast to enhance the first frame.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the decoding of the second frame is performed using a reconstruction loop; and
the video enhancement re-mapping of the second frame is done completely within the reconstruction loop so that motion vectors do not have to be stored.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
the method further comprises selecting one or more regions of the second frame depending on whether a similarity criteria is met for a similarity between the regions of the second frame and corresponding regions of the first frame, the similarity criteria including measures of similarity in addition to correspondence between regions of the first frame and second frame; and
the re-mapping of the regions of the second frame based on the video enhancement re-mapping strategy for the first frame is only performed for the selected regions of the second frame.

### Claim 7 (depends on 1)

The method of claim 1, wherein:
the method further comprises recovering from the video stream, values of DC coefficients for the second frame, the DC coefficient values being the differences between the first frame and the second frame, in the predicted values of image blocks, after motion compensation, motion compensation being the re-mapping of regions depending on the motion vectors during decoding; and
the method further comprises selecting regions of the second frame depending on whether the values of the DC coefficients for the regions meet a similarity criteria; and
the re-mapping of the regions of the second frame based on the video enhancement re-mapping strategy for the first frame is only performed for the selected regions of the second frame.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
the selection of regions of the second frame for re-mapping based on the video enhancement re-mapping strategy for the first frame, depends on the relation between the values of the DC coefficients for blocks of the region of the second frame and a predetermined or calculated DC coefficient threshold;

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the method further comprises selecting regions of the second frame depending on the respective values of motion vectors for the regions; and
the re-mapping of the regions of the second frame based on the video enhancement re-mapping strategy for the first frame is only performed for the selected regions of the second frame.

### Claim 10 (depends on 1)

The method of claim 1, wherein:
the method further comprises selecting regions of the second frame depending on whether the regions meet a similarity criteria depending similarity between the properties of the motion vectors for the region and the properties of the motion vectors for neighboring regions of the respective regions; and
the re-mapping of the regions of the second frame based on the video enhancement re-mapping strategy for the first frame is only performed for the selected regions of the second frame.

### Claim 11 (depends on 10)

The method of claim 10, wherein:
the properties of the motion vectors on which meeting the similarity criteria depends includes the similarity between the value of the motion vector for a region and the motion vectors for its neighboring regions.

### Claim 12 (depends on 10)

The method of claim 10, wherein:
the properties of the motion vectors on which meeting the similarity criteria depends includes the orthogonality between the value of the motion vector for a region and the motion vectors for its neighboring regions.

### Claim 13 (depends on 1)

The method of claim 1, wherein:
the first frame is an I-frame and the second frame is a subsequent non-I-frame;
the video stream is an MPEG stream of packets and the non-I-frame is a P-frame or a B-frame;
the video enhancement re-mapping strategy for the first frame includes re-mapping intensity values to adjust the contrast to enhance the first frame;
the method further comprises selecting one or more regions of the second frame depending on whether a similarity criteria is met for a similarity between the regions of the second frame and corresponding regions of the first frame and the re-mapping of the regions of the second frame based on the video enhancement re-mapping strategy for the first frame is only performed for the selected regions of the second frame;
the method further comprises recovering from the video stream, values of DC coefficients for the second frame, the DC coefficient values being the differences between the first frame and the second frame, in the predicted values of image blocks, after motion compensation, motion compensation being the re-mapping of regions depending on the motion vectors during decoding; and the meeting of the similarity criteria depends on the values of the DC coefficients;
the meeting of the similarity criteria depends on comparing the properties of the motion vectors for the regions and neighboring regions of respective regions;
the properties of the motion vectors on which meeting the similarity criteria depends includes the similarity between the value of the motion vector for a region and the motion vectors for its neighboring regions;
the properties of the motion vectors on which meeting the similarity criteria depends includes the orthogonality between the value of the motion vector for a region and the motion vectors for its neighboring regions.

### Claim 14

A video decoder, comprising:
an input for receiving a video stream containing encoded frame based video information including an encoded first frame and an encoded second frame, the encoding of the second frame depends on the encoding of the first frame, the encoding of the second frame includes motion vectors indicating differences in positions between regions of the second frame and corresponding regions of the first frame, the motion vectors define correspondence between regions of the second frame and corresponding regions of the first frame;
a decoding unit for decoding the frames, the decoding unit recovers the motion vectors for the second frame;
processing means for determining a re-mapping strategy for video enhancement of the decoded first frame using a region-based analysis, and for re-mapping the first frame using the re-mapping strategy, and for re-mapping one or more regions of the second frame depending on the re-mapping strategy for corresponding regions of the first frame.

### Claim 15 (depends on 14)

The decoder of claim 14 wherein:
the decoder further comprises a buffer;
the decoding unit decodes the first frame and stores the first frame in the buffer,
the processing means re-maps the stored first frame according to the video enhancement re-mapping strategy, and transmits the enhanced first frame;
the decoder further comprises a combiner;
the decoding unit decodes the second frame to determine the differences between the first frame and the second frame, and transmits the differences to the combiner,
the processor again re-maps the intensity values of the first frame depending on the motion vectors of the second frame and transmits the again re-mapped first frame to the combiner;
the combiner combines the again re-mapped first frame with the differences between the first frame and the second frame to produce a decoded second frame that is enhanced.

### Claim 16 (depends on 14)

The decoder of claim 14, wherein:
the processing means selects one or more regions of the second frame depending on whether a similarity criteria is met for a similarity between the regions of the second frame and corresponding regions of the first frame; and
the processing means re-maps the regions of the second frame based on the video enhancement re-mapping strategy for the first frame only for the selected regions of the second frame.

### Claim 17 (depends on 14)

The decoder of claim 14 in which the processing means operates the decoding unit.

### Claim 18 (depends on 14)

A set-top-box, comprising:
a tuner for selecting a video stream of a video program to be played from a plurality of video streams for multiple video programs;
the video decoder of claim 14 for decoding the selected video stream; and
an output for providing the decoded program to a video display.

### Claim 19 (depends on 13)

A video disc player, comprising:
a motor for rotating the disc;
a laser for producing a radiation beam;
an optical system for scanning an information layer of the video disc with the radiation beam, the information layer effecting the beam;
a servo for positioning the optical system;
a radiation detector for detecting beam after it has been effected by the information layer;
processor means to control the servo and motor and to produce a video stream containing encoded information for a group of pictures (GOP) depending on the detection; and
the video decoder of claim 13.

### Claim 20 (depends on 14)

A television, comprising:
a tuner for selecting a video stream of a video program to be played from a plurality of video streams for multiple video programs;
the video decoder of claim 14 for decoding the selected video stream; and
a video display for displaying the decoded frames of the selected video program.

### Claim 21

A method comprising:
receiving a video stream containing encoded information for groups of pictures (GOP), the first picture in a GOP is an I-frame, and a subsequent picture in the GOP is a non-I-frame;
decoding the I-frame;
determining a re-mapping strategy of intensity values to change the contrast to enhance the decoded I-frame using a region-based intensity analysis;
re-mapping the intensity values of the decoded I-frame according to the determined re-mapping strategy;
recovering from the video stream, motion vectors for the subsequent non-I-frame, the motion vectors are differences in the positions of regions in the I-frame and corresponding regions in the non-I-frame;
decoding the subsequent non-I-frame;
determining whether the similarity between corresponding regions meet a similarity criteria;
selecting one or more regions of the non-I-frame depending on whether similarity criteria is met for a similarity between the regions of the non-I-frame and the corresponding regions of the I-frame;
re-mapping the intensity values of the selected regions of the non-I-frame depending on the re-mapping strategy of the corresponding regions of the I-frame so as to change the contrast to enhance the non-I-frame.
