# US 2006/0067585 A1 · Apparatus and method for processing video data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2006/0067585 A1
- Document ID: US20060067585A1
- Status: Published application
- Kind: A1
- Application number: 11/230,686
- Filed: 2005-09-20
- Published: 2006-03-30
- Earliest claimed: 2004-09-21
- Assignees: Euclid Discoveries, LLC
- Inventors: Charles Pace
- Web page: https://patentgrep.com/patent/US20060067585A1

## CPC

- G06T 9/001: {Model-based coding, e.g. wire frame}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/35: using statistical methods
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/20: using video object coding
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- H04N 19/51: Motion estimation or motion compensation
- H04N 19/527: Global motion vector estimation
- H04N 19/53: Multi-resolution motion estimation; Hierarchical motion estimation
- H04N 19/533: Motion estimation using multistep search, e.g. two-dimensional [2D]-log search or one-at-a-time search [OTS]
- H04N 19/537: Motion estimation other than block-based
- H04N 19/54: using feature points or meshes
- H04N 19/553: Motion estimation dealing with occlusions
- H04N 19/61: in combination with predictive coding
- H04N 19/63: using sub-band based transform, e.g. wavelets
- G06T 2207/30241: Trajectory
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An apparatus and methods for processing video data are described. The invention provides a representation of video data that can be used to assess agreement between the data and a fitting model for a particular parameterization of the data. This allows the comparison of different parameterization techniques and the selection of the optimum one for continued video processing of the particular data. The representation can be utilized in intermediate form as part of a larger process or as a feedback mechanism for processing video data. When utilized in its intermediate form, the invention can be used in processes for storage, enhancement, refinement, feature extraction, compression, coding, and transmission of video data. The invention serves to extract salient information in a robust and efficient manner while addressing the problems typically associated with video data sources

## Claims

### Claim 1

A computer apparatus for the purpose of generating an encoded form of video signal data from a plurality of video frames, comprising:
a means of identifying corresponding elements of said object between two or more frames;
a means of modeling such correspondences to generate modeled correspondences;
a means of resampling pel data in said video frames associated with said object, said resampling means utilizing said modeled correspondences;
a means of restoring the spatial positions of the re-sampled pel data, said restoring process utilizing the modeled correspondences;
said object being one or more objects, and
said re-sampled data being the intermediate form of the data.

### Claim 2 (depends on 1)

The method of claim 1 wherein the object is tracked by a tracking method, comprising:
a means of detecting an object in the video frame sequence;
a means of tracking said object through two or more frames of the video frame sequence;
said object detection and tracking means comprising a Viola/Jones face detection algorithm.

### Claim 3 (depends on 1)

The method of claim 1 wherein the object is segmented from video frame using a segmentation method, comprising:
a means of segmenting said pel data associated with said object from other pel data in said video frame sequence;
a means of composing said restored pels together with associated segmentation data to create original video frame;
said segmentation means comprising temporal integration.

### Claim 4 (depends on 1)

The method of claim 1 wherein the correspondence models are factored into global models, comprising:
a means of integrating correspondence measurements into a model of global motion;
said correspondence modeling means comprising a robust sampling consensus for the solution of a two dimensional affine motion model, and
said correspondence modeling means comprising a sampling population based on finite differences generated from block-based motion estimation between two or more video frames in said sequence.

### Claim 5 (depends on 1)

The method of claim 1 wherein the intermediate data is further encoded, comprising:
a means of decomposing said normalized object pel data into an encoded representation;
a means of recomposing said normalized object pel data from encoded representation;
said decomposing means comprising Principal Component Analysis, and
said recomposing means comprising Principal Component Analysis.

### Claim 6 (depends on 1, 2, 3, 4, 5)

The method of claims 2, 3, and 4, 5 wherein the methods are combined, comprising:
a means of tracking a multitude of objects as in claim 2 in order to identify corresponding elements in claim 1;
a means of segmenting the pels as in claim 3 in order to resample pel data in claim 1;
a means of integrating motion parameters as in claim 4 in order to facilitate a compact parameterization of the motion parameters in claim 1;
a means of decomposing and recomposing said normalized and segmented pels as in claim 5.

### Claim 7 (depends on 5)

The method of claim 5 wherein the non-object pels of the frame are modeled as the object pels are in a method, comprising:
said object being the residual, non-object, of the frame when other objects are removed.

### Claim 8 (depends on 5)

The method of claim 5 wherein the segmented and resampled pels are combined with a conventional video compression/decompression process, comprising:
a means of supplying the said re-sampled pels as standard video data to conventional video compression process;
a means of storing and transmitting model correspondence data along with corresponding encoded video data;
whereby said compression/decompression method can enable said conventional video compression method to increase compression efficiency.
