# US 2026/0292287 A1 · Pose aggregation for multi-view temporal video alignment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0292287 A1
- Document ID: US20260292287A1
- Status: Published application
- Kind: A1
- Application number: 19/569,945
- Filed: 2026-03-17
- Published: 2026-09-24
- Earliest claimed: 2025-03-18
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Fabien Paul DELATTRE; Tsung-Wei HUANG; Guan-Ming SU
- Web page: https://patentgrep.com/patent/US20260292287A1

## CPC

- H04N 21/43072: {of multiple content streams on the same device}
- G06T 5/10: using non-spatial domain filtering
- G06T 7/73: using feature-based methods
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system and method for temporally aligning multiple videos captured from different viewpoints. The system includes a 2D pose extractor that extracts joint coordinates of deformable objects from video frames, a view-invariant pose encoder that generates view-invariant descriptors from the joint coordinates, a frame aggregator that combines multiple descriptors from each frame into a single frame-level descriptor, and a temporal aggregator that incorporates motion information from adjacent frames. The system determines temporal alignment between videos by comparing frame descriptors in the frequency domain using an alignment algorithm. The method supports videos captured from arbitrary viewpoints without requiring metadata or audio information, handles varying numbers of subjects in different frames, operates efficiently through frequency domain processing, and can be extended to align three or more videos while maintaining temporal consistency.

## Claims

### Claim 1

A method for temporally aligning multiple videos, comprising:
extracting, using a pose extractor, joint coordinates of deformable objects from frames of multiple input videos;
generating, using a view-invariant pose encoder, view-invariant descriptors from the extracted joint coordinates;
aggregating, using a frame aggregator, multiple view-invariant descriptors from each frame into a single frame-level descriptor;
performing temporal aggregation, using a temporal aggregator, across adjacent frames to generate temporally-aware frame descriptors; and
determining temporal alignment between videos by comparing the temporally-aware frame descriptors using an alignment algorithm.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating view-invariant descriptors comprises:
processing the joint coordinates through the view-invariant pose encoder to generate fixed-dimension embedding vectors that are invariant to viewpoint changes.

### Claim 3 (depends on 1)

The method of claim 1, wherein aggregating multiple view-invariant descriptors comprises:
projecting the view-invariant descriptors using a projector component to a higher-dimensional space; and
applying softmax pooling using a pooling layer to combine multiple descriptors into a single frame-level descriptor.

### Claim 4 (depends on 3)

The method of claim 3, wherein the softmax pooling uses a temperature parameter to control a balance between maximum and average pooling behaviors.

### Claim 5 (depends on 1)

The method of claim 1, wherein performing temporal aggregation comprises:
processing descriptors from multiple adjacent frames through either a transformer-based architecture or an MLP-based architecture to generate temporally-aware frame descriptors.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining temporal alignment comprises:
computing cross-correlation between frame descriptor sequences in the frequency domain using Fast Fourier Transform (FFT) or in the time domain; and
identifying the temporal shift that maximizes similarity between the sequences.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining similarity between frame descriptors comprises:
cosine similarity; or
similarity by parts.

### Claim 8 (depends on 1)

The method of claim 1, wherein the pose extractor is a 2D or 3D pose extractor, and the joint coordinates are 2D or 3D coordinates.

### Claim 9

A system (100) for video alignment, comprising:
a pose extractor configured to identify and extract joint coordinates from video frames;
a view-invariant encoder configured to generate pose descriptors that are invariant to viewpoint changes;
a frame aggregator configured to combine multiple pose descriptors into single frame-level descriptors; and
a temporal aggregator configured to incorporate motion information from adjacent frames.

### Claim 10 (depends on 9)

The system of claim 9, wherein the frame aggregator comprises:
a projector component that maps pose descriptors to a higher-dimensional space; and
a pooling layer that combines multiple projected descriptors into a single frame descriptor.

### Claim 11 (depends on 9)

The system of claim 9, wherein the temporal aggregator implements either:
a transformer-based architecture for processing multiple frames simultaneously; or
an MLP-based architecture for processing a smaller number of adjacent frames.

### Claim 12 (depends on 9)

The system of claim 9, further comprising:
an alignment module configured to compute temporal alignment between videos by comparing frame descriptors in the frequency domain using the alignment algorithm.

### Claim 13

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:
extract pose information using a pose extractor from multiple videos;
generate view-invariant descriptors for the extracted poses using a view-invariant encoder;
aggregate pose descriptors into frame-level descriptors using a frame aggregator;
combine information from adjacent frames using a temporal aggregator; and
determine temporal alignment between the videos using frequency domain or time domain analysis.

### Claim 14

A method of training a video alignment system, comprising:
providing pairs of synchronized video frames from different viewpoints as positive training examples;
providing non-synchronized video frames as negative training examples; and
training the system using contrastive learning with an InfoNCE loss function.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
freezing weights of pre-trained pose extraction and view-invariant encoding modules; and
training only frame aggregation and temporal aggregation modules.

### Claim 16 (depends on 1)

The method of claim 1, wherein the deformable objects comprise human figures, and the joint coordinates correspond to human body joints.

### Claim 17 (depends on 1)

The method of claim 1, further comprising:
extending the alignment to three or more videos by maintaining consistency across pairwise alignments.
