# US 2025/0148676 A1 · Method and Apparatus for the Acquisition, Storage and Display of Three-Dimensional Videos at Variable Frame Rates

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148676 A1
- Document ID: US20250148676A1
- Status: Published application
- Kind: A1
- Application number: 18/937,257
- Filed: 2024-11-05
- Published: 2025-05-08
- Earliest claimed: 2023-11-06
- Assignees: Cinemersive Labs Ltd
- Applicants: Cinemersive Labs Ltd
- Inventors: Victor Lempitskiy; Kirill Dmitrenko; Anuar Taskynov
- Web page: https://patentgrep.com/patent/US20250148676A1

## CPC

- G06T 9/001: {Model-based coding, e.g. wire frame}
- G06T 15/40: Hidden part removal
- G06T 9/002: {using neural networks}
- G06T 13/20: Three-dimensional [3D] animation
- G06T 15/04: Texture mapping
- G06T 15/205: {Image-based rendering}
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 17/205: {Re-meshing}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for displaying a three-dimensional video at a different frame rate from which the video was stored or acquired are described. In one aspect, a frame N corresponding to a frame number N in the video is received at a video reader and decoder. A 3D mesh including a plurality of vertices, a texture, and one or more offset vectors associated with each vertex of the 3D mesh in the frame N are retrieved and transmitted to a rendering device. At least one vertex of the 3D mesh is adjusted according to the corresponding offset vectors and a playback time moment. The playback time moment is between N and N+1, where N+1 is a frame number corresponding to a frame N+1 immediately following the frame N. The adjusted 3D mesh and the texture are rendered according to one or more camera parameters associated with a display device.

## Claims

### Claim 1

A method for displaying a three-dimensional video comprising a plurality of frames, at a different frame rate from which the video was stored or acquired, the method comprising:
receiving, at a video reader and decoder, a frame N corresponding to a frame number N in the video;
retrieving, at the video reader and decoder, a 3D mesh including a plurality of vertices, a texture, and one or more offset vectors associated with each vertex of the 3D mesh in the frame N;
transmitting, to a rendering device, the 3D mesh, the texture, and the offset vectors associated with the frame N;
adjusting, at the rendering device, at least one vertex of the 3D mesh according to the corresponding offset vectors and a playback time moment, wherein the playback time moment is between N and N+1, wherein N+1 is a frame number corresponding to a frame N+1 immediately following the frame N in the video; and
rendering the adjusted 3D mesh and the texture according to one or more camera parameters associated with a display device, wherein the display device is configured to display the video.

### Claim 2 (depends on 1)

The method of claim 1, wherein the retrieving the offset vector further comprises:
substantially optimizing a closeness measure between the frame N and the frame N+1; and
determining the offset vectors based on the substantially optimized closeness measure.

### Claim 3 (depends on 1)

The method of claim 1, wherein the retrieving the offset vectors further comprises:
determining a closeness measure between frame N and frame N+1 via a neural network; and
determining the offset vectors based on the determined closeness measure.

### Claim 4 (depends on 1)

The method of claim 1, wherein the retrieving the offset vectors further comprises:
determining one or more offset vectors such that rasterized projections on a set of two-dimensional views of a scene in the video match one or more optical flow fields between the frame N and the frame N+1 of the scene for the two-dimensional views.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
comparing a rendered image of offset-adjusted frame N, defined as the frame N after the adjusting, with a rendered image of the frame N+1;
computing a loss function based on the comparison; and
adjusting the offset vectors based on the loss function.

### Claim 6 (depends on 5)

The method of claim 5, wherein the method further comprises using the computed loss function for training a neural network that is used to predict the offset vectors.

### Claim 7

A system for displaying a three-dimensional video comprising a plurality of frames, at a different frame rate from which the video was stored or acquired, the system comprising:
a video reader and decoder configured to receive a frame N corresponding to a frame number N in the video;
the video reader and decoder further configured to retrieve a 3D mesh including a plurality of vertices, a texture and one or more offset vectors associated with each vertex of the 3D mesh in the frame N, and transmit, to a rendering device, the 3D mesh, the texture, the and offset vectors associated with the frame N;
the rendering device configured to adjust all vertices of the 3D mesh according to the corresponding offset vectors and a playback time moment, wherein the playback time moment is between N and N+1, wherein N+1 is a frame number corresponding to a frame N+1 immediately following the frame N in the video; and
the rendering device configured to rendering the adjusted 3D mesh and the texture according to camera parameters associated with a display device, wherein the display device is configured to display the video.

### Claim 8 (depends on 7)

The system of claim 7, wherein the retrieving the offset vector further comprises:
substantially optimizing a closeness measure between the frame N and the frame N+1; and
determining the offset vectors based on the substantially optimized closeness measure.

### Claim 9 (depends on 7)

The system of claim 7, wherein the retrieving the offset vectors further comprises:
determining a closeness measure between frame N and frame N+1 via a neural network; and
determining the offset vectors based on the determined closeness measure.

### Claim 10 (depends on 7)

The system of claim 7, wherein the retrieving the offset vectors further comprises:
determining one or more offset vectors such that rasterized projections on a set of two-dimensional views of a scene in the video match one or more optical flow fields between the frame N and the frame N+1 of the scene for the two-dimensional views.

### Claim 11 (depends on 7)

The system of claim 7, further comprising:
comparing a rendered image of offset-adjusted frame N, defined as the frame N after the adjusting, with a rendered image of the frame N+1;
computing a loss function based on the comparison; and
adjusting the offset vectors based on the loss function.

### Claim 12 (depends on 11)

The system of claim 11, wherein the method further comprises using the computed loss function for training a neural network that is used to predict the offset vectors.

### Claim 13 (depends on 1)

A machine-readable storage medium storing a set of instructions that are executable by one or more processors of a system, video reader and decoder, and a rendering device for displaying a three-dimensional video at a different frame rate from which the video was stored or acquired, wherein the set of instructions is configured to perform the method of claim 1.

### Claim 14

A method for training a neural network for offset vectors determination, the method comprising:
comparing a rendered image of an offset-adjusted mesh of a frame N corresponding to a frame number N in a video further comprised of a plurality of frames, with a rendered image of the mesh of a frame N+1 corresponding to a frame number N+1 in the video;
computing a loss function based on the comparing; and
training a neural network used to predict the offset vectors with the loss function.

### Claim 15 (depends on 14)

A machine-readable storage medium storing a set of instructions that are executable by one or more processors of a system for training a neural network for offset determination, wherein the set of instructions are configured to perform the method of claim 14.

### Claim 16 (depends on 1)

The method of claim 1, wherein the playback time interval between N and N+1 is a semi-interval [N, N+1).

### Claim 17 (depends on 1)

The method of claim 1, wherein the display device is a virtual reality headset.

### Claim 18 (depends on 7)

The system of claim 7, wherein the display device is a virtual reality headset.

### Claim 19 (depends on 7)

The system of claim 7, wherein the playback time interval between N and N+1 is a semi-interval [N, N+1).

### Claim 20 (depends on 1)

The method of claim 1, wherein the three-dimensional video is constructed from any of a random distribution of cameras pointing towards one or more three-dimensional scenes, and multiview-stereo data.
