# US 2025/0095115 A1 · Film grain analysis, synthesis, and removal

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095115 A1
- Document ID: US20250095115A1
- Status: Published application
- Kind: A1
- Application number: 18/471,194
- Filed: 2023-09-20
- Published: 2025-03-20
- Earliest claimed: 2023-09-20
- Assignees: Disney Enterprises, Inc.; Beijing YoJaJa Software Technology Development Co., Ltd.; ETH Zürich (Eidgenössische Technische Hochschule Zürich)
- Applicants: Disney Enterprises, Inc.; Beijing YoJaJa Software Technology Development Co., Ltd.; ETH Zürich (Eidgenössische Technische Hochschule Zürich)
- Inventors: Abdelaziz Djelouah; Yang Zhang; Roberto Gerson De Albuquerque Azevedo; Elham Amin Mansour; Mingyang Song; Christopher Richard Schroers; Yuanyi Xue; Scott Labrozzi; Wenhao Zhang; Xuewei Meng; Jeroen Schulte
- Web page: https://patentgrep.com/patent/US20250095115A1

## CPC

- G06T 5/70: Denoising; Smoothing
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20204: Removing film grain; Adding simulated film grain
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

In some embodiments, a grain analysis system is configured for analyzing a first video frame and outputting respective first film grain information for film grain that is included in the first video frame or configured for analyzing a second video frame and outputting second film grain information. At least one of a grain removal system and a grain synthesis system is included. The grain removal system is configured for removing the film grain from the first video frame using the first film grain information to generate a third video frame corresponding to the first video frame with film grain removed. The grain analysis system is separate from the grain removal system. The grain synthesis system is configured for synthesizing film grain for the third video frame using the first film grain information or the second film grain information. The grain analysis system is separate from the grain synthesis system.

## Claims

### Claim 1

A system comprising:
a grain analysis system configured for analyzing a first video frame and outputting respective first film grain information for film grain that is included in the first video frame or configured for analyzing a second video frame and outputting second film grain information; and
at least one of:
a grain removal system configured for removing the film grain from the first video frame using the first film grain information to generate a third video frame corresponding to the first video frame with film grain removed, wherein the grain analysis system is separate from the grain removal system; or
a grain synthesis system configured for synthesizing film grain for the third video frame using the first film grain information or the second film grain information, wherein the grain analysis system is separate from the grain synthesis system.

### Claim 2 (depends on 1)

The system of claim 1, wherein the grain analysis system provides the first film grain information to either the grain removal system or the grain synthesis system and the second film grain information to the grain synthesis system.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the system includes the grain removal system,
the grain analysis system outputs the first film grain information to the grain removal system, and
the grain removal system removes the film grain from the first video frame using the first film grain information to output the third video frame.

### Claim 4 (depends on 3)

The system of claim 3, wherein the first film grain information is edited after being output by the grain analysis system and before being used by the grain removal system to remove the film grain.

### Claim 5 (depends on 1)

The system of claim 1, wherein the system includes the grain removal system, and the grain removal system comprises:
a transformer configured to receive a plurality of first video frames and the first film grain information for the plurality of first video frames, and is trained to output the plurality of first video frames with film grain removed.

### Claim 6 (depends on 5)

The system of claim 5, wherein the transformer is configured to perform spatial processing and temporal processing simultaneously to remove the film grain from the plurality of first video frames, wherein the temporal processing is based on temporal features from multiple first video frames in the plurality of first video frames and spatial processing is based on spatial features of a single first video frame in the plurality of first video frames.

### Claim 7 (depends on 1)

The system of claim 1, further comprising:
an encoder configured to:
encode the third video frame to generate an encoded third video frame, and
encode the first film grain information to generate encoded first film grain information, and
a decoder configured to decode the encoded third video frame to generate a decoded third video frame, and to decode the encoded first film grain information to generate decoded first film grain information, wherein the system includes the grain synthesis system, and the grain synthesis system synthesizes the film grain for the decoded third video frame using the decoded first film grain information.

### Claim 8 (depends on 1)

The system of claim 1, wherein:
the system includes the grain removal system and the grain synthesis system,
the grain analysis system and the grain removal system are located on a server, and
the grain synthesis system is located on a receiver.

### Claim 9 (depends on 1)

The system of claim 1, wherein:
the system includes the grain synthesis system, and
the grain synthesis system injects noise into the third video frame in multiple stages to synthesize the film grain.

### Claim 10 (depends on 9)

The system of claim 9, wherein the grain synthesis system comprises:
a first block configured to combine the third video frame and the first film grain information or the second film grain information to generate combined third video frame and grain information features,
a plurality of second blocks configured to extract features from the combined third video frame and grain information features; and
a plurality of third blocks configured to receive the extracted features from the plurality of second blocks, inject noise into the extracted features, and output the synthesized film grain.

### Claim 11 (depends on 10)

The system of claim 10, wherein:
respective second blocks output extracted features that are combined with features in which noise was injected by respective third blocks.

### Claim 12 (depends on 11)

The system of claim 11, further comprising:
a combination block configured to:
combine a noise map with extracted features from a second block that is last in a series of the plurality of second blocks to generate a combined noise map and extracted features, and
input the combined noise map and extracted features into a third block in a series of the plurality of third blocks.

### Claim 13 (depends on 1)

The system of claim 1, wherein:
the system includes the grain synthesis system, and
the grain synthesis system synthesizes the film grain for the third video frame using the second film grain information.

### Claim 14 (depends on 13)

The system of claim 13, wherein the second video frame includes film grain that includes characteristics for the film grain that is inserted into the third video frame.

### Claim 15 (depends on 1)

The system of claim 1, wherein the grain analysis system comprises:
an analysis encoder configured to analyze the first video frame or the second video frame and output latent code for the first video frame or the second video frame, and
a decoder configured to analyze the latent code and output the first film grain information or the second film grain information.

### Claim 16 (depends on 15)

The system of claim 15, wherein the analysis encoder and the decoder are trained to output first film grain information or second film grain information that have similar settings in a space.

### Claim 17 (depends on 1)

The system of claim 1, wherein:
a combination of two or more of the grain analysis system, the grain removal system, or the grain synthesis system are jointly trained together.

### Claim 18 (depends on 1)

The system of claim 1, wherein:
different combinations of two or more of the grain analysis system, the grain removal system, or the grain synthesis system are jointly trained together, and
different inputs and known outputs are used to train the different combinations.

### Claim 19

A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:
analyzing, by a grain analysis system, a first video frame and outputting respective first film grain information for film grain that is included in the first video frame or analyzing a second video frame and outputting second film grain information; and
at least one of:
removing, by a grain removal system, the film grain from the first video frame using the first film grain information to generate a third video frame corresponding to the first video frame with film grain removed, wherein the grain analysis system is separate from the grain removal system; or
synthesizing, by a grain synthesis system, film grain for the third video frame using the first film grain information or the second film grain information, wherein the grain analysis system is separate from the grain synthesis system.

### Claim 20

A method comprising:
analyzing, by a grain analysis system, a first video frame and outputting respective first film grain information for film grain that is included in the first video frame or analyzing a second video frame and outputting second film grain information; and
at least one of:
removing, by a grain removal system, the film grain from the first video frame using the first film grain information to generate a third video frame corresponding to the first video frame with film grain removed, wherein the grain analysis system is separate from the grain removal system; or
synthesizing, by a grain synthesis system, film grain for the third video frame using the first film grain information or the second film grain information, wherein the grain analysis system is separate from the grain synthesis system.
