# US 2024/0267537 A1 · Method, apparatus, and medium for video processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0267537 A1
- Document ID: US20240267537A1
- Status: Published application
- Kind: A1
- Application number: 18/618,814
- Filed: 2024-03-27
- Published: 2024-08-08
- Earliest claimed: 2021-09-27
- Assignees: Bytedance Inc.
- Applicants: Bytedance Inc.
- Inventors: Ye-Kui WANG
- Web page: https://patentgrep.com/patent/US20240267537A1

## CPC

- H04N 7/0117: {involving conversion of the spatial resolution of the incoming video signal (for graphics images G09G2340/0407)}
- H04N 21/2362: Generation or processing of Service Information [SI]
- H04N 5/45: Picture in picture {, e.g. displaying simultaneously another television channel in a region of the screen}
- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/172: the region being a picture, frame or field
- H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/188: {the unit being a video data packet, e.g. a network abstraction layer [NAL] unit}
- H04N 19/33: in the spatial domain
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 21/234363: {by altering the spatial resolution, e.g. for clients with a lower screen resolution}
- H04N 21/2365: Multiplexing of several video streams
- H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window}
- H04N 21/4345: {Extraction or processing of SI, e.g. extracting service information from an MPEG stream}
- H04N 21/4347: {Demultiplexing of several video streams}
- H04N 21/8451: {using Advanced Video Coding [AVC]}
- H04N 21/85406: {involving a specific file format, e.g. MP4 format}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing comprises: performing a conversion between a media file of a first video and a bitstream of the first video, wherein the media file comprises a first indication indicating a first set of coded video data units representing a target picture-in-picture region in the first video, the first set of coded video data units being replaceable by a second set of coded video units associated with a second video. The proposed method advantageously makes it possible to support picture-in-picture services in a media file based on ISO base media file format (ISOBMFF).

## Claims

### Claim 1

A method for video processing, comprising:
performing a conversion between a media file of a first video and a bitstream of the first video,
wherein the media file comprises a first indication indicating a first set of coded video data units representing a target picture-in-picture region in the first video, the first set of coded video data units being replaceable by a second set of coded video units associated with a second video.

### Claim 2 (depends on 1)

The method of claim 1, wherein a spatial resolution of the second video being smaller than a spatial resolution of the first video.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first indication comprises a list of region identities (IDs) identifying regions in the first video.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
for one region ID in the list of region IDs,
replacing a first coded video data unit having the region ID in the first set of coded video data units with a second coded video data unit having the region ID in the second set of coded video data units.

### Claim 5 (depends on 3)

The method of claim 3, wherein the first video is coded with versatile video coding (VVC), and
a region ID in the list of region identities is a subpicture ID identifying a subpicture in the first video.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first set of coded video data units comprise a video coding layer network abstraction layer (VCL NAL) unit, and
the second set of coded video data units comprise a VCL NAL unit.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first indication is included in a data structure in the media file.

### Claim 8 (depends on 7)

The method of claim 7, wherein the data structure is a “pinp” entity group.

### Claim 9 (depends on 8)

The method of claim 8, wherein an entity in the “pinp” entity group is a track carrying the bitstream of the first video.

### Claim 10 (depends on 7)

The method of claim 7, wherein the data structure further includes a second indication indicating a set of tracks carrying the bitstream of the first video.

### Claim 11 (depends on 10)

The method of claim 10, wherein the second indication comprises one of:
a value equal to the number of tracks in the set of tracks,
a list of indices indicating identities (IDs) of tracks in the set of tracks, or
a list of track IDs of tracks in the set of tracks.

### Claim 12 (depends on 7)

The method of claim 7, wherein the size of the target picture-in-picture region is smaller than a size of the first video, and the data structure further includes position information and size information of the target picture-in-picture region.

### Claim 13 (depends on 12)

The method of claim 12, wherein the position information indicates a horizontal position and a vertical position of a top-left corner of the target picture-in-picture region, and
the size information indicates a width and a height of the target picture-in-picture region.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
if the media file comprises a third indication indicating that the first set of coded video data units are irreplaceable by the second set of coded video data units, determining a first region in a first video for the second video; and
overlaying the second video on the first video in the first region.

### Claim 15 (depends on 14)

The method of claim 14, wherein the media file further comprises position information and size information of the target picture-in-picture region, and wherein determining the first region comprises:
determining the first region based on the target picture-in-picture region.

### Claim 16 (depends on 15)

The method of claim 15, wherein the position information indicates a horizontal position and a vertical position of a top-left corner of the target picture-in-picture region, and
the size information indicates a width and a height of the target picture-in-picture region.

### Claim 17 (depends on 1)

The method of claim 1, wherein the conversion comprises generating the media file and storing the bitstream to the media file, or
wherein the conversion comprises parsing the media file to reconstruct the bitstream.

### Claim 18

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to perform acts comprising:
performing a conversion between a media file of a first video and a bitstream of the first video,
wherein the media file comprises a first indication indicating a first set of coded video data units representing a target picture-in-picture region in the first video, the first set of coded video data units being replaceable by a second set of coded video units associated with a second video.

### Claim 19

A non-transitory computer-readable storage medium storing instructions that cause a processor to perform acts comprising:
performing a conversion between a media file of a first video and a bitstream of the first video,
wherein the media file comprises a first indication indicating a first set of coded video data units representing a target picture-in-picture region in the first video, the first set of coded video data units being replaceable by a second set of coded video units associated with a second video.

### Claim 20

A non-transitory computer-readable recording medium storing a media file of a first video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
performing a conversion between the media file and a bitstream of the first video,
wherein the media file comprises a first indication indicating a first set of coded video data units representing a target picture-in-picture region in the first video, the first set of coded video data units being replaceable by a second set of coded video units associated with a second video.
