# US 2021/0344936 A1 · Method and device for transcoding video

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0344936 A1
- Document ID: US20210344936A1
- Status: Published application
- Kind: A1
- Application number: 17/088,442
- Filed: 2020-11-03
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: Wangsu Science & Technology Co., Ltd.
- Applicants: Wangsu Science & Technology Co., Ltd.
- Inventors: Anjie LIU
- Web page: https://patentgrep.com/patent/US20210344936A1

## CPC

- H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
- H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- H04N 19/115: Selection of the code volume for a coding unit prior to coding
- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/1883: {the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]}
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment}
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure discloses a method and device for transcoding a video, which belongs to the video processing technology. The method includes: acquiring a target frame image of a video to be transcoded, and generating a global feature map of the target frame image based on a feature extraction module of a semantic segmentation model; performing feature segmentation on the global feature map based on a feature segmentation module of the semantic segmentation model, and determining a multi-level ROI of the target frame image; and using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively.

## Claims

### Claim 1

A method for transcoding a video, comprising:
acquiring a target frame image of a video to be transcoded, and generating a global feature map of the target frame image based on a feature extraction module of a semantic segmentation model;
performing feature segmentation on the global feature map based on a feature segmentation module of the semantic segmentation model, and determining a multi-level region of interest (ROI) of the target frame image; and
using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively;
wherein determining the multi-level ROI of the target frame image comprises:
determining multiple components of at least one target object in the target frame image; and
determining the multi-level ROI of the target frame image based on a preset transcoding priority corresponding to each of the multiple components.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The method according to claim 1, further comprising:
training the semantic segmentation model periodically based on image materials in a preset training material set, wherein the image materials are marked with contour lines of various components of different objects.

### Claim 4 (depends on 1)

The method according to claim 1, wherein before generating the global feature map of the target frame image based on the feature extraction module of the semantic segmentation model, the method further comprises:
detecting a target video type to which the video to be transcoded belongs, and calling a target semantic segmentation model corresponding to the target video type of the video to be transcoded; wherein dedicated semantic segmentation models are trained for frame images in videos of different video types, and one semantic segmentation model is dedicated to performing the semantic segmentation on frame images in videos of one video type; or
detecting a target image type to which the target frame image belongs, and calling a target semantic segmentation model corresponding to the target image type of the target frame image; wherein dedicated semantic segmentation models are trained for frame images of different image types, and one semantic segmentation model is dedicated to performing the semantic segmentation on frame images of one image type; or
detecting a target video type to which the video to be transcoded belongs and a target image type to which the target frame image belongs, and calling a target semantic segmentation model corresponding to the target video type and the target image type; wherein, dedicated semantic segmentation models are trained for frame images of different image types in videos of different video types, and one semantic segmentation model is dedicated to performing the semantic segmentation on frame images of one image type in the video of one video type.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the higher the transcoding priority corresponding to a ROI, the higher the transcoding rate of the ROI; and the other regions are transcoded with a lowest transcoding rate.

### Claim 6 (depends on 5)

The method according to claim 5, wherein using the different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises:
selecting a target-level transcoding rate sequentially in a descending order of the transcoding rate; and
selecting, in a descending order of the transcoding priority, a transcoding region corresponding to the target-level transcoding rate from unselected ROIs according to a transcoding area corresponding to the target-level transcoding rate.

### Claim 7 (depends on 6)

The method according to claim 6, further comprising:
adjusting the transcoding area corresponding to the transcoding rate of each level periodically according to current device performance load and line bandwidth load, and attribute information of the video to be transcoded.

### Claim 8 (depends on 1)

The method according to claim 1, wherein after determining multiple components of the at least one target object in the target frame image, the method further comprises:
adjusting the transcoding priority corresponding to the components of each target object according to a video type of the video to be transcoded and an object type of each target object.

### Claim 9 (depends on 1)

The method according to claim 1, wherein using the different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises:
receiving information of feature clarity or fuzz of the video to be transcoded, and determining a target feature region in the target frame image according to the information of feature clarity or fuzz; and
using the different transcoding rates to transcode the multi-level ROI, the target feature region and other regions of the target frame image respectively.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the semantic segmentation model uses a TensorRT engine to implement accelerated operations.

### Claim 11

A background server, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set which, when loaded and executed by the processor, causes the processor to perform a method for transcoding a video; wherein the method comprises:
acquiring a target frame image of a video to be transcoded, and generating a global feature map of the target frame image based on a feature extraction module of a semantic segmentation model;
performing feature segmentation on the global feature map based on a feature segmentation module of the semantic segmentation model, and determining a multi-level region of interest (ROI) of the target frame image; and
using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively;
wherein determining the multi-level ROI of the target frame image comprises:
determining multiple components of at least one target object in the target frame image; and
determining the multi-level ROI of the target frame image based on a preset transcoding priority corresponding to each of the multiple components.

### Claim 12

(canceled)

### Claim 13 (depends on 11)

The background server according to claim 11, wherein the method further comprises:
training the semantic segmentation model periodically based on image materials in a preset training material set, wherein the image materials are marked with contour lines of various components of different objects.

### Claim 14 (depends on 11)

The background server according to claim 11, wherein before generating the global feature map of the target frame image based on the feature extraction module of the semantic segmentation model, the method further comprises:
detecting a target video type to which the video to be transcoded belongs, and calling a target semantic segmentation model corresponding to the target video type of the video to be transcoded; wherein dedicated semantic segmentation models are trained for frame images in videos of different video types, and one semantic segmentation model is dedicated to performing the semantic segmentation on frame images in videos of one video type; or
detecting a target image type to which the target frame image belongs, and calling a target semantic segmentation model corresponding to the target image type of the target frame image; wherein dedicated semantic segmentation models are trained for frame images of different image types, and one semantic segmentation model is dedicated to performing the semantic segmentation on frame images of one image type; or
detecting a target video type to which the video to be transcoded belongs and a target image type to which the target frame image belongs, and calling a target semantic segmentation model corresponding to the target video type and the target image type; wherein, dedicated semantic segmentation models are trained for frame images of different image types in videos of different video types, and one semantic segmentation model is dedicated to performing the semantic segmentation on frame images of one image type in the video of one video type.

### Claim 15 (depends on 11)

The background server according to claim 11, wherein the higher the transcoding priority corresponding to a ROI, the higher the transcoding rate of the ROI; and the other regions are transcoded with a lowest transcoding rate.

### Claim 16 (depends on 15)

The background server according to claim 15, wherein using the different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises:
selecting a target-level transcoding rate sequentially in a descending order of the transcoding rate; and
selecting, in a descending order of the transcoding priority, a transcoding region corresponding to the target-level transcoding rate from unselected ROIs according to a transcoding area corresponding to the target-level transcoding rate.

### Claim 17 (depends on 16)

The background server according to claim 16, wherein the method further comprises:
adjusting the transcoding area corresponding to each level of the transcoding rate periodically according to current device performance load and line bandwidth load, and attribute information of the video to be transcoded.

### Claim 18 (depends on 11)

The background server according to claim 11, wherein after determining multiple components of the at least one target object in the target frame image, the method further comprises:
adjusting the transcoding priority corresponding to the components of each target object according to a video type of the video to be transcoded and an object type of each target object.

### Claim 19 (depends on 11)

The background server according to claim 11, wherein using the different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises:
receiving information of feature clarity or fuzz of the video to be transcoded, and determining a target feature region in the target frame image according to the information of feature clarity or fuzz; and
using the different transcoding rates to transcode the multi-level ROI, the target feature region and other regions of the target frame image respectively.

### Claim 20

A computer readable storage medium storing at least one instruction, at least one program, a code set or an instruction set which, when loaded and executed by a processor, causes the processor to perform a method for transcoding a video; wherein the method comprises:
acquiring a target frame image of a video to be transcoded, and generating a global feature map of the target frame image based on a feature extraction module of a semantic segmentation model;
performing feature segmentation on the global feature map based on a feature segmentation module of the semantic segmentation model, and determining a multi-level region of interest (ROI) of the target frame image; and
using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively;
wherein determining the multi-level ROI of the target frame image comprises:
determining multiple components of at least one target object in the target frame image; and
determining the multi-level ROI of the target frame image based on a preset transcoding priority corresponding to each of the multiple components.
