# US 2021/0344937 A1 · Method and device for transcoding video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344937 A1 - Document ID: US20210344937A1 - Status: Published application - Kind: A1 - Application number: 17/095,728 - Filed: 2020-11-11 - Published: 2021-11-04 - Earliest claimed: 2020-04-30 - Assignees: Wangsu Science & Technology Co., Ltd. - Applicants: Wangsu Science & Technology Co., Ltd. - Inventors: Anjie LIU - Web page: https://patentgrep.com/patent/US20210344937A1 ## CPC - H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06T 7/11: Region-based segmentation - G06T 9/002: {using neural networks} - H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) - H04N 19/167: Position within a video image, e.g. region of interest [ROI] - H04N 19/20: using video object coding - H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) - H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution - G06T 2207/10016: Video; Image sequence - G06T 2207/20081: Training; Learning - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present disclosure is related to video processing technologies and discloses a method and a device for transcoding a video. The method includes: reconstructing an existing semantic segmentation model based on a processing layer supported by a TensorRT engine; loading a reconstructed semantic segmentation model into the TensorRT engine to generate a TensorRT acceleration model; performing semantic segmentation on a video frame image through the TensorRT acceleration model to generate a multi-level ROI of the video frame image; using different transcoding rates to transcode the multi-level ROI and other regions of the video frame image respectively. ## Claims ### Claim 1 A method for transcoding a video, comprising: reconstructing an existing semantic segmentation model based on a processing layer supported by a TensorRT engine; loading a reconstructed semantic segmentation model into the TensorRT engine to generate a TensorRT acceleration model; performing semantic segmentation on a video frame image through the TensorRT acceleration model to generate a multi-level ROI of the video frame image; and using different transcoding rates to transcode the multi-level ROI and other regions of the video frame image respectively. ### Claim 2 (depends on 1) The method according to claim 1, wherein the semantic segmentation model is an existing PSPNet model; and reconstructing the existing semantic segmentation model based on the processing layer supported by the TensorRT engine comprises: respectively replacing a self-adaptive pooling layer and a bilinear upsampling layer in the existing PSPNet model with processing layers having similar effects and supported by the TensorRT engine. ### Claim 3 (depends on 2) The method according to claim 2, wherein respectively replacing the adaptive pooling layer and bilinear upsampling layer in the existing PSPNet model with processing layers having similar effects and supported by the TensorRT engine comprises: replacing the self-adaptive pooling layer in the existing PSPNet model with a multi-scale mean pooling layer; and replacing the bilinear upsampling layer in the existing PSPNet model with a transpose convolution layer. ### Claim 4 (depends on 3) The method according to claim 3, wherein replacing the bilinear upsampling layer in the existing PSPNet model with the transpose convolution layer comprises: replacing a multi-scale bilinear upsampling layer in the existing PSPNet model with a nearest neighbor pool upsampling layer and a shared transpose convolution layer. ### Claim 5 (depends on 1) The method according to claim 1, wherein loading the reconstructed semantic segmentation model into the TensorRT engine to generate the TensorRT acceleration model comprises: training the reconstructed semantic segmentation model periodically based on image materials in a preset training material set to generate a weight file, wherein the image materials are marked with contour lines of various components of different objects; and loading a model file of the reconstructed semantic segmentation model and the weight file into the TensorRT engine to generate the TensorRT acceleration model. ### Claim 6 (depends on 1) The method according to claim 1, wherein performing the semantic segmentation on the video frame image through the TensorRT acceleration model to generate the multi-level ROI of the video frame image comprises: acquiring a target frame image of a video to be transcoded, and generating a global feature map of the target frame image based on a feature extraction module of the TensorRT acceleration model; performing feature segmentation on the global feature map through a feature segmentation module of the TensorRT acceleration model to determine multiple components of at least one target object in the target frame image; and determining the multi-level ROI of the target frame image based on a transcoding priority corresponding to each of the components. ### Claim 7 (depends on 6) The method according to claim 6, wherein using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises: selecting a target-level transcoding rate sequentially in a descending order of transcoding rates; and selecting, in a descending order of transcoding priorities, a transcoding region corresponding to the target-level transcoding rate from unselected ROIs according to a transcoding area corresponding to the target-level transcoding rate. ### Claim 8 (depends on 7) The method according to claim 7, further comprising: adjusting the transcoding area corresponding to a transcoding rate of each level periodically according to current device performance load and line bandwidth load, and attribute information of the video to be transcoded. ### Claim 9 (depends on 6) The method according to claim 6, wherein after determining multiple components of at least one target object in the target frame image, the method further comprises: adjusting the transcoding priority corresponding to components of each target object according to a video type of the video to be transcoded and an object type of each of the target object. ### Claim 10 (depends on 1) The method according to claim 1, wherein using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises: receiving information of feature clarity or fuzz of the video to be transcoded, and determining a target feature region in the target frame image according to the information of feature clarity or fuzz; and using the different transcoding rates to transcode the multi-level ROI, the target feature region and other regions of the target frame image respectively. ### Claim 11 A background server, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set which, when loaded and executed by the processor, causes the processor to perform a method for transcoding a video; wherein the method comprises: reconstructing an existing semantic segmentation model based on a processing layer supported by a TensorRT engine; loading a reconstructed semantic segmentation model into the TensorRT engine to generate a TensorRT acceleration model; performing semantic segmentation on a video frame image through the TensorRT acceleration model to generate a multi-level ROI of the video frame image; and using different transcoding rates to transcode the multi-level ROI and other regions of the video frame image respectively. ### Claim 12 (depends on 11) The background server according to claim 11, wherein the semantic segmentation model is an existing PSPNet model; and reconstructing the existing semantic segmentation model based on the processing layer supported by the TensorRT engine comprises: respectively replacing a self-adaptive pooling layer and a bilinear upsampling layer in the existing PSPNet model with processing layers having similar effects and supported by the TensorRT engine. ### Claim 13 (depends on 12) The background server according to claim 12, wherein respectively replacing the adaptive pooling layer and bilinear upsampling layer in the existing PSPNet model with processing layers having similar effects and supported by the TensorRT engine comprises: replacing the self-adaptive pooling layer in the existing PSPNet model with a multi-scale mean pooling layer; and replacing the bilinear upsampling layer in the existing PSPNet model with a transpose convolution layer. ### Claim 14 (depends on 13) The background server according to claim 13, wherein replacing the bilinear upsampling layer in the existing PSPNet model with the transpose convolution layer comprises: replacing a multi-scale bilinear upsampling layer in the existing PSPNet model with a nearest neighbor pool upsampling layer and a shared transpose convolution layer. ### Claim 15 (depends on 11) The background server according to claim 11, wherein loading the reconstructed semantic segmentation model into the TensorRT engine to generate the TensorRT acceleration model comprises: training the reconstructed semantic segmentation model periodically based on image materials in a preset training material set to generate a weight file, wherein the image materials are marked with contour lines of various components of different objects; and loading a model file of the reconstructed semantic segmentation model and the weight file into the TensorRT engine to generate the TensorRT acceleration model. ### Claim 16 (depends on 11) The background server according to claim 11, wherein performing the semantic segmentation on the video frame image through the TensorRT acceleration model to generate the multi-level ROI of the video frame image comprises: acquiring a target frame image of a video to be transcoded, and generating a global feature map of the target frame image based on a feature extraction module of the TensorRT acceleration model; performing feature segmentation on the global feature map through a feature segmentation module of the TensorRT acceleration model to determine multiple components of at least one target object in the target frame image; and determining the multi-level ROI of the target frame image based on a transcoding priority corresponding to each of the components. ### Claim 17 (depends on 16) The background server according to claim 16, wherein using different transcoding rates to transcode the multi-level ROI and other regions of the target frame image respectively comprises: selecting a target-level transcoding rate sequentially in a descending order of transcoding rates; and selecting, in a descending order of transcoding priorities, a transcoding region corresponding to the target-level transcoding rate from unselected ROIs according to a transcoding area corresponding to the target-level transcoding rate. ### Claim 18 (depends on 17) The background server according to claim 17, wherein the method further comprises: adjusting the transcoding area corresponding to a transcoding rate of each level periodically according to current device performance load and line bandwidth load, and attribute information of the video to be transcoded. ### Claim 19 (depends on 16) The background server according to claim 16, wherein after determining multiple components of at least one target object in the target frame image, the method further comprises: adjusting the transcoding priority corresponding to components of each target object according to a video type of the video to be transcoded and an object type of each of the target object. ### Claim 20 A computer readable storage medium storing at least one instruction, at least one program, a code set or an instruction set, when loaded and executed by a processor, causes the processor to perform a method for transcoding a video; wherein the method comprises: reconstructing an existing semantic segmentation model based on a processing layer supported by a TensorRT engine; loading a reconstructed semantic segmentation model into the TensorRT engine to generate a TensorRT acceleration model; performing semantic segmentation on a video frame image through the TensorRT acceleration model to generate a multi-level ROI of the video frame image; and using different transcoding rates to transcode the multi-level ROI and other regions of the video frame image respectively.