# US 12,192,540 B2 · Video processing method and apparatus, storage medium, and device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,540 B2 - Document ID: US12192540B2 - Status: Granted - Kind: B2 - Application number: 18/263,077 - Filed: 2022-01-25 - Published: 2024-04-11 - Granted: 2025-01-07 - Earliest claimed: 2021-01-26 - Assignees: Beijing Youzhuju Network Technology Co., Ltd. - Applicants: Beijing Youzhuju Network Technology Co., Ltd. - Inventors: Yuechun Jiang; Aiqing Dong; Yiming Fan; Xiaoshuai Cao; Ke Deng - Web page: https://patentgrep.com/patent/US12192540B2 ## CPC - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/2743: Video hosting of uploaded data from client - H04N 21/439: Processing of audio elementary streams - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} - H04N 21/854: Content authoring - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method includes steps described below. In response to detecting that a voice-over entry corresponding to an online video is triggered, a voice-over recording page corresponding to the online video is provided for a voice-over actor-side client; the online video is played on the voice-over recording page, and speech information of a voice-over actor collected by the voice-over actor-side client in a process of playing a target video clip which does not include a speech feature in the online video is acquired; a corresponding voice-over video is generated according to the speech information and the online video; the voice-over video is sent to an uploader-side client corresponding to the online video for review by an uploader, and a review result fed back by the uploader-side client is received; and in response to determining that the review result satisfies a preset requirement, the voice-over video is published. ## Claims ### Claim 1 A video processing method, applied to a server side, wherein the video processing method comprises: in response to detecting that a voice-over entry corresponding to an online video is triggered, providing a voice-over recording page corresponding to the online video for a voice-over actor-side client; controlling the voice-over actor-side client to play the online video on the voice-over recording page, and acquiring speech information of a voice-over actor collected by the voice-over actor-side client in a process of playing a target video clip in the online video, wherein the target video clip does not comprise a speech feature; generating a corresponding voice-over video according to the speech information and the online video; sending the voice-over video to an uploader-side client corresponding to the online video for review by an uploader, and receiving a review result fed back by the uploader-side client; and in response to determining that the review result satisfies a preset requirement, publishing the voice-over video, wherein sending the voice-over video to the uploader-side client corresponding to the online video for review by the uploader comprises: assessing a voice-over effect of the voice-over video; and sending an assessment result and the voice-over video to the uploader-side client corresponding to the online video for review by the uploader. ### Claim 2 (depends on 1) The method according to claim 1, wherein assessing the voice-over effect of the voice-over video comprises: assessing the voice-over effect of the voice-over video based on at least one of following dimensions: a matching degree between oral content and video content which correspond to a same target video clip, a matching degree between voice-over duration and video duration which correspond to a same target video clip, an overlap degree between oral content of the target video clip and lines in the online video, or whether a voice-over tone satisfies a preset tone requirement. ### Claim 3 (depends on 2) The method according to claim 2, wherein assessing the voice-over effect of the voice-over video based on the matching degree between the oral content and the video content which correspond to the same target video clip comprises: for a current target video clip, recognizing a display object that appears in the current target video clip, recognizing a described object that appears in corresponding oral content, and determining whether the display object and the described object are consistent; determining that an assessment score is high based on a determination result that the display object and the described object are consistent, or determining that an assessment score is low based on a determination result that the display object and the described object are not consistent; assessing the voice-over effect of the voice-over video based on the matching degree between the voice-over duration and the video duration which correspond to the same target video clip comprises: for a current target video clip, determining whether a difference between video duration of the current target video clip and voice-over duration corresponding to the current target video clip is within a first preset duration range; determining that an assessment score is high based on a determination result that the difference is within the first preset duration range, or determining that an assessment score is low based on a determination result that the difference is not within the first preset duration range; assessing the voice-over effect of the voice-over video based on the overlap degree between the oral content of the target video clip and the lines in the online video comprises: for a current target video clip, determining whether an overlap degree between oral content corresponding to the current target video clip and lines within a preset range of the online video is less than a preset overlap degree threshold; determining that an assessment score is high based on a determination result that the overlap degree is less than the preset overlap degree threshold, or determining that an assessment score is low based on a determination result that the overlap degree is greater than or equal to the preset overlap degree threshold, wherein the preset range comprises a second preset duration range before the current target video clip and a third preset duration range after the current target video clip; and assessing the voice-over effect of the voice-over video based on whether the voice-over tone satisfies the preset tone requirement comprises: for a current target video clip, determining whether a fluctuation of voice-over pitch corresponding to the current target video clip is within a preset fluctuation range; determining that an assessment score is high based on a determination result that the fluctuation is within the preset fluctuation range, or determining that an assessment score is low based on a determination result that the fluctuation is not within the preset fluctuation range. ### Claim 4 (depends on 2) The method according to claim 2, after assessing the voice-over effect of the voice-over video, further comprising: in response to determining that a target video clip of which a voice-over tone does not satisfy the preset tone requirement exists, adjusting the voice-over tone corresponding to the target video clip of which the voice-over tone does not satisfy the preset tone requirement by using a preset audio technology to obtain an adjusted voice-over video, and re-determining an assessment result; wherein sending the assessment result and the adjusted voice-over video to the uploader-side client corresponding to the online video for review by the uploader comprises: sending the re-determined assessment result and the adjusted voice-over video to the uploader-side client corresponding to the online video for review by the uploader. ### Claim 5 (depends on 1) The method according to claim 1, wherein the review result comprises adjustment suggestion information; and after receiving the review result fed back by the uploader-side client, the method further comprises: determining a to-be-adjusted clip in the voice-over video according to the adjustment suggestion information; controlling the voice-over actor-side client to play the to-be-adjusted clip and display adjustment suggestion information corresponding to the to-be-adjusted clip on the voice-over recording page, and acquiring rerecorded speech information of the voice-over actor which is collected by the voice-over actor-side client in the playing process; adjusting the voice-over video according to the rerecorded speech information to generate a corresponding adjusted voice-over video; and sending the adjusted voice-over video to the uploader-side client for review, and receiving a new review result fed back by the uploader-side client. ### Claim 6 (depends on 1) The method according to claim 1, before controlling the voice-over actor-side client to play the online video on the voice-over recording page, further comprising: acquiring text description information corresponding to the target video clip, wherein the text description information is obtained by recognizing video content of the target video clip through a preset machine learning model, or the text description information is provided by the uploader; and correspondingly, while controlling the voice-over actor-side client to play the target video clip on the voice-over recording page, further comprising: controlling the voice-over actor-side client to display the text description information corresponding to a current target video clip on the voice-over recording page, and instructing the voice-over actor to provide a voice-over referring to the text description information. ### Claim 7 (depends on 1) The method according to claim 1, before detecting that the voice-over entry corresponding to the online video is triggered, further comprising: receiving a voice-over adding request for the online video sent by the voice-over actor-side client, and forwarding the voice-over adding request to the uploader-side client; and in a case of receiving voice-over authorization information from the uploader-side client, opening the voice-over entry corresponding to the online video to the voice-over actor-side client. ### Claim 8 (depends on 1) The method according to claim 1, before detecting that the voice-over entry corresponding to the online video is triggered, further comprising: receiving a voice-over entry opening request for the online video sent by the uploader-side client; sending a voice-over invitation to the voice-over actor-side client according to the voice-over entry opening request; and after detecting that the voice-over actor accepts the voice-over invitation, providing the voice-over entry corresponding to the online video for the voice-over actor-side client. ### Claim 9 (depends on 1) The method according to claim 1, after publishing the voice-over video, further comprising: playing the voice-over video corresponding to the online video on an audience side. ### Claim 10 (depends on 9) The method according to claim 9, wherein playing the voice-over video corresponding to the online video on the audience side comprises: in a process of playing the online video on the audience side, in response to detecting that a preset switching condition is satisfied, switching to the voice-over video for playing; wherein the preset switching condition comprises one of: detecting that an audience-side client receives an operation that conforms to an operation manner for blind or visually impaired people; detecting that an audience-side client receives a preset speech switching command; or determining that an audience-side client detects no gaze information of a current audience within preset duration. ### Claim 11 (depends on 1) The method according to claim 1, further comprising: receiving fed-back evaluation information for a plurality of voice-over videos corresponding to the online video, wherein the evaluation information is sent by an audience side; determining tag information corresponding to the plurality of voice-over videos, respectively, according to the evaluation information; and providing tag information corresponding to a current voice-over video while presenting the current voice-over video corresponding to the online video. ### Claim 12 (depends on 1) The method according to claim 1, further comprising: receiving fed-back evaluation information for a plurality of voice-over videos corresponding to the online video, wherein the evaluation information is sent by an audience side; determining evaluation scores corresponding to the plurality of voice-over videos, respectively, according to the evaluation information; and assessing voice-over adding capabilities of voice-over actors according to the evaluation scores. ### Claim 13 A video processing method, applied to a voice-over actor-side client, wherein the video processing method comprises: in response to a trigger operation by a voice-over actor on a voice-over entry corresponding to an online video, sending trigger information to a server side, wherein the trigger information is used for instructing the server side to confirm that it is detected that the voice-over entry corresponding to the online video is triggered; in response to a voice-over recording page display instruction for the online video sent by the server side, loading and displaying a voice-over recording page corresponding to the online video; receiving video stream data corresponding to the online video sent by the server side, and playing the online video on the voice-over recording page; collecting speech information of the voice-over actor in a process of playing a target video clip in the online video, wherein the target video clip does not comprise a speech feature; and sending the collected speech information to the server side, wherein the speech information is used for instructing the server side to perform: generating a corresponding voice-over video according to the speech information and the online video, sending the voice-over video to an uploader-side client corresponding to the online video for review by an uploader, receiving a review result fed back by the uploader-side client, and in response to determining that the review result satisfies a preset requirement, publishing the voice-over video, wherein sending the voice-over video to the uploader-side client corresponding to the online video for review by the uploader comprises: assessing a voice-over effect of the voice-over video; and sending an assessment result and the voice-over video to the uploader-side client corresponding to the online video for review by the uploader. ### Claim 14 A video processing method, applied to an uploader-side client, wherein the video processing method comprises: receiving a voice-over video sent by a server side, wherein the voice-over video is generated by the server side according to speech information of a voice-over actor and an online video uploaded by an uploader, the speech information is collected by a voice-over actor-side client in a process of playing a target video clip in the online video, and in a case where the server side detects that a voice-over entry corresponding to the online video is triggered, the server side provides a voice-over recording page corresponding to the online video for the voice-over actor-side client and controls the voice-over actor-side client to play the online video on the voice-over recording page, wherein the target video clip does not comprise a speech feature; playing the voice-over video and instructing the uploader to review the voice-over video; and receiving a review result input by the uploader, and feeding back the review result to the server side, wherein the review result is used for instructing the server side to publish the voice-over video in a case where the review result satisfies a preset requirement. ### Claim 15 (depends on 1) A non-transitory computer-readable storage medium configured to store a computer program which, when executed by a processor, causes the processor to implement the method according to claim 1. ### Claim 16 (depends on 1) A computer device, comprising a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the computer program, when executed by the processor, causes the processor to implement the method according to claim 1. ### Claim 17 (depends on 15) The storage medium according to claim 15, wherein the computer program causes the processor to implement sending the voice-over video to the uploader-side client corresponding to the online video for review by the uploader by: assessing a voice-over effect of the voice-over video; and sending an assessment result and the voice-over video to the uploader-side client corresponding to the online video for review by the uploader. ### Claim 18 (depends on 17) The storage medium according to claim 17, wherein the computer program causes the processor to implement assessing the voice-over effect of the voice-over video by: assessing the voice-over effect of the voice-over video based on at least one of following dimensions: a matching degree between oral content and video content which correspond to a same target video clip, a matching degree between voice-over duration and video duration which correspond to a same target video clip, an overlap degree between oral content of the target video clip and lines in the online video, or whether a voice-over tone satisfies a preset tone requirement. ### Claim 19 (depends on 16) The computer device according to claim 16, wherein the computer program causes the processor to implement sending the voice-over video to the uploader-side client corresponding to the online video for review by the uploader by: assessing a voice-over effect of the voice-over video; and sending an assessment result and the voice-over video to the uploader-side client corresponding to the online video for review by the uploader.