# US 11,308,993 B2 · Short video synthesis method and apparatus, and device and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,993 B2 - Document ID: US11308993B2 - Status: Granted - Kind: B2 - Application number: 16/981,772 - Filed: 2019-05-24 - Published: 2021-04-01 - Granted: 2022-04-19 - Earliest claimed: 2018-05-28 - Assignees: Guangzhou Huya Information Technology Co., Ltd. - Applicants: Guangzhou Huya Information Technology Co., Ltd. - Inventors: Zihao Xu; Xiaodong Wu - Web page: https://patentgrep.com/patent/US11308993B2 ## CPC - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 10/82: using neural networks - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G11B 27/06: Cutting and rejoining; Notching, or perforating record carriers otherwise than by recording styli (record carriers with provision for splicing G11B23/20) - G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Provided are a short video synthesis method and apparatus, a device and a storage medium. The method includes: obtaining a video frame set corresponding to a to-be-processed video, where each video frame in the video frame set carries a timestamp; respectively inputting all video frames in the video frame set into an interest frame identification model to obtain a plurality of to-be-synthesized video frames, where the interest frame identification model is used for identifying a video frame meeting an interest degree condition in the input video frames; and splicing, according to the timestamp carried by the each to-be-synthesized video frame, the plurality of to-be-synthesized video frames to form a short video corresponding to the to-be-processed video. ## Claims ### Claim 1 A short video synthesis method, comprising: obtaining a video frame set corresponding to a to-be-processed video, wherein each video frame in the video frame set carries a timestamp; respectively inputting all video frames in the video frame set into an interest frame identification model to obtain a plurality of to-be-synthesized video frames, wherein the interest frame identification model is used for identifying a video frame meeting an interest degree condition in the input video frames; and splicing, according to the timestamp carried by the each to-be-synthesized video frame, the plurality of to-be-synthesized video frames to form a short video corresponding to the to-be-processed video; wherein the interest frame identification model is obtained through training a set deep learning model by using a plurality of video frame samples marked with an interest label. ### Claim 2 (depends on 1) The method of claim 1, wherein before respectively inputting all video frames in the video frame set into the interest frame identification model to obtain the plurality of to-be-synthesized video frames, the method further comprises: obtaining a plurality of video frame samples marked with an interest degree label; and training a set deep learning model by using images of the plurality of video frame samples to obtain the interest frame identification model. ### Claim 3 (depends on 2) The method of claim 2, wherein the interest degree label comprises: a label of interest, and a label of non-interest. ### Claim 4 (depends on 1) The method of claim 1, wherein before respectively inputting all video frames in the video frame set into the interest frame identification model to obtain the plurality of to-be-synthesized video frames, the method further comprises: obtaining a plurality of video frame samples which corresponds to at least one video category and is marked with an interest degree label; and training at least one set deep learning model by using the plurality of video frame samples to obtain at least one interest frame identification model respectively corresponding to the at least one video category; wherein respectively inputting all video frames in the video frame set into the interest frame identification model to obtain the plurality of to-be-synthesized video frames, comprising: respectively inputting all video frames in the video frame set into an interest frame identification model matched with a video category of the to-be-processed video to obtain a plurality of to-be-synthesized video frames. ### Claim 5 (depends on 4) The method of claim 4, wherein obtaining the plurality of video frame samples which corresponds to at least one video category and is marked with the interest degree label comprises: obtaining a plurality of video frame samples marked with the interest degree label; and inputting the plurality of video frame samples into a pre-trained video category determination model to obtain a video frame sample corresponding to at least one video category. ### Claim 6 (depends on 4) The method of claim 4, wherein the interest degree label comprises: a label of interest, and a label of non-interest. ### Claim 7 (depends on 1) The method of claim 1, wherein after the short video is obtained, the method further comprises: pushing the short video to a set video publishing platform; obtaining a short video with a network evaluation parameter exceeding a preset threshold, and determining the short video with the network evaluation parameter exceeding the preset threshold as a target short video, wherein the network evaluation parameter comprises a video score or a number of video clicks; classifying the target short video according to video categories; and splicing target short videos of a same video category to obtain at least one long video. ### Claim 8 (depends on 7) The method of claim 7, wherein after the at least one long video is obtained, the method further comprises: pushing the at least one long video to the set video publishing platform. ### Claim 9 (depends on 1) A non-transitory computer-readable storage medium, configured to store a computer program, wherein, when executed by a processor, the computer program implements the method of claim 1. ### Claim 10 A short video synthesis apparatus, comprising: a video frame set obtaining module, configured to obtain a video frame set corresponding to a to-be-processed video, wherein each video frame in the video frame set carries a timestamp; a to-be-synthesized video frame obtaining module, configured to respectively input all video frames in the video frame set into an interest frame identification model to obtain a plurality of to-be-synthesized video frames, wherein the interest frame identification model is used for identifying a video frame meeting an interest degree condition in the input video frames; and a short video forming module, configured to splice, according to the timestamp carried by the each to-be-synthesized video frame, the plurality of to-be-synthesized video frames to form a short video corresponding to the to-be-processed video; wherein the interest frame identification model is obtained through training a set deep learning model by using a plurality of video frame samples marked with an interest label. ### Claim 11 A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, when executing the computer program, the processor is configured to: obtain a video frame set corresponding to a to-be-processed video, wherein each video frame in the video frame set carries a timestamp; respectively input all video frames in the video frame set into an interest frame identification model to obtain a plurality of to-be-synthesized video frames, wherein the interest frame identification model is used for identifying a video frame meeting an interest degree condition in the input video frames; and splice, according to the timestamp carried by the each to-be-synthesized video frame, the plurality of to-be-synthesized video frames to form a short video corresponding to the to-be-processed video; wherein the interest frame identification model is obtained through training a set deep learning model by using a plurality of video frame samples marked with an interest label. ### Claim 12 (depends on 11) The computer device of claim 11, wherein before respectively inputting all video frames in the video frame set into the interest frame identification model to obtain the plurality of to-be-synthesized video frames, the processor is further configured to: obtain a plurality of video frame samples marked with an interest degree label; and train a set deep learning model by using images of the plurality of video frame samples to obtain the interest frame identification model. ### Claim 13 (depends on 12) The computer device of claim 12, wherein the interest degree label comprises: a label of interest, and a label of non-interest. ### Claim 14 (depends on 11) The computer device of claim 11, wherein before respectively inputting all video frames in the video frame set into the interest frame identification model to obtain the plurality of to-be-synthesized video frames, the processor is further configured to: obtain a plurality of video frame samples which corresponds to at least one video category and is marked with an interest degree label; and train at least one set deep learning model by using the plurality of video frame samples to obtain at least one interest frame identification model respectively corresponding to the at least one video category; wherein when respectively inputting all video frames in the video frame set into the interest frame identification model to obtain the plurality of to-be-synthesized video frames, the processor is configured to: respectively input all video frames in the video frame set into an interest frame identification model matched with a video category of the to-be-processed video to obtain a plurality of to-be-synthesized video frames. ### Claim 15 (depends on 14) The computer device of claim 14, wherein the interest degree label comprises: a label of interest, and a label of non-interest. ### Claim 16 (depends on 14) The computer device of claim 14, wherein when obtaining the plurality of video frame samples which corresponds to at least one video category and is marked with the interest degree label, the processor is configured to: obtain a plurality of video frame samples marked with the interest degree label; and input the plurality of video frame samples into a pre-trained video category determination model to obtain a video frame sample corresponding to at least one video category. ### Claim 17 (depends on 11) The computer device of claim 11, wherein after the short video is obtained, the processor is further configured to: push the short video to a set video publishing platform; obtain a short video with a network evaluation parameter exceeding a preset threshold, and determine the short video with the network evaluation parameter exceeding the preset threshold as a target short video, wherein the network evaluation parameter comprises a video score or a number of video clicks; classify the target short video according to video categories; and splice target short videos of a same video category to obtain at least one long video. ### Claim 18 (depends on 17) The computer device of claim 17, wherein after the at least one long video is obtained, the processor is further configured to: push the at least one long video to the set video publishing platform.