Back to results
US 11,900,682 B2Granted
Method and apparatus for video clip extraction, and storage medium
AssigneeBeijing Xiaomi Pinecone Electronics Co., Ltd.
InventorsJiagao Hu, Fei Wang, Pengfei Yu, Daiguo Zhou
Life of this patent
Estimating the term…
Abstract
A method for video clip extraction includes: obtaining a video, and sampling the video to obtain N video frames, wherein N is a positive integer; inputting the N video frames to a pre-trained frame feature extraction model to obtain a feature vector of each video frame in N video frames; determining scores of the N video frames based on a pre-trained scoring model; and extracting target video clips from the video based on the scores of the N video frames.
Claims17
A method for video clip extraction, comprising:
- obtaining a video, and sampling the video to obtain N video frames, wherein N is a positive integer;
- inputting the N video frames to a pre-trained frame feature extraction model to obtain a feature vector of each video frame in the N video frames;
- determining scores of the N video frames based on a pre-trained scoring model, wherein, for an i-th frame in the N video frames, feature vectors of K video frames centered on the i-th frame are input to the pre-trained scoring model, to obtain a score of the i-th frame, wherein i is a positive integer less than or equal to N, and K is a positive integer;
- and
- extracting target video clips from the video based on the scores of the N video frames;
- wherein the method further comprises:
- determining training data to train the pre-trained scoring model based on data pairs composed of first clips and second clips via a multi-frame fusion layer;
- wherein the data pairs are obtained by:
- obtaining a sample video comprising one or more of labeled video clips and unlabeled video clips, wherein each labeled video clip is labeled with a target attribute for characterizing the target video clip, or a non-target attribute for characterizing the non-target video clip;
- and
- obtaining the data pairs composed of first clips and second clips, based on the target attributes, the non-target attributes and the unlabeled video clips.
8 more elements
A device for video clip extraction, comprising:
- a processor;
- and
- a memory for storing instructions executable by the processor;
- wherein the processor is configured to:
- obtain a video and sample the video to obtain N video frames, wherein N is a positive integer;
- input the N video frames to a pre-trained frame feature extraction model to obtain a feature vector of each video frame in the N video frames;
- determine scores of the N video frames based on the a pre-trained scoring model, wherein for an i-th frame in the N video frames, feature vectors of K video frames centered on the i-th frame are input to the pre-trained scoring model, to obtain a score of the i-th frame, wherein i is a positive integer less than or equal to N, and K is a positive integer;
- and
- extract target video clips from the video based on the scores of the N video frames:,
- wherein the processor is further configured to:
- determine training data to train the pre- trained scoring model based on data pairs composed of first clips and second clips via a multi- frame fusion layer;
- wherein the data pairs are obtained by:
- obtaining a sample video comprising one or more of labeled video clips and unlabeled video clips, wherein each labeled video clip is labeled with a target attribute for characterizing the target video clip, or a non-target attribute for characterizing the non-target video clip;
- and
- obtaining the data pairs composed of first clips and second clips, based on the target attributes, the non-target attributes and the unlabeled video clips.
12 more elements
A non-transitory computer-readable storage medium having instructions stored thereon, when the instructions are executed by a processor, a method for video clip exaction is executed, the method comprising:
- obtaining a video, and sampling the video to obtain N video frames, wherein N is a positive integer;
- inputting the N video frames to a pre-trained frame feature extraction model to obtain a feature vector of each video frame in the N video frames;
- determining scores of the N video frames based on the a pre-trained scoring model, wherein, for an i-th frame in the N video frames, feature vectors of K video frames centered on the i-th frame are input to the pre-trained scoring model, to obtain a score of the i-th frame, wherein i is a positive integer less than or equal to N, and the K is a positive integer;
- and
- extracting target video clips from the video based on the scores of the N video frames:,.
- wherein the method further comprises:
- determining training data to train the pre-trained scoring model based on data pairs composed of first clips and second clips via a multi-frame fusion layer;
- wherein the data pairs are obtained by:
- obtaining a sample video comprising one or more of labeled video clips and unlabeled video clips, wherein each labeled video clip is labeled with a target attribute for characterizing the target video clip, or a non-target attribute for characterizing the non-target video clip;
- and
- obtaining the data pairs composed of first clips and second clips, based on the target attributes, the non-target attributes and the unlabeled video clips.
8 more elements
Dependencies are inferred from the claim text.