Back to results
US 2020/0372255 A1Published application
Video format classification and metadata injection using machine learning
AssigneeVimeo, Inc.
InventorOr Fleisher
Life of this patent
Estimating the term…
Abstract
Methods, systems, and computer program products for classifying a spatial format of a video file. The system includes one or more processors and a memory coupled to the processors. The memory stores data comprising program code that, when executed by the processors, causes the system to allow video sharing platforms to support multiple formats without asking users to manually identify the format of the video.
Claims43
A fully-automatic system for classifying the spatial format of a video file:
- a video pre-processing stage, splitting and resizing an input video file into a set of frames;
- a convolutional neural network, receiving the set of frames and predicting one or more confidence values for each of the frames respectively, wherein the confidence values are one-hot encoded confidence values signifying either 2D, spherical (equirectangular) or stereo spherical (top-bottom equirectangular) projections detected per frame;
- and
- a discriminator stage, receiving the confidence values from the convolutional neural network and determining the spatial format of video based upon a mean confidence value calculated from the sum of per-frame confidence values;
- and
- a tagging stage adds the correct metadata associated with the video file corresponding to the format of the video determined at the discriminator stage.
3 more elements
A method for classifying the format of a video file, comprising the steps of:
- splitting an input video file into a set of frames;
- generating, by a convolutional neural network, one or more confidence value predictions for each of the frames in the set respectively with respect to one or more of a two-dimensional video format, a spherical video format and/or a stereo spherical video format;
- and
- determining the format of the video file based upon a mean confidence value calculated from the confidence value predictions;
- and
- developing metadata associated with the video file corresponding to the determined format of the video.
3 more elements
A system for classifying the spatial format of a video file:
- a video pre-processing stage, splitting an input video file into a set of frames;
- a convolutional neural network, receiving the set of frames and predicting one or more confidence values for each of the frames respectively, wherein the confidence values signify one or more spatial video formats detected per frame;
- and
- a discriminator stage, receiving the per-frame confidence values from the convolutional neural network and determining the spatial format of video based upon an overall confidence value calculated from the plurality of per-frame confidence values;
- and
- an action stage that takes appropriate action with respect to the video file based upon the determined spatial format of the video, the appropriate action including meta-tagging the video file according to the determined spatial format, storing the video file according to the determined spatial format, playing the video file according to the determined spatial format, transmitting the video file according to the determined spatial format, and/or displaying the result of the determined spatial format to a user.
3 more elements
Dependencies are inferred from the claim text.