Methods and systems for temporal resampling for multi-task machine vision
AssigneeAlibaba Innovation Private Limited
InventorsShurun Wang, Zhao Wang, Yan Ye, Shiqi Wang
Life of this patent
Estimating the term…
Abstract
A method for temporal resampling for multi-task machine vision is provided. The method includes receiving a bitstream of a video sequence after temporal resampling; and constructing a target frame from the bitstream using a frame construction model.
Claims17
A method for temporal resampling for multi-task machine vision, comprising:
- receiving a bitstream of a video sequence after temporal resampling;
- and
- constructing a target frame from the bitstream using a frame construction model, wherein an input of the frame construction model is a channel-wise concatenation of an average residual of the target frame and a prediction of the target frame, the average residual of the target frame is obtained based on resampled frames and a resampling factor.
A method for temporal resampling for multi-task machine vision, comprising:
- receiving a bitstream of a video sequence after temporal resampling;
- and
- constructing a target frame from the bitstream using a frame construction model;
- wherein the frame construction model is trained with feature maps, the feature maps being extracted from corresponding one or more machine analysis models for the target frame and the constructed frame of the target frame, respectively, and one or more loss functions of corresponding one or more machine analysis models are obtained with label information.
An apparatus for temporal resampling for multi-task machine vision, the apparatus comprising:
- a memory figured to store instructions;
- and
- one or more processors configured to execute the instructions to cause the apparatus to perform:
- receiving a bitstream of a video sequence after temporal resampling;
- and
- constructing a target frame from the bitstream using a frame construction model wherein an input of the frame construction model is a channel-wise concatenation of an average residual of the target frame and a prediction of the target frame, the average residual of the target frame is obtained based on resampled frames and a resampling factor.
An apparatus for temporal resampling for multi-task machine vision, the apparatus comprising:
- a memory configured to store instructions;
- and
- one or more processors configured to execute the instructions to cause the apparatus to perform:
- receiving a bitstream of a video sequence after temporal resampling;
- and
- constructing a target frame from the bitstream using a frame construction model;
- wherein the frame construction model is trained with feature maps, the feature maps being extracted from corresponding one or more machine analysis models for the target frame and the constructed frame of the target frame, respectively;
- and one or more loss functions of corresponding one or more machine analysis models are obtained with label information.
A method for temporal resampling for multi-task machine vision, comprising:
- receiving a video sequence;
- determining a moving complexity of the video sequence, wherein determining the moving complexity further comprises:
- calculating an average mean absolute difference (MAD) for a testing point with a range;
- sampling all the testing points across the video sequence equally with an interval;
- and
- obtaining the moving complexity of the video sequence based on the average MAD and the interval;
- and
- determining whether to perform temporal resampling based on the moving complexity.
Dependencies are inferred from the claim text.