Method and system for automatically reframing and transforming videos of different aspect ratios

A computer-based system for automatically reframing and transforming videos of different aspect ratios, the system comprising:

• a processor;
• at least one memory module storing computer program code and communicably coupled to said processor, wherein said memory module and said computer program code stored therein are configured, with said processor, to cause said computer-based system to:
• acquire an input, said input comprising a source video in different formats, information regarding reframed resolutions into which the source video is to be converted, and information regarding a genre of the source video;
• extract audio from the source video;
• identify shot change points based on visual data points and extract frames from the source video, wherein the shot change points are moments in the source video where a noticeable transition takes place, thereby denoting a change in scenes or camera perspectives, and wherein said visual data points include colour distribution, motion patterns, and keyframe dissimilarities, and wherein said visual data points are used to detect changes in content and presentation style and to accurately identify exact moments when the shot change takes place;
• process each of the frames of the source video and individually process shots of the source video;
• capture motion from each of the shots of the source video, and identify a most-focused region of each of the frames as salient frames;
• identify visually prominent regions within the source video and extract key features from the source video, and generate saliency maps that highlight distinctive areas in the source video;
• identifying significant regions of interest (ROIs) in the source video using the saliency maps;
• detect text from the source video at predetermined sampling intervals, said processor further configured to pinpoint potential text regions using edge detection, analyze pinpointed potential text regions for distinctive features, including texture and colour, classify the distinctive features as text and non-text, and create bounding boxes around detected text areas;
• generate viewports using the salient frames by:
• obtaining the most focused region of each of the salient frames at a high threshold and a low threshold;
• dividing each of the salient frames into multiple quadrants and calculating the spread of the most focused region of each of the salient frames based on quadrant occupancies;
• and
• feeding the spread and the quadrant occupancies to a series of predefined rules to determine at least one transformation required to be applied for the most focused region of each of the salient frames;
• apply said at least one transformation to automatically transform the most focused region of each of the salient frames and generate transformed frames;
• obtain output frames by cropping the transformed frames, and generate a transformed video by merging cropped frames and adding extracted audio.

A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, said computer-executable instructions, when executed by a computer-based processor, cause said processor to:

• acquire an input, said input comprising a source video in different formats, information regarding reframed resolutions into which the source video is to be converted, and information regarding a genre of the source video;
• extract audio from the source video;
• identify shot change points based on visual data points and extract frames from the source video, wherein the shot change points are moments in the source video where a noticeable transition takes place, thereby denoting a change in scenes or camera perspectives, and wherein said visual data points include colour distribution, motion patterns, and keyframe dissimilarities, and wherein said visual data points are used to detect changes in content and presentation style and to accurately identify exact moments when the shot change takes place;
• process each of the frames of the source video and individually process shots of the source video;
• capture motion from each of the shots of the source video, and identify a most-focused region of each of the frames as salient frames;
• identify visually prominent regions within the source video and extract key features from the source video, and generate saliency maps that highlight distinctive areas in the source video;
• identifying significant regions of interest (ROIs) in the source video using the saliency maps;
• detect text from the source video at predetermined sampling intervals, said processor further configured to pinpoint potential text regions using edge detection, analyze pinpointed potential text regions for distinctive features, including texture and colour, classify the distinctive features as text and non-text, and create bounding boxes around detected text areas;
• generate viewports using the salient frames by:
• obtaining the most focused region of each of the salient frames at a high threshold and a low threshold;
• dividing each of the salient frames into multiple quadrants and calculating the spread of the most focused region of each of the salient frames based on quadrant occupancies;
• and
• feeding the spread and the quadrant occupancies to a series of predefined rules to determine at least one transformation required to be applied for the most focused region of each of the salient frames;
• apply said at least one transformation to automatically transform the most focused region of each of the salient frames and generate transformed frames;
• obtain output frames by cropping the transformed frames, and generate a transformed video by merging cropped frames and adding extracted audio.