Personalized automatic video cropping

A non-transitory computer readable medium having stored thereon software instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• determining a per-frame crop score for two or more candidate crop regions in each frame of a plurality of frames of an input video, wherein the crop score is based on one or more of:
• a score based on face analysis, a score based on active speaker analysis, or an aesthetic score, and wherein the candidate crop regions each have an output aspect ratio that is different than an input aspect ratio of the input video;
• determining a respective motion cost for each candidate crop region by analyzing frame to frame motion of the candidate crop region;
• determining a minimal cost path that represents crop region locations for the input video based on the respective motion cost and the per-frame crop score of the two or more candidate crop regions;
• generating crop keyframing corresponding to the crop region locations along the minimal cost path, wherein the crop keyframing includes a start frame, an end frame, and the crop region location;
• and
• outputting a modified video based on the crop keyframing.

A system comprising:

• one or more processors coupled to a non-transitory computer readable medium having stored thereon software instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• determining a per-frame crop score for two or more candidate crop regions in each frame of a plurality of frames of an input video, wherein the crop score is based on one or more of:
• a score based on face analysis, a score based on active speaker analysis, or an aesthetic score, and wherein the candidate crop regions each have an output aspect ratio that is different than an input aspect ratio of the input video;
• determining a respective motion cost for each candidate crop region by analyzing frame to frame motion of the candidate crop region;
• determining a minimal cost path that represents crop region locations for the input video based on the respective motion cost and the per-frame crop score of the two or more candidate crop regions;
• generating crop keyframing corresponding to the crop region locations along the minimal cost path, wherein the crop keyframing includes a start frame, an end frame, and the crop region location;
• and
• outputting a modified video based on the crop keyframing.