Methodology and apparatus for generating high fidelity zoom for mobile video

A method comprising:

• capturing, by a camera, a video stream at a capture frame rate;
• in order to generate a composite image:
• identifying, in a frame of the video stream captured by the camera, a motion characteristic associated with moving objects in a scene while the camera captured a sliding window of the video stream, and
• for a plurality of frames in the sliding window, controlling, by processing circuitry, a weight of blending of the frame based on the identified motion characteristic to enable the composite image to be generated according to the controlled weights of blending of the plurality of frames in the sliding window
• including, by the processing circuitry, at least two frames of the video stream in the sliding window that contains a number N of consecutive frames of the video stream;
• determining, by the processing circuitry, camera motion information associated with motion of the camera while the camera captured the video stream;
• for each frame in one or more frames of the video stream:
• analyzing, by the processing circuitry, an amount of warp of the frame based on the camera motion information, and
• generating, by the processing circuitry, a set of warp parameters indicating the amount of warp of the frame;
• for frames in the sliding window that includes the frame:
• generating, by the processing circuitry, a warp interpolated frame by applying the warp parameters and an image interpolation technique to the frame;
• generating the composite image by:
• blending the warp interpolated frames corresponding to the sliding window into a single-blended frame according to the controlled weight of blending of each frame in the sliding window;
• and
• producing an output video stream including the composite image generated from the sliding window and a subsequent composite image generated from a subsequent sliding window that includes a subsequent N consecutive frames of the video stream.

A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that, when executed, causes at least one processing device to:

• in order to generate a composite image:
• identify, in a frame of a video stream captured by a camera, a motion characteristic associated with moving objects in a scene while the camera captured a sliding window of the video stream;
• and
• for a plurality of frames in the sliding window, control a weight of blending of the frame based on the identified motion characteristic to enable the composite image to be generated according to the controlled weights of blending of the plurality of frames;
• include at least two frames of the video stream in the sliding window that contains a number N of consecutive frames of the video stream, wherein the video stream is captured by a camera at a capture frame rate;
• determine camera motion information associated with motion of the camera while the camera captured the video stream;
• for each frame in one or more frames of the video stream:
• analyze an amount of warp of the frame based on the camera motion information, and
• generate a set of warp parameters indicating the amount of warp of the frame;
• for frames in the sliding window that includes the frame:
• generate a warp interpolated frame by applying the warp parameters and an image interpolation technique to the frame;
• generate the composite image by:
• blending the warp interpolated frames corresponding to the sliding window into a single-blended frame according to the controlled weight of blending of each frame in the sliding window;
• and
• produce an output video stream including the composite image generated from the sliding window and a subsequent composite image generated from a subsequent sliding window that includes a subsequent N consecutive frames of the video stream.