Image stabilization using machine learning

An apparatus comprising:

• a memory;
• and
• a processor configured to:
• obtain a sequence of frames captured by a video capture device during a period of time;
• collect motion sensor measurements calculated by a motion sensor associated with the video capture device, the motion sensor measurements being calculated based on movement of the video capture device during the period of time;
• receive, at a deep learning network, the motion sensor measurements;
• based on processing the motion sensor measurements using the deep learning network, generating parameters for counteracting motion in one or more frames in the sequence of frames, the motion resulting from the movement of the video capture device during the period of time;
• and
• generate one or more adjusted frames by applying the parameters to the one or more frames in the sequence of frames, the one or more adjusted frames having a reduction in at least some of the motion in the one or more frames.

A non-transitory computer-readable storage medium comprising instructions stored therein instructions which, when executed by one or more processors, cause the one or more processors to:

• obtain a sequence of frames captured by a video capture device during a period of time;
• collect motion sensor measurements calculated by a motion sensor associated with the video capture device, the motion sensor measurements being calculated based on movement of the video capture device during the period of time;
• receive, at a deep learning network, the motion sensor measurements;
• based on processing the motion sensor measurements using the deep learning network, generating parameters for counteracting motion in one or more frames in the sequence of frames, the motion resulting from the movement of the video capture device during the period of time;
• and
• generate one or more adjusted frames by applying the parameters to adjust the one or more frames in the sequence of frames, the one or more adjusted frames having a reduction in at least some of the motion in the one or more frames.