Anti-shake method for panoramic video, and portable terminal

A portable terminal, comprising:

• one or more processors;
• a memory;
• and
• one or more computer programs including a set of computer-executable instructions, where the set of computer-executable instructions are stored in the memory and are configured to be executed by the one or more processors, wherein when being executed by the one or more processors, cause the one or more processors to perform steps of an anti-shake method for a panoramic video:
• obtaining, in real time, an output video frame, a fisheye image corresponding to the output video frame, a timestamp of the pixel in the video frame, and a corresponding camera gyroscope timestamp;
• synchronizing the timestamp of the pixel in the video frame with the corresponding camera gyroscope timestamp, and calculating a rotation matrix of the camera movement in the camera gyroscope timestamp;
• smoothing the camera movement and establishing a coordinate system of a smooth trajectory;
• correcting the fisheye image distortion;
• and
• rendering the fisheye image by means of forward rendering to generate a stable video;
• wherein the step of obtaining, in real time, an output video frame, a fisheye image corresponding to the output video frame, a timestamp of the pixel in the video frame, and a corresponding camera gyroscope timestamp, specifically comprises:
• obtaining the output video frame and the fisheye image corresponding to the output video frame:
• obtaining the timestamp tkframep(x,y) of the pixel p(x,y) in the k video frame in the time axis of a video frame acquisition system, where k is a natural number;
• and
• obtaining the gyroscope timestamp tkgyrop(x,y) in the time axis of the corresponding camera gyroscope acquisition system.

An anti-shake method for a panoramic video, comprising steps of:

• S201, obtaining, in real time, an output video frame, a fisheye image corresponding to the output video frame, a timestamp of the pixel in the video frame, and a corresponding camera gyroscope timestamp;
• S202, synchronizing the timestamp of the pixel in the video frame to the corresponding camera gyroscope acquisition system, and calculating a rotation matrix of the camera movement in the timestamp of the pixel in the current video frame, as an approximate rotation matrix of the camera when collecting the current pixel;
• S203, projecting the output video frame pixels to the corresponding grid points of the spherical model, and using the approximate rotation matrix to rotate the corresponding grid points of the spherical model to the grid points in the sensor coordinate system;
• using a fisheye image distortion correction method to establish a relationship between the grid points in the sensor coordinate system and the pixels in the fisheye image to obtain approximate reverse-mapping pixels;
• S204, calculating the camera gyroscope timestamp of the approximate reverse-mapping pixels in the fisheye image, obtaining the accurate rotation matrix of the camera when collecting the current pixel, and using the accurate rotation matrix to rotate the corresponding grid points of spherical model again to obtain the second grid points in the sensor coordinate system;
• and
• S205, using the fisheye image distortion correction method to establish the relationship between the second grid points in the sensor coordinate system and the pixels in the fisheye image to obtain accurate pixels in the fisheye image, and using a mapping relationship between the output video frame pixels and the accurate pixels in the fisheye image, rendering the fisheye image in reverse rendering to generate a stable video.