Information processing system, method and program

An information processing method, performed by at least one processor and comprising:

• acquiring a captured image captured during flight by an imaging device mounted on an airplane;
• acquiring flight information detected using a sensor mounted on the airplane;
• acquiring motion information in a time series of the airplane or the imaging device by analyzing a plurality of the captured images continuously captured in a time series and the flight information continuously acquired in a time series;
• specifying the flight information acquired simultaneously with or nearly simultaneously with the captured image by matching motion information acquired by analyzing the captured images and motion information acquired by analyzing the flight information to each other;
• acquiring, by correcting the captured image on a basis of information acquired using the flight information, a correction image in which a predetermined direction in a state where the image is erected and a reference direction in the image are matched to each other;
• acquiring direction information in the captured image with the earth as a reference, on a basis of information on a posture of the airplane or the imaging device acquired as the flight information;
• acquiring, by correcting the captured image on a basis of the direction information, the correction image in which the predetermined direction in the state where the image is erected and the reference direction in the image are matched to each other;
• acquiring the direction information on a basis of the flight information acquired simultaneously with or nearly simultaneously with the captured image;
• acquiring the direction information on a basis of specified flight information;
• specifying the flight information acquired simultaneously with or nearly simultaneously with the captured image by interpolating, based on a machine learning model or a statistical model trained using previously acquired flight information, a first sequence of a parameter included in motion information acquired by analyzing the captured images and a second sequence of the parameter included in motion information acquired by analyzing the flight information;
• generating a sequence alignment representing correspondence between the sequences by comparing the interpolated first sequence and the second sequence;
• and
• performing matching based on the generated sequence alignment.

A non-transitory computer readable medium storing a computer program that when executed by at least one processor, causes the at least one processor to:

• acquire a captured image captured during flight by an imaging device mounted on an airplane;
• acquire flight information detected using a sensor mounted on the airplane;
• acquire motion information in a time series of the airplane or the imaging device by analyzing a plurality of the captured images continuously captured in a time series and the flight information continuously acquired in a time series;
• specify the flight information acquired simultaneously with or nearly simultaneously with the captured image by matching motion information acquired by analyzing the captured images and motion information acquired by analyzing the flight information to each other;
• acquire, by correcting the captured image on a basis of information acquired using the flight information, a correction image in which a predetermined direction in a state where the image is erected and a reference direction in the image are matched to each other;
• acquire direction information in the captured image with the earth as a reference, on a basis of information on a posture of the airplane or the imaging device acquired as the flight information;
• acquire, by correcting the captured image on a basis of the direction information, the correction image in which the predetermined direction in the state where the image is erected and the reference direction in the image are matched to each other;
• acquire the direction information on a basis of the flight information acquired simultaneously with or nearly simultaneously with the captured image;
• acquire the direction information on a basis of specified flight information;
• specify the flight information acquired simultaneously with or nearly simultaneously with the captured image by interpolating, based on a machine learning model or a statistical model trained using previously acquired flight information, a first sequence of a parameter included in motion information acquired by analyzing the captured images and a second sequence of the parameter included in motion information acquired by analyzing the flight information;
• generate a sequence alignment representing correspondence between the sequences by comparing the interpolated first sequence and the second sequence;
• and
• perform matching based on the generated sequence alignment.