Method and System for Reconstructing the Three-Dimensional Surface of Tubular Organs

A method for visualising a three-dimensional internal surface of a region of interest in a lumen comprising:

• collecting image data from a camera mounted on an endoscope moving through the region of interest;
• collecting motion data in relation to the motion of the endoscope;
• collecting endoscope shape data in relation to the shape of the endoscope;
• inputting the image data into an algorithm trained to determine camera pose from input image data, and obtaining pose data of the camera relative to the lumen;
• and
• inputting the pose data, the motion data and the endoscope shape data into a model program to virtually construct a three-dimensional model of the internal surface of the region of interest.

A method for visualising a three-dimensional internal surface of a region of interest in a lumen comprising:

• collecting image data from a camera mounted on an endoscope moving through the region of interest, the image data comprising a plurality of frames;
• inputting the plurality of frames into an algorithm trained to determine camera pose and lumen curvature from input image data, and obtaining pose data of the camera relative to the lumen corresponding to each frame and image shape data corresponding to the lumen curvature;
• inputting the pose data and the image shape data into a curved lumen model program;
• outputting a plurality of three-dimensional models of the internal surface of the lumen, each model corresponding to each frame;
• combining the plurality of three-dimensional models to obtain a final three-dimensional model of the internal surface of the region of interest.

A computer-implemented system for modelling an in vivo lumen of a subject, the system comprising:

• at least one processor;
• and
• at least one computer readable storage medium storing computer readable instructions that when executed by the processor configure the system to:
• i) receive endoscope motion data, endoscope image data, and endoscope shape data obtained from within the in vivo lumen of a subject during an endoscopic examination of the lumen;
• ii) process the endoscope motion data, endoscope image data, and endoscope shape data to determine a virtual three dimensional model of the lumen, the virtual three dimensional model of the lumen being formed by a series of contiguous curved cylinder models derived on a per image frame basis of the endoscopic image data, the virtual three dimensional model of the lumen being texture mapped where possible with the endoscope image data in dependence on a field of view of the endoscope and the resulting endoscope image data available.