Systems for tracking disease progression in a patient

A method for tracking an evolution of a disease in a colon of a patient over time, the method comprising:

• obtaining video data representing a colon of a patient;
• segmenting the video data into segments representing portions of the colon;
• detecting one or more features of the colon for each segment of the segments;
• generating, from the one or more features of the colon, a feature vector representing the colon of the patient;
• and
• generating a timeline that tracks the evolution of the disease in the colon of the patient based on the feature vector.

A data processing system for processing video data representing colonoscopies of a patient to track an evolution of a disease in a colon of a patient over time, the data processing system comprising:

• an interface configured to receive video data from a camera, the video data representing a colonoscopy for a patient;
• at least one processor configured to perform operations comprising:
• obtaining the video data via the interface;
• determining a type of disease screening associated with the video data;
• performing coarse alignment of the video data over a time period based on the type of disease screening, the coarse alignment comprising:
• segmenting the video data into segments representing portions of a colon of the patient;
• and
• assigning a time period to each segment;
• performing a medium alignment of the video data based on the coarse alignment, the medium alignment comprising:
• determining a motion of the camera through a portion of the colon represented by a segment;
• and
• determining a location of the camera within the segment for a given time in the time period of the segment;
• performing a fine alignment of the video data based on the medium alignment, the fine alignment comprising:
• associating one or more features of the colon detected in the segment with a feature time stamp;
• generating, from the one or more features of the colon detected based on the fine alignment, a feature vector representing the colon of the patient;
• and
• generating a timeline comprising the feature vector and one or more other feature vectors representing the patient at respective points in the timeline, the timeline representing an evolution of a colon disease over time in the patient;
• and
• a user interface configured to present output data representing the timeline.