Method and system of determining operation pathway based on image matching

A non-transitory computer-readable storage medium that includes a set of instructions which, in response to execution by a processor, cause the processor to perform a method to determine an operation pathway for a patient, wherein the method comprises:

• constructing a three-dimensional model based on a medical image scan of the patient;
• obtaining image information of the patient with a set of two-dimensional optical devices;
• selecting a first set of two-dimensional feature points associated with the three-dimensional model;
• selecting a second set of two-dimensional feature points associated with the image information of the patient;
• transforming the first set of two-dimensional feature points to a first set of three-dimensional feature points;
• transforming the second set of two-dimensional feature points to a second set of three-dimensional feature points;
• matching between the first set of three-dimensional feature points and the second set of three-dimensional feature points to determine a relationship that aligns the first set of three-dimensional feature points and the second set of three-dimensional feature points;
• transforming coordinates from a first coordinate system associated with the three-dimensional model to a second coordinate system associated with the set of two-dimensional optical devices based on the relationship;
• and
• determining the operation pathway in a third coordinate system associated with a robotic arm based on the transformed coordinates in the second coordinate system.

A system to determine an operation pathway for a patient, comprising:

• a processor;
• and
• a non-transitory computer-readable medium having stored thereon program code that, upon being executed by the processor, causes the processor to:
• construct a three-dimensional model based on a medical image scan of the patient;
• obtain image information of the patient with a set of two-dimensional optical devices;
• select a first set of two-dimensional feature points associated with the three-dimensional model;
• select a second set of two-dimensional feature points associated with the image information of the patient;
• transform the first set of two-dimensional feature points to a first set of three-dimensional feature points;
• transform the second set of two-dimensional feature points to a second set of three-dimensional feature points;
• match between the first set of three-dimensional feature points and the second set of three-dimensional feature points to determine a relationship that aligns the first set of three-dimensional feature points and the second set of three-dimensional feature points;
• transform coordinates from a first coordinate system associated with the three-dimensional model to a second coordinate system associated with the set of two-dimensional optical devices based on the relationship;
• and
• determine the operation pathway in a third coordinate system associated with a robotic arm based on the transformed coordinates in the second coordinate system.