Methods and systems for robotic instrument tool tracking with adaptive fusion of kinematics information and image information

A robotic medical system comprising:

• a robotic instrument coupled to a robotic arm to be manipulated and generate raw kinematics information;
• a camera to capture video image information of a portion of the robotic instrument;
• and
• a tool tracking sub-system to receive the raw kinematics information and the video image information of the robotic instrument, the tool tracking sub-system to generate corrected kinematics information for the robotic instrument by adaptively fusing the raw kinematics information and the video image information together, wherein the adaptive fusing of the kinematics information and the video image information together includes synthesizing an image of a model of the robotic instrument in response to the kinematics information, matching the image of the model to the video image information to generate match results, filtering the match results, and forming a plurality of states of the pose information for the robotic instrument in response to the filtered match results and kinematics information.