Object keypoint detection

A system for obtaining object keypoints for an object in a medical scanner, wherein the object keypoints are three dimensional (3D) coordinates with respect to the medical scanner of pre-determined object parts, and wherein the object keypoints are defined inside the object parts, the system comprising:

• a camera system for obtaining two dimensional (2D) images of the object in the medical scanner, wherein the camera system comprises:
• one or more cameras;
• a processor configured to:
• obtain scanner variables from at least one of the medical scanner or the camera system, wherein the medical scanner contains a first moving part and/or wherein the camera system contains a second moving part and wherein the scanner variables contain the position of the first moving part of the medical scanner and/or the second moving part of the camera system;
• determine object keypoint projections in 2D coordinates based in inputting the 2D images into a neural network trained to learn the spatial relationships between object keypoint projections corresponding to different parts of the object, wherein the neural network is configured to output the object keypoint projections in each one of the input images;
• and
• determine the object keypoints in 3D coordinates by triangulating the object keypoint projections with respect to the camera system based on the scanner variables.