Automated determination of a canonical pose of a 3d objects and superimposition of 3d objects using deep learning

A computer-implemented method for automated superimposition of a first 3D object represented by a first 3D data set and a second 3D object represented by a second 3D data set, the method comprising:

• a processor of a computer providing one or more first blocks of voxels of a first voxel representation of the first 3D object associated with a first coordinate system and one or more second blocks of voxels of a second voxel representation of the second 3D object associated with a second coordinate system to the input of a first 3D deep neural network, the first 3D deep neural network being trained to generate canonical pose information associated with a canonical coordinate system defined relative to a position of part of the 3D dental structure;
• the processor receiving first and second canonical pose information from the output of the 3D deep neural network, the first canonical pose information comprising for each voxel of the one or more first blocks a prediction of a first position of the voxel in the canonical coordinate system;
• and, the second canonical pose information comprising for each voxel of the one or more second blocks a prediction of a second position of the voxel in the canonical coordinate system, the first and second position being defined by first and second canonical coordinates respectively;
• the processor using the first canonical pose information to determine a first orientation and scale of the axes and first position of the origin of the axes in the first coordinate system and using the second canonical pose information to determine a second orientation and scale of the axes and a second position of the origin of the axes of the canonical coordinate system in the second coordinate system;
• the processor using the first orientation, scale and the first position to determine first transformation parameters, including first rotation, translation and/or scaling parameters, for transforming coordinates of the first coordinate system into coordinates of the canonical coordinate system;
• and, using the second orientation, scale and the second position to determine second transformation parameters, including second rotation, translation and/or scaling parameters, for transforming coordinates of the second coordinate system into canonical coordinates;
• and,
• the processor determining a superimposition of the first 3D object and the second 3D object, the determining including using the first and second transformation parameters to form a first and second canonical representation of the first and second 3D dental structure respectively.

A computer-implemented method for training a 3D deep neural network to automatically determine a canonical pose of a 3D dental structure represented by a 3D data set, comprising:

• receiving training data and associated target data, the training data including a voxel representation of a 3D object and the target data including canonical coordinate values of a canonical coordinate system for each voxel of the voxel representation, wherein the canonical coordinate system is a predetermined coordinate system defined relative to a position of part of the 3D object;
• selecting one or more blocks of voxels, the one or more blocks representing one or more subsamples of the voxel representation of a predetermined size and applying a random 3D rotation to the one or more subsamples;
• applying the same rotation to the target data;
• offering the one or more blocks to the input of the 3D deep neural network and the 3D deep neural network predicting for each voxel of the one or more blocks canonical coordinates of the canonical coordinate system;
• and
• optimising values of the network parameters of the 3D deep neural network by minimizing a loss function representing a deviation between the coordinate values predicted by the 3D deep neural network and the canonical coordinates associated with the target data.

A computer system adapted for automatically determining a canonical pose of a 3D object represented by a 3D data set, comprising:

• a computer readable storage medium having computer readable program code embodied therewith, the program code including at least one trained 3D deep neural network, and at least one processor coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the at least one processor is configured to perform executable operations comprising:
• providing one or more blocks of voxels of a voxel representation of the 3D object associated with a first coordinate system to the input of a first 3D deep neural network, the first 3D neural network being trained to generate canonical pose information associated with a canonical coordinate system defined relative to a position of part of the 3D object;
• receiving canonical pose information from the output of the first 3D deep neural network, the canonical pose information comprising for each voxel of the one or more blocks a prediction of a position of the voxel in the canonical coordinate system, the position being defined by canonical coordinates;
• using the canonical coordinates to determine an orientation and scale of the axes of the canonical coordinate system and a position of the origin of the canonical coordinate system relative to the axis and the origin of the first 3D coordinate system and using the orientation, scale and the position to determine transformation parameters, including, rotation, translation and/or scaling parameters, for transforming coordinates of the first coordinate system into canonical coordinates;
• and,
• determining a canonical representation of the 3D object, the determining including applying the transformation parameters to coordinates of the voxels of the voxel representation or the 3D data set used for determining the voxel representation.

A computer system adapted for automated superimposition of a first 3D object represented by a first 3D data set and a second 3D object a represented by a second 3D data set, comprising:

• a computer readable storage medium having computer readable program code embodied therewith, the program code including at least one trained 3D deep neural network, and at least one processor coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the at least one processor is configured to perform executable operations comprising:
• providing one or more first blocks of voxels of a first voxel representation of the first 3D dental structure associated with a first coordinate system and one or more second blocks of voxels of a second voxel representation of the second 3D dental structure associated with a second coordinate system to the input of a 3D deep neural network;
• the 3D deep neural network being trained to generate canonical pose information associated with a canonical coordinate system defined relative to a position of part of the 3D object;
• receiving first and second canonical pose information from the output of the 3D deep neural network, the first canonical pose information comprising for each voxel of the one or more first blocks a prediction of a first position of the voxel in the canonical coordinate system;
• and, the second canonical pose information comprising for each voxel of the one or more second blocks a prediction of a second position of the voxel in the canonical coordinate system, the first and second position being defined by first and second canonical coordinates respectively;
• using the first canonical pose information to determine a first orientation and scale of the axes and first position of the origin of the axes in the first coordinate system and using the second canonical pose information to determine a second orientation and scale of the axes and a second position of the origin of the axes of the canonical coordinate system in the second coordinate system;
• using the first orientation and the first position to determine first transformation parameters, including first rotation, translation and/or scaling parameters, for transforming coordinates of the first coordinate system into coordinates of the canonical coordinate system;
• and, using the second orientation and the second position to determine second transformation parameters, including second rotation, translation and/or scaling parameters, for transforming coordinates of the second coordinate system into canonical coordinates;
• and,
• determining a superimposition of the first 3D object and the second 3D object, the determining including using the first and second transformation parameters to form a first and second canonical representation of the first and second 3D object respectively.