Anomaly tracking system and method using enterprise digital twins based mixed reality

A method for anomaly detection, the method comprising:

• tracking, on a backend computer system having a processor, an anomaly of an actual object in a 3D scene to generate a mixed reality environment that visualizes the anomaly of the actual object in the mixed reality environment;
• training, by the processor, at least two deep learning models using at least one benchmark dataset to generate at least two sets of labels for one or more objects in the 3D scene;
• performing, by the processor, recursive cluster segmentation of the one or more objects in the 3D scene to generate at least two sets of histograms of cluster labels for the at least two trained deep learning models;
• merging, by the processor, the sets of histograms of cluster labels for the at least two trained deep learning models for each cluster to generate a vector for each cluster;
• training, by the processor, a machine learning model based on the merged sets of labels from the trained deep learning models which reduce a complexity of a point cloud of the 3D scene by representing raw RGB and XYZ data of the point cloud in a histogram/distribution of labels of each 3D point;
• and
• performing, by the processor using the machine learning model, inference to identify the anomaly in an object of the one or more objects in the 3D scene.