Back to results
US 11,688,142 B2Granted
Automatic multi-dimensional model generation and tracking in an augmented reality environment
AssigneeInternational Business Machines Corporation
InventorsBing Zhou, Sinem Guven Kaya
Life of this patent
Estimating the term…
Abstract
Systems, computer-implemented methods, and computer program products to facilitate automatic multi-dimensional model generation and tracking in an augmented reality environment are provided. According to an embodiment, a system can comprise a processor that executes computer executable components stored in memory. The computer executable components comprise a label component that annotates a multi-dimensional point cloud representation of an object present in augmented reality data. The computer executable components further comprise a content generation component that generates a multi-dimensional model of a component of the object based on the multi-dimensional point cloud representation.
Claims20
A system, comprising:
- a processor that executes computer executable components stored in memory, the computer executable components comprising:
- a label component that annotates a multi-dimensional point cloud representation of a physical object in a physical environment represented in augmented reality data, comprising:
- identify subsets of points in the multi-dimensional point cloud representation respectively corresponding to distinct physical components of the physical object, and
- annotate the subsets of points with respective annotations identifying the distinct physical components;
- a content generation component that generates a multi-dimensional model of a physical component of the distinct physical components of the physical object based on the multi-dimensional point cloud representation;
- and
- an overlay component that:
- crops the multi-dimensional point cloud representation to generate a second multi-dimensional point cloud representation of the physical component;
- tracks, in real-time, positions and orientations of the physical component in the physical environment;
- superimposes, in the real-time, the second multi-dimensional point cloud representation onto the physical component at the positions and the orientations in the augmented reality environment;
- wherein the label component annotates the multi-dimensional point cloud representation to generate annotated image data of the physical object and to improve a model training process that uses the annotated image data as training data to train a model to detect and segment a second component representing the physical component in an augmented reality environment.
8 more elements
A computer-implemented method, comprising:
- annotating, by a system operatively coupled to a processor, a multi-dimensional point cloud representation of a physical object present in augmented reality data, comprising:
- identifying subsets of points in the multi-dimensional point cloud representation respectively corresponding to distinct physical components of the physical object, and
- annotating the subsets of points with respective annotations identifying the distinct physical components;
- generating, by the system, a multi-dimensional model of a physical component of the distinct physical components of the physical object based on the multi-dimensional point cloud representation;
- cropping, by the system, the multi-dimensional point cloud representation to generate a second multi-dimensional point cloud representation of the physical component;
- tracking, by the system, in real-time, positions and orientations of the physical component in the physical environment;
- and
- superimposing, by the system, in the real-time, the second multi-dimensional point cloud representation onto the physical component at the positions and the orientations in the augmented reality environment;
- and
- wherein the annotating comprises annotating the multi-dimensional point cloud representation to generate annotated image data of the physical object and to improve a model training process that uses the annotated image data as training data to train a model to detect and segment a second component representing the physical component in an augmented reality environment.
7 more elements
A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
- annotate a multi-dimensional point cloud representation of a physical object present in augmented reality data, comprising:
- identify subsets of points in the multi-dimensional point cloud representation respectively corresponding to distinct physical components of the physical object, and
- annotate the subsets of points with respective annotations identifying the distinct physical components;
- generate a multi-dimensional model of a physical component of the distinct physical components of the physical object based on the multi-dimensional point cloud representation;
- crop the multi-dimensional point cloud representation to generate a second multi-dimensional point cloud representation of the physical component;
- track, in real-time, positions and orientations of the physical component in the physical environment;
- and
- superimpose, in the real-time, the second multi-dimensional point cloud representation onto the physical component at the positions and the orientations in the augmented reality environment;
- and
- wherein the annotating comprises annotating the multi-dimensional point cloud representation to generate annotated image data of the physical object and to improve a model training process that uses the annotated image data as training data to train a model to detect and segment a second component representing the physical component in an augmented reality environment.
7 more elements
Dependencies are inferred from the claim text.