Back to results
US 12,755,420 B2Granted
Human-robot collaboration for 3D functional mapping
AssigneeIntel Corporation
InventorDavid Gonzalez Aguirre
Life of this patent
Estimating the term…
Abstract
Various aspects of techniques, systems, and use cases may be used for human-robot collaboration for three-dimensional (3D) functional mapping. An example technique may include receiving identification of a direction or location based on a user gaze identified via an extended reality device, causing environmental data of an environment to be captured using a sensor of a robotic device corresponding to the direction or location based on receiving the identification, and detecting, within the environmental data, at least one physical feature of the environment. The example technique may include determining, from a user input, an annotation to apply to the at least one physical feature, and labeling the at least one physical feature with the annotation.
Claims17
At least one machine readable medium including instructions, which when executed by processing circuitry, cause the processing circuitry to perform operations comprising:
- receiving identification of a direction or location based on a user gaze identified via an extended reality device;
- causing environmental data of an environment to be captured using a sensor of a moveable robotic device by outputting a control signal to move the moveable robotic device, the environmental data corresponding to the direction or location based on receiving the identification, wherein the sensor is registered to the extended reality device in seven dimensions including six degrees of freedom comprising x, y, z position and pitch, yaw, roll orientation, and time, for spatial and temporal alignment between the sensor of the moveable robotic device and the extended reality device;
- detecting, within the environmental data, at least one physical feature of the environment;
- determining, from a user input, an annotation to apply to the at least one physical feature;
- labeling the at least one physical feature with the annotation using a machine learning trained model to determine that the annotation applies to the at least one physical feature;
- and
- causing an indication to be displayed in the extended reality device, based on an output of the machine learning trained model, the indication suggesting that the annotation applies to the at least one physical feature, and receiving a user confirmation of the applicability of the annotation to the at least one physical feature.
4 more elements
A system comprising:
- an extended reality device including a display, the extended reality device to output an indication of a direction or location based on a detected orientation or user gaze;
- and
- a moveable robotic device including:
- a sensor to capture environmental data of an environment corresponding to the direction or location after the moveable robotic device receives a control signal to move the moveable robotic device, wherein the sensor is registered to the extended reality device in seven dimensions including six degrees of freedom comprising x, y, z position and pitch, yaw, roll orientation, and time, for spatial and temporal alignment between the sensor of the moveable robotic device and the extended reality device;
- processing circuitry;
- and
- memory, including instructions, which when executed by the processing circuitry, cause the processing circuitry to perform operations to:
- receive an identification of the direction or location from the extended reality device;
- detect, within the environmental data, at least one physical feature of the environment;
- determine, from a user input, an annotation to apply to the at least one physical feature;
- and
- label the at least one physical feature with the annotation using a machine learning trained model to determine that the annotation applies to the at least one physical feature;
- and
- cause an indication to be displayed in the extended reality device, based on an output of the machine learning trained model, the indication suggesting that the annotation applies to the at least one physical feature, and receiving a user confirmation of the applicability of the annotation to the at least one physical feature.
11 more elements
An apparatus comprising:
- means for receiving identification of a direction or location based on a user gaze identified via an extended reality device;
- means for capturing environmental data of an environment by outputting a control signal to move a moveable robotic device, the environmental data corresponding to the direction or location based on receiving the identification, wherein a sensor of the moveable robotic device is registered to the extended reality device in seven dimensions including six degrees of freedom comprising x, y, z position and pitch, yaw, roll orientation, and time, for spatial and temporal alignment between the sensor of the moveable robotic device and the extended reality device;
- means for detecting, within the environmental data, at least one physical feature of the environment;
- means for determining, from a user input, an annotation to apply to the at least one physical feature;
- and
- means for labeling the at least one physical feature with the annotation using a machine learning trained model to determine that the annotation applies to the at least one physical feature;
- and
- means for causing an indication to be displayed in the extended reality device, based on an output of the machine learning trained model, the indication suggesting that the annotation applies to the at least one physical feature, and receiving a user confirmation of the applicability of the annotation to the at least one physical feature.
5 more elements
Dependencies are inferred from the claim text.