Back to results
US 12,567,214 B2Granted
Control structures for procedural guidance systems
AssigneeLabLightAR, Inc.
InventorsStacia R. Engel, William Peria, Roger Brent
Life of this patent
Estimating the term…
Abstract
An augmented reality system includes a knowledge base and an ontology. The knowledgebase includes properties of objects, substances, and materials found in work environments, and the ontology includes settings for permitted entities, objects and materials, and properties and relations including dependency relationships among them, such that the ontology encodes procedures carried out in the work environment in a computable format.
Claims20
An augmented reality guidance system, comprising:
- an augmented reality device;
- one or more databases storing a knowledge base and an ontology;
- one or more processors;
- and
- memory including a plurality of computer-executable components that are executable by the one or more processors to perform a plurality of actions, the plurality of actions comprising:
- receiving, from the augmented reality device, information related to an operator completing at least one protocol with respect to one or more objects in an augmented reality environment, each protocol in the at least one protocol including one or more steps;
- determining based on the information and data in at least one of the ontology or the knowledge base, a next step in a protocol of the at least one protocol with respect to the one or more objects that is to be completed by the operator;
- and operator;
- presenting, via an augmented reality headset of the augmented reality device, augmenting information for the one or more objects that guides the operator in completing the next step in the protocol for the one or more objects in the augmented reality environment;
- based on a control structure of the one or more objects, determining whether the operator made an error in completing the next step in the protocol, wherein the control structure indicates permitted actions associated with the one or more objects and forbidden actions associated with the one or more objects;
- and
- in response to determining that the operator made the error in completing the next step in the protocol, updating the protocol using machine learning, data identifying the next step in the protocol, and data identifying the error.
9 more elements
One or more non-transitory computer-readable media storing computer-executable instructions that upon execution cause one or more processors to perform acts comprising:
- receiving, from an augmented reality device, images detected by a camera of the augmented reality device related to an operator completing at least one protocol with respect to one or more physical objects in a physical workspace, each protocol in the at least one protocol including one or more steps;
- determining, based on the images and data in at least one of an ontology or a knowledge base, a next step in a protocol of the at least one protocol with respect to the one or more physical objects that is to be completed by the operator;
- and operator;
- presenting, via an augmented reality headset of the augmented reality device, augmenting information for the one or more physical objects that guides the operator in completing the next step in the protocol for the one or more physical objects, the augmenting information being presented as augmentations to one or more corresponding physical object depictions of the one or more physical objects in an augmented reality environment that includes a physical workspace depiction of the physical workspace;
- based on a control structure of the one or more objects, determining whether the operator made an error in completing the next step in the protocol, wherein the control structure indicates permitted actions associated with the one or more physical objects and forbidden actions associated with the one or more physical objects;
- and
- in response to determining that the operator made the error in completing the next step in the protocol, updating the protocol using machine learning, data identifying the next step in the protocol, and data identifying the error.
4 more elements
A computer-implemented method, comprising:
- receiving, from an augmented reality device, information related to an operator completing at least one protocol with respect to one or more-virtual objects in an augmented reality environment, each protocol in the at least one protocol including one or more steps;
- determining based on the information and data in at least one of an ontology or a knowledge base, a next step in a protocol of the at least one protocol with respect to the one or more objects that is to be completed by the operator, wherein the knowledge base comprises at least one of (1) properties of at least one of objects, materials, or operations that are organized according to a given protocol of the at least one protocol, (2) settings for the given protocol of the at least one protocol and one or more given steps in the given protocol of the at least one protocol, and (3) entities that are members of multiple classes;
- and
- presenting, via an augmented reality headset of the augmented reality device, augmenting information for the one or more-virtual objects that guides the operator in completing the next step in the protocol for the one or more-virtual objects in the augmented reality environment.
1 more element
Dependencies are inferred from the claim text.