Enhanced accessibility in mixed reality experience for collaboration tools
AssigneeMicrosoft Technology Licensing, LLC
InventorsRajesh Rangarajan, Shen Vikul Chauhan
Life of this patent
Estimating the term…
Abstract
Described herein is are systems and methods for interpreting gesture(s) and/or sign(s) using a machine-learned model. Information regarding gesture(s) and/or sign(s) is received from a first user. The information can be received via a mixed reality device of the first user and/or a second user. Probabilities that the gesture(s) or sign(s) have particular meanings are calculated using a machine-trained model. The gesture(s) and/or sign(s) are interpreted in accordance with the calculated probabilities. Information regarding the interpreted gesture(s) and/or sign(s) are provided (e.g., displayed as visual text and/or an audible output) to the second user.
Claims20
A system comprising:
- a processor;
- and
- a memory having computer-executable instructions stored thereupon which, when executed by the processor, cause the system to:
- receive information regarding a sequence of gestures or signs from a first user, the information received via a mixed reality device, the sequence including a first gesture or sign and a second gesture or sign, wherein the first gesture or sign occurs in the sequence before the second gesture or sign;
- calculate probabilities that the second gesture or sign has particular meanings using a machine-trained model, the probabilities being based at least on the first gesture or sign that occurs in the sequence before the second gesture or sign;
- interpret the second gesture or the sign in accordance with the calculated probabilities;
- and
- provide information regarding the interpreted second gesture or sign to a second user.
A method comprising:
- receiving information regarding a sequence of gestures or signs from a first user, the information received via a mixed reality device, the sequence including a first gesture or sign and a second gesture or sign, wherein the first gesture or sign occurs in the sequence before the second gesture or sign;
- based at least on the first gesture or sign, calculating probabilities that the second gesture or sign has particular meanings using a machine-trained model;
- interpreting the second gesture or the sign in accordance with the calculated probabilities;
- and
- providing information regarding the interpreted second gesture or sign to a second user.
A computer storage media storing computer-readable instructions that, when executed, cause a computing device to:
- receive information regarding a particular gesture or sign from a first user, the information received via a mixed reality device, wherein at least one other gesture or sign is received sequentially with the particular gesture or sign;
- calculate probabilities that the particular gesture or sign has particular meanings using a machine-trained model based at least on the at least one other gesture or sign received sequentially with the particular gesture or sign;
- interpret the particular gesture or sign in accordance with the calculated probabilities;
- and
- output information regarding the interpreted particular gesture or sign.
Dependencies are inferred from the claim text.