# US 2025/0095302 A1 · Wearable Electronic Devices And Extended Reality Systems Including Neuromuscular Sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095302 A1
- Document ID: US20250095302A1
- Status: Published application
- Kind: A1
- Application number: 18/890,622
- Filed: 2024-09-19
- Published: 2025-03-20
- Earliest claimed: 2018-01-25
- Assignees: Meta Platforms Technologies, LLC
- Applicants: Meta Platforms Technologies, LLC
- Inventors: Daniel Wetmore
- Web page: https://patentgrep.com/patent/US20250095302A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- A61B 5/1107: {Measuring contraction of parts of the body, e.g. organ or muscle}
- A61B 5/22: Ergometry; Measuring muscular strength or the force of a muscular blow
- A61B 5/4519: {Muscles (measuring muscular strength A61B5/224; using electromyography [EMG] A61B5/389)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
- G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
- G16H 40/63: for local operation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

The disclosed system for interacting with objects in an extended reality (XR) environment generated by an XR system may include (1) neuromuscular sensors configured to sense neuromuscular signals from a wrist of a user and (2) at least one computer processor programmed to (a) determine, based at least in part on the sensed neuromuscular signals, information relating to an interaction of the user with an object in the XR environment and (b) instruct the XR system to, based on the determined information relating to the interaction of the user with the object, augment the interaction of the user with the object in the XR environment. Other embodiments of this aspect include corresponding apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A non-transitory, computer-readable storage medium including executable instructions that, when executed by one or more processors, cause the one or more processors to:
while a head-wearable device and a wrist-wearable device are communicatively coupled and worn by a user:
obtain speech data, based on a speech input from the user, at a microphone of the head-wearable device;
determine text based on the speech data;
obtain neuromuscular data, based on a neuromuscular input from the user, at a neuromuscular sensor of the wrist-wearable device;
cause the text to be modified based on the neuromuscular data to produce modified text; and
cause the modified text to be presented to the user.

### Claim 22 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
obtain additional neuromuscular data, based on an additional neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device;
cause the text to be modified based on the additional neuromuscular data to produce additional modified text; and
cause the additional modified text to be presented to the user.

### Claim 23 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
obtain second speech data, based on a second speech input from the user, at the microphone of the head-wearable device;
determine second text based on the second speech data;
obtain second neuromuscular data, based on a second neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device;
cause the second text to be modified based on the second neuromuscular data to produce second modified text; and
cause the second modified text to be presented to the user.

### Claim 24 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
in response to obtaining fourth neuromuscular data, based on fourth neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device, cause the modified text to be sent to another user.

### Claim 25 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device, the wrist-wearable device, and another wrist-wearable device are communicatively coupled and worn by the user:
obtain other neuromuscular data, based on another neuromuscular input from the user, at a neuromuscular sensor of the other wrist-wearable device;
cause the text to be modified based on the other neuromuscular data to produce other modified text; and
cause the other modified text to be presented to the user, wherein the wrist-wearable device is worn on a first wrist of the user and the other wrist-wearable device is worn on a second wrist of the user.

### Claim 26 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device, the wrist-wearable device, and another wrist-wearable device are communicatively coupled and worn by the user:
in response to determining the text based on the speech data, predict an error in the text, based on one or more trained statistical models; and
present a suggested correction, based on one or more trained statistical models, to the user, wherein:
the neuromuscular data, based on the neuromuscular input from the user, indicates that the user accepts the suggested correction; and
causing the text to be modified based on the neuromuscular data to produce the modified text includes making the suggested correction to the text.

### Claim 27 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein:
the speech input from the user is a plurality of words spoken by the user; and
determining the text based on the speech data includes converting the plurality of words spoken by the user to a textual representation of the plurality of words.

### Claim 28 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein:
the neuromuscular sensor of the wrist-wearable device is an electromyography (EMG) sensor; and
the neuromuscular data is EMG data.

### Claim 29 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein the neuromuscular input from the user is one or more of one or more typing actions, one or more tapping gestures, one or more writing gestures, one or more drawing gestures, one or more finger gestures, and one or more one-handed gestures.

### Claim 30 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein causing the text to be modified based on the neuromuscular data includes at least one of adding one or more characters to the text, removing one or more characters from the text, changing one or more characters of the text, and moving one or more characters of the text.

### Claim 31 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein causing the text to be presented to the user includes at least one of:
presenting a visual representation of the text at a display of one or more of the head-wearable device, the wrist wearable device, and another device communicatively coupled to the head-wearable device and the wrist-wearable device; and
presenting an audio representation of the text at a speaker of one or more of the head-wearable device, the wrist wearable device, and the other device.

### Claim 32 (depends on 21)

The non-transitory, computer-readable storage medium of claim 21, wherein:
the head-wearable device is at least one of a pair of smart glasses and an extended-reality (XR) headset; and
the wrist-wearable device is a smart watch.

### Claim 33

A method, the method comprising:
while a head-wearable device and a wrist-wearable device are communicatively coupled and worn by a user:
obtaining speech data, based on a speech input from the user, at a microphone of the head-wearable device;
determining text based on the speech data;
obtaining neuromuscular data, based on a neuromuscular input from the user, at a neuromuscular sensor of the wrist-wearable device;
causing the text to be modified based on the neuromuscular data; and
causing the text to be presented to the user.

### Claim 34 (depends on 33)

The method of claim 33, the method further comprising:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
obtaining additional neuromuscular data, based on an additional neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device;
causing the text to be modified based on the additional neuromuscular data to produce additional modified text; and
causing the additional modified text to be presented to the user.

### Claim 35 (depends on 33)

The method of claim 33, the method further comprising:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
obtaining second speech data, based on a second speech input from the user, at the microphone of the head-wearable device;
determining second text based on the second speech data;
obtaining second neuromuscular data, based on a second neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device;
causing the second text to be modified based on the second neuromuscular data to produce second modified text; and
causing the second modified text to be presented to the user.

### Claim 36 (depends on 33)

The method of claim 33, the method further comprising:
while the head-wearable device, the wrist-wearable device, and another wrist-wearable device are communicatively coupled and worn by the user:
obtaining other neuromuscular data, based on another neuromuscular input from the user, at a neuromuscular sensor of the other wrist-wearable device;
causing the text to be modified based on the other neuromuscular data to produce other modified text; and
causing the other modified text to be presented to the user, wherein the wrist-wearable device is worn on a first wrist of the user and the other wrist-wearable device is worn on a second wrist of the user.

### Claim 37

An extended-reality (XR) system including:
a head-wearable device;
a wrist-wearable device communicatively coupled to the head-wearable device;
one or more processors;
one or more memory devices including medium including executable instructions that, when executed by the one or more processors, cause the one or more processors to:
while the head-wearable device and the wrist-wearable device are worn by a user:
obtain speech data, based on a speech input from the user, at a microphone of the head-wearable device;
determine text based on the speech data;
obtain neuromuscular data, based on a neuromuscular input from the user, at a neuromuscular sensor of the wrist-wearable device;
cause the text to be modified based on the neuromuscular data; and
cause the text to be presented to the user.

### Claim 38 (depends on 37)

The XR system of claim 37, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
obtain additional neuromuscular data, based on an additional neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device;
cause the text to be modified based on the additional neuromuscular data to produce additional modified text; and
cause the additional modified text to be presented to the user.

### Claim 39 (depends on 37)

The XR system of claim 37, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device and the wrist-wearable device are communicatively coupled and worn by the user:
obtain second speech data, based on a second speech input from the user, at the microphone of the head-wearable device;
determine second text based on the second speech data;
obtain second neuromuscular data, based on a second neuromuscular input from the user, at the neuromuscular sensor of the wrist-wearable device;
cause the second text to be modified based on the second neuromuscular data to produce second modified text; and
cause the second modified text to be presented to the user.

### Claim 40 (depends on 37)

The XR system of claim 37, wherein the executable instructions further cause the one or more processors to:
while the head-wearable device, the wrist-wearable device, and another wrist-wearable device are communicatively coupled and worn by the user:
obtain other neuromuscular data, based on another neuromuscular input from the user, at a neuromuscular sensor of the other wrist-wearable device;
cause the text to be modified based on the other neuromuscular data to produce other modified text; and
cause the other modified text to be presented to the user, wherein the wrist-wearable device is worn on a first wrist of the user and the other wrist-wearable device is worn on a second wrist of the user.