# US 11,543,887 B2 · User interface control of responsive devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,543,887 B2 - Document ID: US11543887B2 - Status: Granted - Kind: B2 - Application number: 16/935,816 - Filed: 2020-07-22 - Published: 2021-04-08 - Granted: 2023-01-03 - Earliest claimed: 2018-08-05 - Assignees: Pison Technology, Inc. - Applicants: Pison Technology, Inc. - Inventors: Dexter W. Ang; David O Cipoletta; Samuel J. Karnes; Michael P. Kowalczyk; Salil H. Patel; Steven D. Trambert - Web page: https://patentgrep.com/patent/US11543887B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G01C 25/005: {initial alignment, calibration or starting-up of inertial devices} - G02B 27/0172: {characterised by optical features} - G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/0485: Scrolling or panning - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 21/316: {by observing the pattern of computer usage, e.g. typical user behaviour} - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming - H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation - H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Among other things, a user interface device has a sensor configured to detect, at a wrist of a human, nerve or other tissue electrical signals associated with an intended contraction of a muscle to cause a rapid motion of a finger. An output provides information representative of the nerve or other tissue electrical signals associated with the intended contraction of the muscle to an interpreter of the information. ## Claims ### Claim 1 A system for gesture control, the system comprising: a wearable device configured to be worn at a wrist of a person and comprising: a biopotential sensor configured to obtain, at the wrist of the person, biopotential data, the biopotential data indicating nerve or other tissue electrical signals associated with an intended contraction of a muscle; and a wrist motion sensor configured to obtain wrist motion data, the wrist motion data indicating a motion of the person's wrist; and a second device, the second device being in communication with both the first device and a third device comprising a machine learning classifier, the second device being configured to obtain second device data using an interface of the second device, the second device data being distinct from the biopotential data and the wrist motion data; the system being configured to: obtain the biopotential data using the biopotential sensor of the wearable device; obtain the wrist motion data using the wrist motion sensor of the wearable device; transmit, from the wearable device to the second device, the biopotential data and the wrist motion data; obtain, using the second device, the second device data using the interface of the second device; and transmit, using the second device, each of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or derivatives of any of (i)-(iii), to the third device comprising the machine learning classifier, the machine learning classifier of the third device being configured to: process each of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii); and generate an interpreted output based on a combination of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii). ### Claim 2 (depends on 1) The apparatus of claim 1 comprising the third device, the third device being configured to take an action in response to the interpreted output. ### Claim 3 (depends on 1) The apparatus of claim 1 in which the wrist motion sensor is an inertial measurement unit that is calibrated using a calibration technique. ### Claim 4 (depends on 3) The apparatus of claim 3 in which the inertial measurement unit is configured to access a value corresponding to a reference point. ### Claim 5 (depends on 4) The apparatus of claim 4 in which the reference point is independent of the position of the inertial measurement unit. ### Claim 6 (depends on 4) The apparatus of claim 4 in which a vector provided by the inertial measurement unit is compared to a second vector that is calculated using the value corresponding to the reference point. ### Claim 7 (depends on 3) The apparatus of claim 3 in which the calibration technique is performed after a pre-defined number of spatial calculations of the inertial measurement unit. ### Claim 8 (depends on 3) The apparatus of claim 3 in which the calibration technique is performed in response to a command from the human. ### Claim 9 (depends on 1) The apparatus of claim 1, wherein the second device is in communication with a fourth device comprising a second machine learning classifier, the second machine learning classifier being configured to generate a second interpreted output based on one or more of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii), the second interpreted output being different than the interpreted output generated by the third device. ### Claim 10 (depends on 1) The apparatus of claim 1 in which the machine learning classifier is configured to provide the output to a controller configured to effect an action by a responsive device. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the controller is configured to effect the action by altering a presentation by the responsive device. ### Claim 12 (depends on 10) The apparatus of claim 10, wherein the controller is configured to cause the responsive device to perform a zoom. ### Claim 13 (depends on 10) The apparatus of claim 10, wherein the controller is configured to cause the responsive device to capture an image, video, or a sound, or a combination of them. ### Claim 14 (depends on 10) The apparatus of claim 10 in which the responsive device is configured to provide a presentation to a user of the user interface device. ### Claim 15 (depends on 1) The apparatus of claim 1, wherein the machine learning classifier is configured to generate the interpreted output based on a joint consideration of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii). ### Claim 16 (depends on 1) The apparatus of claim 1, wherein the machine learning classifier is configured to generate the interpreted output based on a separate consideration of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii). ### Claim 17 (depends on 1) The system of claim 1, wherein the second device comprises a microphone and the second data comprises audio data. ### Claim 18 A method for gesture control, the method comprising: obtaining, using a wearable device configured to be worn at a wrist of a person, biopotential data obtained using a biopotential sensor of the wearable device, the biopotential data indicating nerve or other tissue electrical signals associated with an intended contraction of a muscle; obtaining, using a wrist motion sensor of the wearable device, wrist motion data the wrist motion data indicating a motion of the person's wrist; wirelessly transmitting, from the wearable device to a second device, the biopotential data and the wrist motion data; obtaining, using the second device, second data, the second device data being distinct from the biopotential data and the wrist motion data; wirelessly transmitting, using the second device, each of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or derivatives of any of (i)-(iii) to a third device comprising a machine learning classifier, the machine learning classifier of the third device being configured to: process each of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii); and generate an interpreted output based on a combination of (i) the biopotential data, (ii) the wrist motion data, and (iii) the second device data, or the derivatives of any of (i)-(iii). ### Claim 19 (depends on 18) The method of claim 18, further comprising causing a responsive action to perform an action in response to the interpreted output. ### Claim 20 (depends on 19) The method of claim 19, wherein the machine learning classifier has been trained using training data.