# US 2019/0156219 A1 · Detecting Human Input Activity In a Cognitive Environment Using Wearable Inertia and Audio Sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0156219 A1
- Document ID: US20190156219A1
- Status: Published application
- Kind: A1
- Application number: 15/818,140
- Filed: 2017-11-20
- Published: 2019-05-23
- Earliest claimed: 2017-11-20
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Victor C. Dibia
- Web page: https://patentgrep.com/patent/US20190156219A1

## CPC

- G06N 20/00: Machine learning
- A61B 5/1118: {Determining activity level}
- A61B 5/681: {Wristwatch-type devices}
- A61B 5/6824: {Arm or wrist}
- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- A61B 5/168: {Evaluating attention deficit, hyperactivity}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06N 5/02: Knowledge representation; Symbolic representation
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A mechanism is provided in a data processing system comprising a processor and a memory. The memory comprises instructions which are executed by the processor to cause the processor to be specifically configured to implement a recognizer module for detecting user input in a cognitive environment. The recognizer module receives sensor signals from at least one wearable device being worn by a user. The recognizer module analyzes the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user. The recognizer module communicates the at least one user input indicator to a cognitive system executing within the cognitive environment. The cognitive system performs at least one cognitive action based on the at least one user input indicator.

## Claims

### Claim 1

A method, in a data processing system comprising a processor and a memory wherein the memory comprises instructions which are executed by the processor to cause the processor to be specifically configured to implement a recognizer module for detecting user input in a cognitive environment, the method comprising:
receiving, by the recognizer module, sensor signals from at least one wearable device being worn by a user;
analyzing, by the recognizer module, the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user;
communicating, by the recognizer module, the at least one user input indicator to a cognitive system executing within the cognitive environment; and.
performing, by the cognitive system, at least one cognitive action based on the at least one user input indicator.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sensor signals comprise at least one vibration signal from an accelerometer or gyroscope in the at least one wearable device.

### Claim 3 (depends on 2)

The method of claim 2, wherein the at least one user input indicator comprises a wrist angle indicator that indicates an estimated wrist angle associated with the user input activity of the user.

### Claim 4 (depends on 3)

The method of claim 3, wherein the at least one cognitive action comprises activating an actuator to adjust a chair height.

### Claim 5 (depends on 3)

The method of claim 3, wherein the at least one cognitive action comprises activating an actuator to adjust a keyboard configuration.

### Claim 6 (depends on 1)

The method of claim 1, wherein the at least one user input indicator comprises a stress level indicator that indicates stress level associated with the user input activity of the user.

### Claim 7 (depends on 6)

The method of claim 6, wherein the recognizer module determines the stress level of the user based on an intensity of key strikes during the user input activity.

### Claim 8 (depends on 6)

The method of claim 6, wherein the sensor signals comprise at least one heart rate signal from a heart rate monitor in the at least one wearable device.

### Claim 9 (depends on 6)

The method of claim 6, wherein the at least one cognitive action comprises activating at least one of an actuator to dim lighting, an ambient lighting actuator, an actuator to control music, an actuator to control temperature.

### Claim 10 (depends on 1)

The method of claim 1, wherein the machine learning model is a user-specific machine learning model trained for the user of the at least one wearable device.

### Claim 11 (depends on 1)

The method of claim 1, wherein the at least one wearable device includes a smartwatch device.

### Claim 12 (depends on 1)

The method of claim 1, wherein the at least one wearable device includes a fitness band.

### Claim 13

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a processor of a data processing system, causes the data processing system to implement a recognizer module for detecting user input in a cognitive environment, wherein the computer readable program causes the data processing system to:
receive, by the recognizer module, sensor signals from at least one wearable device being worn by a user;
analyze, by the recognizer module, the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user;
communicate, by the recognizer module, the at least one user input indicator to a cognitive system executing within the cognitive environment; and
perform by the cognitive system, at least one cognitive action based on the at least one user input indicator.

### Claim 14 (depends on 13)

The computer program product of claim 13, wherein the sensor signals comprise at least one vibration signal from an accelerometer or gyroscope in the at least one wearable device.

### Claim 15 (depends on 14)

The computer program product of claim 14, wherein the at least one user input indicator comprises a wrist angle indicator that indicates an estimated wrist angle associated with the user input activity of the user.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the at least one cognitive action comprises activating an actuator to adjust a chair height or activating an actuator to adjust a keyboard configuration.

### Claim 17 (depends on 13)

The computer program product of claim 13, wherein the at least one user input indicator comprises a stress level indicator that indicates stress level associated with the user input activity of the user.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the at least one cognitive action comprises activating at least one of an actuator to dim lighting, an ambient lighting actuator, an actuator to control music, an actuator to control temperature.

### Claim 19 (depends on 13)

The computer program product of claim 13, wherein the at least one wearable device includes a smartwatch device or a fitness band.

### Claim 20

An apparatus comprising:
a processor; and
a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to implement a recognizer module for detecting user input in a cognitive environment, wherein the instructions cause the processor to:
receive, by the recognizer module, sensor signals from at least one wearable device being worn by a user;
analyze, by the recognizer module, the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user;
communicate, by the recognizer module, the at least one user input indicator to a cognitive system executing within the cognitive environment; and
perform by the cognitive system, at least one cognitive action based on the at least one user input indicator.
