# US 10,368,784 B2 · Sensor data damping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,368,784 B2
- Document ID: US10368784B2
- Status: Granted
- Kind: B2
- Application number: 14/311,878
- Filed: 2014-06-23
- Published: 2015-12-24
- Granted: 2019-08-06
- Earliest claimed: 2014-06-23
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Christopher Richard Marlow; Eike Jens Umlauf; Andrew John Preston
- Web page: https://patentgrep.com/patent/US10368784B2

## CPC

- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Sensor data damping is described, for example, to remove jitter from sensor data to enable control of a computing device. In various examples damped sensor data is compared with a threshold and clamped to the threshold in the case a difference between the sensor data and the damped sensor data is above the threshold. In various examples the damped data has low-latency and is used to control a downstream system such as a game system, natural user interface, robotic system, augmented reality system or other system. In examples, the threshold is adjusted on the fly on the basis of any one or more of: state data from the downstream system, frequency of clamping, velocity of the sensor data. In some examples, the sensor data comprises human/animal joint positions from a depth sensor.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving a stream of sensor data from a sensor configured to capture depth data;
damping the sensor data from the sensor configured to capture depth data to produce damped sensor data;
determining that a difference between a current item of the stream of sensor data from the sensor configured to capture depth data and a corresponding damped item of the damped sensor data exceeds a threshold;
on the basis of the determining, modifying the damped item by the threshold; and
outputting the modified damped item in a stream of output data and not the damped item, thereby reducing latency in the stream of output between the damped item and the sensor data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sensor data comprises features or characteristics calculated from raw sensor data.

### Claim 3 (depends on 1)

The method of claim 1 wherein the sensor data is received from a capture device and comprises a stream of images.

### Claim 4 (depends on 1)

The method of claim 1 comprising dynamically adjusting the threshold on the basis of state data obtained from a computing device to be controlled, at least in part, using output of the damping logic.

### Claim 5 (depends on 1)

The method of claim 1 comprising dynamically adjusting the threshold on the basis of data velocity of the sensor data stream.

### Claim 6 (depends on 5)

The method of claim 5 comprising calculating a data velocity of the sensor data stream.

### Claim 7 (depends on 1)

The method of claim 1 comprising dynamically adjusting the threshold on the basis of past threshold clamping behavior of the damping logic with respect to the sensor data stream.

### Claim 8 (depends on 7)

The method of claim 7 comprising calculating past threshold clamping behavior of the damping logic as frequency of replacing items of the damped sensor data by the threshold.

### Claim 9 (depends on 1)

The method of claim 1 comprising dynamically adjusting the threshold on the basis of a combination of any two or more of: state data obtained from a computing device, data velocity of the sensor data stream, past threshold clamping behavior of the damping logic.

### Claim 10 (depends on 1)

The method of claim 1 comprising setting the threshold on the basis of background jitter of the sensor data observed during a stationary state of the sensor and/or an environment of the sensor.

### Claim 11 (depends on 1)

The method of claim 1 wherein the stream of sensor data comprises joint positions computed from a stream of images of an articulated entity.

### Claim 12 (depends on 1)

The method of claim 1 comprising repeating the calculating and comparing steps in order to produce a stream of output data to control a computing device.

### Claim 13 (depends on 12)

The method of claim 12 wherein the computing device comprises a game system.

### Claim 14 (depends on 12)

The method of claim 12 wherein the computing device comprises a natural user interface.

### Claim 15

A method at a damping logic comprising:
receiving a stream of sensor data from a sensor configured to capture depth data, the sensor data comprising joint position data calculated from a stream of images;
damping the sensor data from the sensor configured to capture depth data to produce damped sensor data;
calculating a difference between a current item of the stream of sensor data from the sensor configured to capture depth data and a corresponding damped item from the damping logic;
comparing the difference with a threshold;
on the basis of the comparison, modifying the damped item by the threshold;
replacing the damped item with the modified damped item, and thereby reducing latency in the stream of sensor data; and
using the stream of sensor data, including the modified damped item, to control a downstream system.

### Claim 16

A system comprising:
a sensor configured to capture depth data;
a processor;
memory having damping logic at least partly stored in the memory and executable by the processor to:
receive a stream of sensor data from the sensor configured to capture depth data;
damp the sensor data from the sensor configured to capture depth data to produce damped sensor data;
calculate a difference between a current item of the stream of sensor data from the sensor configured to capture depth data and a corresponding damped item from the damping logic;
compare the difference with a threshold;
on the basis of the comparison, modify the damped item by the threshold; and
replace the damped item in a stream of output data with a modified damped item adjusted by the threshold, thereby reducing latency in the stream of output data.

### Claim 17 (depends on 16)

The system of claim 16, wherein the sensor data comprises features or characteristics calculated from raw sensor data.

### Claim 18 (depends on 16)

The system of claim 16, wherein the damping logic is executable by the processor to repeat the calculating and comparing steps in order to produce a stream of output data to control a computing device.

### Claim 19 (depends on 16)

The system of claim 16, wherein the damping logic is executable by the processor to dynamically adjust the threshold on the basis of any one or more of: state data obtained from a computing device, data velocity of the sensor data stream, past threshold clamping behavior of the damping logic.

### Claim 20 (depends on 16)

The system of claim 16, wherein the damping logic is at least partially implemented using hardware logic selected from any one or more of: a field-programmable gate array, a program-specific integrated circuit, a program-specific standard product, a system-on-a-chip, or a complex programmable logic device.