# US 11,308,733 B2 · Gesture detection using ultrasonic clicks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,733 B2
- Document ID: US11308733B2
- Status: Granted
- Kind: B2
- Application number: 16/531,995
- Filed: 2019-08-05
- Published: 2021-02-11
- Granted: 2022-04-19
- Earliest claimed: 2019-08-05
- Assignees: Bose Corporation
- Applicants: Bose Corporation
- Inventors: Prakash Manandhar; Ayan Agrawal; Somasundaram Meiyappan; Nathan Blagrove
- Web page: https://patentgrep.com/patent/US11308733B2

## CPC

- G06F 3/043: using propagating acoustic waves
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 18/24: Classification techniques
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- H04R 1/1008: {Earpieces of the supra-aural or circum-aural type}
- H04R 1/1041: {Mechanical or electronic switches, or control elements (switches in general H01H)}
- G06F 2218/12: Classification; Matching
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Systems and methods for detecting gestures using ultrasonic clicks are described. For example, some methods include playing an ultrasonic click using an audio driver, wherein the ultrasonic click is a modulated audio pulse with a duration less than one hundred periods of a modulation frequency waveform of the ultrasonic click; accessing a recording, captured using a microphone, of the ultrasonic click; applying an ultrasonic bandpass filter to the recording to obtain a filtered signal; determining a classification of a gesture based on the filtered signal; and transmitting, storing, or displaying an indication of the classification of the gesture.

## Claims

### Claim 1

A system comprising:
a microphone;
an audio driver; and
a processing apparatus configured to:
play an ultrasonic click using the audio driver, wherein the ultrasonic click is a modulated audio pulse with a duration less than one hundred periods of a modulation frequency waveform of the ultrasonic click;
access a recording, captured using the microphone, of the ultrasonic click;
apply an ultrasonic bandpass filter to the recording to obtain a filtered signal; and
determine a classification of a gesture based on the filtered signal.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processing apparatus is further configured to:
input data based on the filtered signal to a machine learning module to obtain the classification of the gesture.

### Claim 3 (depends on 1)

The system of claim 1, wherein the microphone is a first microphone, the recording is a first recording, and the filtered signal is a first filtered signal; and further comprising:
a second microphone, wherein the processing apparatus is configured to:
access a second recording, captured using the second microphone, of the ultrasonic click;
apply the ultrasonic bandpass filter to the second recording to obtain a second filtered signal; and
determine the classification of the gesture based on the second filtered signal.

### Claim 4 (depends on 1)

The system of claim 1, wherein the ultrasonic click is a first ultrasonic click, the recording is a first recording, and the filtered signal is a first filtered signal; and the processing apparatus is further configured to:
play a second ultrasonic click using the audio driver, wherein the second ultrasonic click has an amplitude different from a corresponding amplitude of the first ultrasonic click;
access a second recording, captured using the microphone, of the second ultrasonic click;
apply the ultrasonic bandpass filter to the second recording to obtain a second filtered signal; and
determine the classification of the gesture based on the second filtered signal.

### Claim 5 (depends on 1)

The system of claim 1, further comprising:
a fastening implement, attached to the audio driver and the microphone, configured to fasten the audio driver and the microphone to a part of a body of a user when the fastening implement is worn by the user.

### Claim 6 (depends on 5)

The system of claim 5, wherein the part of the body is an ear and the fastening implement includes a curved arm shaped to fit around a back of the ear, between the ear and a skull of the user.

### Claim 7 (depends on 1)

The system of claim 1, wherein the processing apparatus is further configured to:
subtract a background signal from the filtered signal to obtain a residual signal, wherein the classification of the gesture is determined based on the residual signal.

### Claim 8 (depends on 1)

The system of claim 1, wherein the processing apparatus is further configured to:
determine a Doppler shift of the filtered signal, wherein the classification of the gesture is determined based on the Doppler shift.

### Claim 9 (depends on 1)

The system of claim 1, wherein the processing apparatus is further configured to:
determine an envelope of the filtered signal, wherein the classification of the gesture is determined based on the envelope.

### Claim 10 (depends on 1)

The system of claim 1, wherein the processing apparatus is further configured to:
determine an amplitude and a phase of the filtered signal, wherein the classification of the gesture is determined based on the amplitude and the phase.

### Claim 11 (depends on 1)

The system of claim 1, wherein the processing apparatus is further configured to:
apply time-gain-compensation to the filtered signal.

### Claim 12 (depends on 1)

The system of claim 1, wherein the modulation frequency waveform is at a frequency between 18 kilohertz and 25 kilohertz.

### Claim 13

A method comprising:
playing an ultrasonic click using an audio driver, wherein the ultrasonic click is a modulated audio pulse with a duration less than one hundred periods of a modulation frequency waveform of the ultrasonic click;
accessing a recording, captured using a microphone, of the ultrasonic click;
applying an ultrasonic bandpass filter to the recording to obtain a filtered signal;
determining a classification of a gesture based on the filtered signal; and
transmitting, storing, or displaying an indication of the classification of the gesture.

### Claim 14 (depends on 13)

The method of claim 13, wherein determining the classification of the gesture based on the filtered signal comprises:
inputting data based on the filtered signal to a machine learning module to obtain the classification of the gesture.

### Claim 15 (depends on 13)

The method of claim 13, wherein the microphone is a first microphone, the recording is a first recording, and the filtered signal is a first filtered signal; and further comprising:
accessing a second recording, captured using a second microphone, of the ultrasonic click;
applying the ultrasonic bandpass filter to the second recording to obtain a second filtered signal; and
determining the classification of the gesture based on the second filtered signal.

### Claim 16 (depends on 13)

The method of claim 13, wherein the ultrasonic click is a first ultrasonic click, the recording is a first recording, and the filtered signal is a first filtered signal; and further comprising:
playing a second ultrasonic click using the audio driver, wherein the second ultrasonic click has an amplitude different from a corresponding amplitude of the first ultrasonic click;
accessing a second recording, captured using the microphone, of the second ultrasonic click;
applying the ultrasonic bandpass filter to the second recording to obtain a second filtered signal; and
determining the classification of the gesture based on the second filtered signal.

### Claim 17 (depends on 13)

The method of claim 13, further comprising:
subtracting a background signal from the filtered signal to obtain a residual signal, wherein the classification of the gesture is determined based on the residual signal.

### Claim 18 (depends on 13)

The method of claim 13, further comprising:
determining a Doppler shift of the filtered signal, wherein the classification of the gesture is determined based on the Doppler shift.

### Claim 19 (depends on 13)

The method of claim 13, further comprising:
determining an envelope of the filtered signal, wherein the classification of the gesture is determined based on the envelope.

### Claim 20 (depends on 13)

The method of claim 13, further comprising:
determining an amplitude and a phase of the filtered signal, wherein the classification of the gesture is determined based on the amplitude and the phase.

### Claim 21 (depends on 13)

The method of claim 13, further comprising:
applying time-gain-compensation to the filtered signal.

### Claim 22 (depends on 1)

The system of claim 1, wherein the modulated audio pulse is shaped according to a shaping function that sets the duration based on an envelope of the modulation frequency waveform.

### Claim 23 (depends on 1)

The system of claim 1, wherein the classification is determined without any time-of-flight measurement associated with the recording or the filtered signal.

### Claim 24 (depends on 13)

The method of claim 13, wherein the modulated audio pulse is shaped according to a shaping function that sets the duration based on an envelope of the modulation frequency waveform.

### Claim 25 (depends on 13)

The method of claim 13, wherein the classification is determined without any time-of-flight measurement associated with the recording or the filtered signal.