# US 2017/0113042 A1 · Controlling ear stimulation in response to image analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0113042 A1
- Document ID: US20170113042A1
- Status: Published application
- Kind: A1
- Application number: 15/340,058
- Filed: 2016-11-01
- Published: 2017-04-27
- Earliest claimed: 2015-03-27
- Assignees: eQuility LLC
- Applicants: Elwha LLC
- Inventors: Eleanor V. Goodall; Roderick A. Hyde; Muriel Y. Ishikawa; Jordin T. Kare; Melanie K. Kitzan; Eric C. Leuthardt; Mark A. Malamud; Stephen L. Malaska; Nathan P. Myhrvold; Brittany Scheid; Katherine E. Sharadin; Elizabeth A. Sweeney; Clarence T. Tegreene; Charles Whitmer; Lowell L. Wood, JR.; Victoria Y.H. Wood
- Web page: https://patentgrep.com/patent/US20170113042A1

## CPC

- A61N 1/36036: {of the outer, middle or inner ear}
- A61H 23/02: with electric or magnetic drive
- A61N 1/0456: {Specially adapted for transcutaneous electrical nerve stimulation [TENS]}
- A61N 1/0472: {Structure-related aspects}
- A61N 1/36031: {using physiological parameters for adjustment}
- A61N 1/37247: {User interfaces, e.g. input or presentation means}
- A61N 1/37258: {Alerting the patient}
- A61N 2/002: {in combination with another treatment}
- A61N 2/006: {for magnetic stimulation of nerve tissue}
- A61N 5/0622: {Optical stimulation for exciting neural tissue}
- A61N 7/00: Ultrasound therapy (lithotripsy A61B17/22, A61B17/225; massage using supersonic vibration A61H23/00 {; using ultrasound for introducing media into the body A61M37/0092})
- A61N 1/0484: {Garment electrodes worn by the patient}
- A61N 1/36021: {for treatment of pain}
- A61N 1/36025: {for treating a mental or cerebral condition}
- A61N 2/02: using magnetic fields produced by coils, including single turn loops or electromagnets (A61N2/12 takes precedence)
- A61N 2005/0647: {the applicator adapted to be worn on the head}
- A61N 2007/0026: {Stimulation of nerve tissue}
- A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity
- A63F 13/424: involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
- A61N: ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY (measurement of bioelectric currents A61B; surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body A61B18/00; anaesthetic apparatus in general A61M; incandescent lamps H01K; infrared radiators for heating H05B)
- A61H: PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY (electrotherapy, magnetotherapy, radiation therapy or ultrasound therapy A61N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

Systems and related methods for controlling an ear stimulation device with a personal computing device, based on analysis of an image of a user of the personal computing device captured with a camera associated with the personal computing device, are described. One or more parameter is determined from the processed image, including, but not limited, to user identity, emotion, physiological condition, or placement of a stimulator earpiece, and control of the ear stimulation device is based thereon. If the earpiece is located incorrectly, in an aspect, delivery of a stimulus is prevented. In another aspect, the user is notified of the need to reposition an incorrectly located earpiece.

## Claims

### Claim 1

A method of controlling an ear stimulation device with a personal computing device, comprising:
capturing, with image capture circuitry on the personal computing device, via a user-facing camera associated with the personal computing device, an image of a user of the personal computing device;
processing the image, using image processing circuitry on the personal computing device, to determine at least one parameter; and
controlling, with neural stimulus control signal determination circuitry on the personal computing device, based at least in part on the at least one parameter, delivery of a stimulus to at least one nerve innervating an ear of the user with the ear stimulation device.

### Claim 2 (depends on 1)

The method of claim 1, wherein the image processing circuitry includes application software on the personal computing device.

### Claim 3 (depends on 1)

The method of claim 1, wherein the at least one parameter is indicative of at least one emotion of the user.

### Claim 4 (depends on 1)

The method of claim 1, wherein the at least one parameter is indicative of a physiological condition of the user.

### Claim 5 (depends on 1)

The method of claim 1, wherein the at least one parameter is indicative of a medical condition of the user.

### Claim 6 (depends on 1)

The method of claim 1, wherein the at least one parameter is indicative of an identity of the user.

### Claim 7 (depends on 1)

The method of claim 1, wherein the at least one parameter is a heart rate of the user.

### Claim 8 (depends on 1)

The method of claim 1, wherein the at least one parameter is related to eye position or eye movement of the user.

### Claim 9 (depends on 1)

The method of claim 1, wherein the at least one parameter is indicative of a position of an earpiece of the ear stimulation device with respect to the ear of the user.

### Claim 10 (depends on 9)

The method of claim 9, including delivering, under control of notification circuitry on the personal computing device, a notification to the user informing the user of the need to adjust the position of the earpiece with respect to the ear of the user.

### Claim at position 11

-13. (canceled)

### Claim 14 (depends on 1)

The method of claim 1, wherein the personal computing device includes a phone, watch, wearable device, tablet computer, laptop computer, or desktop computer.

### Claim 15 (depends on 1)

The method of method of claim 1, including
processing the image, using the image processing circuitry, to determine
a presence of at least one earpiece of the ear stimulation device located at an ear of the user;
an ear of the user at which the at least one earpiece is located, the ear selected from a right ear of the user and a left ear of the user; and
at least one attribute of the at least one earpiece indicative of usability of the at least one earpiece with one of the left or the right ear of the user;
determining, using application software on the personal computing device, an ear at which the earpiece is usable, based on the at least one attribute of the at least one earpiece;
determining, using application software on the personal computing device, whether the ear at which the at least one earpiece is located is the ear at which the earpiece is usable; and
if the ear at which the at least one earpiece is located is not the ear at which the earpiece is usable, sending a control signal from the personal computing device to the ear stimulation device, under control of the neural stimulus control signal determination circuitry, to prevent delivery of a stimulus to the ear at which the earpiece is located via the earpiece.

### Claim 16 (depends on 15)

The method of claim 15, including receiving, with handshake circuitry on the personal computing device, a handshake signal from ear stimulation device control circuitry associated with the ear stimulation device.

### Claim 17 (depends on 16)

The method of claim 16, including capturing the image of the user of the personal computing device responsive to receiving the handshake signal from the ear stimulation device control circuitry.

### Claim 18 (depends on 16)

The method of claim 16, including sending a handshake signal to the ear stimulation device control circuitry responsive to determining the presence of the at least one earpiece located at the ear of the user in the image.

### Claim 19 (depends on 15)

The method of claim 15, including delivering, under control of notification circuitry on the personal computing device, a notification to the user informing the user of the need to switch the earpiece to the other ear if the ear at which the at least one earpiece is located is not the ear at which the earpiece is usable.

### Claim 20 (depends on 19)

The method of claim 19, wherein delivering the notification to the user includes delivering a text notification.

### Claim 21 (depends on 19)

The method of claim 19, wherein delivering the notification to the user includes delivering a visible notification.

### Claim 22 (depends on 19)

The method of claim 19, wherein delivering the notification to the user includes delivering an audio notification.

### Claim 23 (depends on 15)

The method of claim 15, wherein determining the at least one attribute of the at least one earpiece includes determining a shape of the at least one earpiece.

### Claim 24 (depends on 15)

The method of claim 15, wherein determining the at least one attribute of the at least one earpiece includes determining a color of the at least one earpiece.

### Claim 25 (depends on 15)

The method of claim 15, wherein determining the at least one attribute of the at least one earpiece includes determining the presence of an indicia on the at least one earpiece or an attachment to the at least one earpiece.

### Claim 26 (depends on 25)

The method of claim 25, wherein the attachment to the at least one earpiece includes a cable connected to the at least one earpiece.

### Claim 27

An ear stimulation device control system, comprising:
a personal computing device;
a user-facing camera associated with the personal computing device;
image capture circuitry adapted to capture an image of a user of the personal computing device from the user-facing camera;
image processing circuitry configured to process the image to determine at least one parameter; and
neural stimulus control signal determination circuitry configured to control delivery of a stimulus to at least one nerve innervating an ear of the user with an ear stimulation device, based at least in part on the at least one parameter.

### Claim 28 (depends on 27)

The system of claim 27, wherein the at least one parameter is indicative of at least one emotion of the user.

### Claim 29 (depends on 27)

The system of claim 27, wherein the at least one parameter is indicative of a physiological condition of the user.

### Claim 30 (depends on 27)

The system of claim 27, wherein the at least one parameter is indicative of an identity of the user.

### Claim 31 (depends on 27)

The system of claim 27, wherein the at least one parameter is a heart rate of the user.

### Claim 32 (depends on 27)

The system of claim 27, wherein the at least one parameter is related to eye position or eye movement of the user.

### Claim 33 (depends on 27)

The system of claim 27, wherein the at least one parameter is indicative of a position of the earpiece with respect to the ear of the user.

### Claim 34 (depends on 27)

The system of claim 27, wherein the personal computing device includes a phone, watch, wearable device, tablet computer, laptop computer, or desktop computer.

### Claim 35 (depends on 27)

The system of claim 27, wherein the image processing circuitry includes an earpiece location module configured to process the image to determine
a presence of at least one earpiece of the ear stimulation device located at an ear of the user;
an ear of the user at which the at least one earpiece is located, the ear selected from a right ear of the user and a left ear of the user; and
at least one attribute of the at least one earpiece indicative of usability of the at least one earpiece with one of the left or the right ear of the user; and
wherein the neural stimulus control signal determination circuitry is configured to
determine an ear at which the earpiece is usable, based on the at least one attribute of the at least one earpiece;
determine whether the ear at which the at least one earpiece is located is the ear at which the earpiece is usable; and
if the ear at which the at least one earpiece is located is not the ear at which the earpiece is usable, send a control signal from the personal computing device to the ear stimulation device to prevent delivery of the stimulus to the a least one nerve innervating the ear of the user.

### Claim 36 (depends on 35)

The system of claim 35, including handshake circuitry adapted to receive a handshake signal from ear stimulation device control circuitry associated with the ear stimulation device.

### Claim 37 (depends on 36)

The system of claim 36, wherein the image capture circuitry is adapted to capture the image of the user of the personal computing device responsive to receiving the handshake signal from the ear stimulation device control circuitry.

### Claim 38 (depends on 36)

The system of claim 36, wherein the neural stimulus control signal determination circuitry is configured to send a handshake signal to the ear stimulation device control circuitry responsive to determining the presence of the at least one earpiece located at the ear of the user in the image.

### Claim 39 (depends on 35)

The system of claim 35, including
an output device; and
notification circuitry adapted to provide a notification via the output device instructing the user of to switch the earpiece to the other ear if the ear at which the at least one earpiece is located is not the ear at which the earpiece is usable.

### Claim 40 (depends on 39)

The system of claim 39, wherein the output device is adapted to deliver a text notification to the user.

### Claim 41 (depends on 39)

The system of claim 39, wherein the output device is adapted to deliver a visible notification to the user.

### Claim 42 (depends on 39)

The system of claim 39, wherein the output device is adapted to deliver an audio notification to the user.

### Claim 43 (depends on 35)

The system of claim 35, wherein the at least one attribute of the at least one earpiece includes a shape of the at least one earpiece.

### Claim 44 (depends on 35)

The system of claim 35, wherein the at least one attribute of the at least one earpiece includes a color of the at least one earpiece.

### Claim 45 (depends on 35)

The system of claim 35, wherein the at least one attribute of the at least one earpiece includes a presence of an indicia on the at least one earpiece or an attachment to the at least one earpiece.

### Claim 46 (depends on 45)

The system of claim 45, wherein the attachment to the at least one earpiece includes a cable connected to the at least one earpiece.
