# US 2021/0063434 A1 · Methods and apparatus for detecting individual health related events

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0063434 A1
- Document ID: US20210063434A1
- Status: Published application
- Kind: A1
- Application number: 16/994,524
- Filed: 2020-08-14
- Published: 2021-03-04
- Earliest claimed: 2019-08-26
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Gierad LAPUT; Jared LeVan ZERBE; William C. ATHAS; Andreas Edgar SCHOBEL; Shawn R. SCULLY; Brian H. TSANG; Kevin LYNCH; Charles MAALOUF; Shiwen ZHAO
- Web page: https://patentgrep.com/patent/US20210063434A1

## CPC

- G01J 1/4228: {arrangements with two or more detectors, e.g. for sensitivity compensation}
- G01P 15/18: in two or more dimensions
- A61B 5/0022: {Monitoring a patient using a global network, e.g. telephone networks, internet}
- A61B 5/02416: {using photoplethysmograph signals, e.g. generated by infrared radiation (A61B5/14552 takes precedence)}
- A61B 5/1125: {Grasping motions of hands}
- A61B 5/14542: {for measuring blood gases (A61B5/14551 takes precedence)}
- A61B 5/28: for electrocardiography [ECG]
- A61B 5/486: {Biofeedback (using electroencephalography [EEG] A61B5/375)}
- A61B 5/681: {Wristwatch-type devices}
- G01J 1/4204: {with determination of ambient light (solar light G01J2001/4266)}
- G01P 13/00: Indicating or recording presence, absence, or direction, of movement (electric switches H01H; counting moving objects G06M7/00)
- G10L 25/51: for comparison or discrimination
- H04R 1/08: Mouthpieces; {Microphones;} Attachments therefor
- A61B 5/1112: {Global tracking of patients, e.g. by using GPS}
- A61B 2560/0209: adapted for power saving
- A61B 2562/0204: Acoustic sensors
- A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
- A61B 2562/0257: Proximity sensors
- A61B 2562/0271: Thermal or temperature sensors
- G01J 2001/0257: {portable}
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G})
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Individual health related events (e.g., handwashing events) can be detected based on multiple sensors including motion and audio sensors. Detecting a qualifying handwashing event can include detecting a qualifying scrubbing event based on motion data (e.g., accelerometer data) and a qualifying rinsing event based on audio data. In some examples, power consumption can be reduced by implementing one or more power saving mitigations.

## Claims

### Claim 1

A wearable device comprising:
an audio sensor configured to acquire audio data;
a motion sensor configured to acquire motion data; and
processing circuitry programmed to detect a qualifying handwashing event using the audio data and the motion data.

### Claim 2 (depends on 1)

The device of claim 1, wherein the processing circuitry is further programmed to:
detect, using the motion data, a qualifying motion event; and
in response to detecting the qualifying motion event, trigger acquisition of the audio data by the audio sensor.

### Claim 3 (depends on 1)

The device of claim 1, wherein detecting the qualifying handwashing event using the audio data and the motion data comprises detecting a qualifying scrubbing event using the motion data and detecting a qualifying rinsing event using the audio data.

### Claim 4 (depends on 3)

The device of claim 3, wherein detecting the qualifying handwashing event includes detecting the qualifying rinsing event within a threshold period of time of detecting the qualifying scrubbing event.

### Claim 5 (depends on 1)

The device of claim 1, wherein the processing circuitry includes a first processor and a second processor, wherein the first processor consumes less power during operation than the second processor.

### Claim 6 (depends on 5)

The device of claim 5, wherein detecting the qualifying event includes detecting a triggering condition, and wherein the processing circuitry is further programmed to:
in response to detecting the triggering condition, trigger processing by the second processor.

### Claim 7 (depends on 5)

The device of claim 5, wherein detecting the qualifying handwashing event includes the first processor detecting a triggering condition, and wherein the processing circuitry is further programmed to:
in response to the first processor detecting the triggering condition, trigger processing by the second processor.

### Claim 8 (depends on 7)

The device of claim 7, wherein the first processor is further programmed to wake up the second processor in response to the triggering condition.

### Claim 9 (depends on 7)

The device of claim 7, wherein the triggering condition includes a qualifying motion event detected using the motion data.

### Claim 10 (depends on 7)

The device of claim 7, wherein the triggering condition includes both a qualifying motion event detected using the motion data and a qualifying audio event detected using the audio data.

### Claim 11 (depends on 7)

The device of claim 7, wherein the triggering condition includes detecting the qualifying handwashing event with a confidence level less than a threshold confidence level.

### Claim 12 (depends on 7)

The device of claim 7, wherein the processing by the second processor includes detecting a qualifying rinsing event using the audio data.

### Claim 13 (depends on 7)

The device of claim 7, wherein the processing by the second processor includes detecting the qualifying handwashing event using the audio data.

### Claim 14 (depends on 7)

The device of claim 7, wherein the processing by the second processor includes detecting the qualifying handwashing event using the audio data and motion data.

### Claim 15 (depends on 7)

The device of claim 7, wherein the first processor detecting the triggering condition includes detecting a first qualifying audio event using a first machine learning audio model applied to the audio data, and the processing by the second processor including detecting a second qualifying audio event using a second machine learning model applied to the audio data, the second machine learning model different than the first machine learning model.

### Claim 16 (depends on 6)

The device of claim 6, wherein the first processor detecting the triggering condition includes detecting a first qualifying motion event using a first machine learning motion model applied to the motion data, and the processing by the second processor including detecting a second qualifying motion event using a second machine learning model applied to the motion data, the second machine learning model different than the first machine learning model.

### Claim 17 (depends on 1)

The wearable device of claim 1, the processing circuitry further programmed to:
in accordance with a determination that the audio sensor is obstructed, process the audio data using a first machine learning audio model for an obstructed audio sensor applied to the audio data; and
in accordance a determination that the audio sensor is unobstructed, process the audio data using a second machine learning audio model for an unobstructed audio sensor applied to the audio data.

### Claim 18 (depends on 17)

The wearable device of claim 17, further comprising:
one or more ambient light sensors;
wherein the determination that the audio sensor is obstructed or unobstructed is based on a differential measurement from the one or more ambient light sensors.

### Claim 19

A method comprising:
at a wearable device including an audio sensor configured to acquire audio data and a motion sensor configured to acquire motion data:
detecting a qualifying handwashing event using the audio data and the motion data.

### Claim 20

A non-transitory computer readable storage medium storing instructions, which when executed by one or more processing circuits of a wearable device including an audio sensor configured to acquire audio data and a motion sensor configured to acquire motion data, cause the one or more processing circuits to detect a qualifying handwashing event using the audio data and the motion data.
