# US 2022/0279266 A1 · Activity detection using a hearing instrument

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0279266 A1
- Document ID: US20220279266A1
- Status: Published application
- Kind: A1
- Application number: 17/664,155
- Filed: 2022-05-19
- Published: 2022-09-01
- Earliest claimed: 2019-11-27
- Assignees: Starkey Laboratories, Inc.
- Applicants: Starkey Laboratories, Inc.
- Inventors: Vishal Vijayakumar; Matthew D. Kleffner; Tao Zhang
- Web page: https://patentgrep.com/patent/US20220279266A1

## CPC

- H04R 1/1041: {Mechanical or electronic switches, or control elements (switches in general H01H)}
- H04R 25/505: {using digital signal processing}
- H04R 25/507: {implemented by neural network or fuzzy logic}
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A computing system includes a memory and at least one processor. The memory is configured to store motion data indicative of motion of a hearing instrument. The at least one processor is configured to determine a type of activity performed by a user of the hearing instrument and output data indicating the type of activity performed by the user.

## Claims

### Claim 1

A hearing instrument comprising:
a memory configured to store a plurality of machine trained activity models; and
at least one processor configured to:
determine, by applying a hierarchy of the plurality of machine trained activity models to motion data indicative of motion of the hearing instrument, a type of activity performed by a user of the hearing instrument; and
responsive to determining the type of activity performed by the user, output data indicating the type of activity performed by the user.

### Claim 2 (depends on 1)

The hearing instrument of claim 1, wherein the hierarchy of the plurality of machine trained activity models includes a first machine trained activity model trained to detect a first type activity and a second machine trained activity model trained to detect a second type activity different than the first type of activity, wherein the at least one processor is configured to apply the hierarchy of machine trained activity models by at least being configured to:
apply the first machine trained activity model to the motion data to determine whether the user is performing the first type of activity;
responsive to determining that the user is not performing the first type of activity, apply the second machine trained activity model to the motion data to determine whether the user is performing the second type of activity; and
responsive to determining that the user is performing the second type of activity, determining the second type of activity is the type of activity performed by the user.

### Claim 3 (depends on 2)

The hearing instrument of claim 2, wherein the plurality of machine trained activity models includes a third machine trained activity model trained to detect a first sub-type activity that is associated with the second type of activity and a fourth machine trained activity model trained to detect a second sub-type of activity that is associated with the second type of activity, wherein the first sub-type of activity is different than the second sub-type of activity, and wherein the at least one processor is further configured to:
apply the third machine trained activity model to the motion data to determine whether the user is performing the first sub-type of activity;
responsive to determining that the user is not performing the first sub-type of activity, apply the fourth machine trained activity model to the motion data to determine whether the user is performing the second sub-type of activity; and
responsive to determining that the user is performing the second sub-type of activity, determining the second sub-type of activity is the type of activity performed by the user.

### Claim 4 (depends on 1)

The hearing instrument of claim 1, wherein the hearing instrument is a first hearing instrument, and wherein the at least one processor is further configured to:
receive, from a second hearing instrument, data indicating another type of activity performed by the user;
determine whether the type of activity is the same as the other type of activity; and
responsive to determining that the type of the activity is different than the other type of activity, output an indication of the motion data to an edge computing device.

### Claim 5 (depends on 1)

The hearing instrument of claim 1, wherein the at least one processor is further configured to:
determine the type of activity is unknown in response to determining that the type of the activity is neither the first type of activity or the second type of activity; and
responsive to determining that the type of activity is unknown, output an indication of the motion data to an edge computing device.

### Claim 6 (depends on 1)

The hearing instrument of claim 1, wherein the type of activity is a type of activity associated with a first time period, and wherein the at least one processor is further configured to:
determine a type of activity performed by the user during a second time period that is within a threshold amount of time of the first time period;
determine a type of activity performed by the user during a third time period that is within the threshold amount of time of the second time period; and
responsive to determining that the type of the activity performed by the user during the first time period is different than the type of the activity performed by the user during the second time period and that the type of the activity performed by the user during the second time period is different than the type of the activity performed by the user during the third time period, perform an action to re-assign at least one of the type of activity performed by the user during the first time period, the second time period, or the third time period.

### Claim 7 (depends on 6)

The hearing instrument of claim 6, wherein the type of activity is a type of activity associated with a first time period, and wherein the at least one processor is configured to perform the action to re-assign the at least one type of activity by at least being configured to:
responsive to determining that the type of the activity performed by the user during the first time period is the same as the type of the activity performed by the user during the third time period, assign the type of activity performed by the user during the second time period as the type of activity performed by the user during the first time period and third time period.

### Claim 8 (depends on 6)

The hearing instrument of claim 6, wherein the type of activity is a type of activity associated with a first time period, and wherein the at least one processor is configured to perform the action to re-assign the at least one type of activity by at least being configured to:
output a command causing an edge computing device to determine the type of activity performed during each respective time period of the first time period, the second time period, and the third time period, wherein the command includes an indication of the motion data.

### Claim 9 (depends on 1)

The hearing instrument of claim 1, wherein the at least one processor is further configured to:
receive data indicative of an update to a machine trained activity model of the plurality of machine trained activity models; and
update the machine trained activity model stored in the memory.

### Claim 10 (depends on 1)

The hearing instrument of claim 1, wherein the at least one processor is further configured to determine an updated hierarchy of the plurality of machine trained activity models.

### Claim 11 (depends on 10)

The hearing instrument of claim 10, wherein the at least one processor is further configured to determine the updated hierarchy of the plurality of machine trained activity models by at least being configured to:
determine, based on historical activity data associated with the user, a type of activity most frequently performed by the user; and
assign a machine trained activity model associated with the type of activity most frequently performed as a first machine trained activity model in the updated hierarchy.

### Claim 12

A method comprising:
receiving, by at least one processor, motion data indicative of motion of a hearing instrument;
determining, by the at least one processor, a type of activity performed by a user of the hearing instrument by applying a hierarchy of the plurality of machine trained activity models to the motion data; and
responsive to determining the type of activity performed by the user, outputting data indicating the type of activity performed by the user.

### Claim 13 (depends on 12)

The method of claim 12, wherein the hierarchy of the plurality of machine trained activity models includes a first machine trained activity model trained to detect a first type activity and a second machine trained activity model trained to detect a second type activity different than the first type of activity, wherein applying the hierarchy of machine trained activity models comprises:
applying, by the at least one processor, the first machine trained activity model to the motion data to determine whether the user is performing the first type of activity;
responsive to determining that the user is not performing the first type of activity, applying, by the at least one processor, the second machine trained activity model to the motion data to determine whether the user is performing the second type of activity; and
responsive to determining that the user is performing the second type of activity, determining, by the at least one processor, the second type of activity is the type of activity performed by the user.

### Claim 14 (depends on 13)

The method of claim 13, wherein the plurality of machine trained activity models includes a third machine trained activity model trained to detect a first sub-type activity that is associated with the second type of activity and a fourth machine trained activity model trained to detect a second sub-type of activity that is associated with the second type of activity, wherein the first sub-type of activity is different than the second sub-type of activity, the method further comprising:
applying, by the at least one processor, the third machine trained activity model to the motion data to determine whether the user is performing the first sub-type of activity;
responsive to determining that the user is not performing the first sub-type of activity, applying, by the at least one processor, the fourth machine trained activity model to the motion data to determine whether the user is performing the second sub-type of activity; and
responsive to determining that the user is performing the second sub-type of activity, determining, by the at least one processor, the second sub-type of activity is the type of activity performed by the user.

### Claim 15 (depends on 12)

The method of claim 12, wherein the hearing instrument is a first hearing instrument, the method further comprising:
receiving, by the at least one processor, from a second hearing instrument, data indicating another type of activity performed by the user;
determining, by the at least one processor, whether the type of activity is the same as the other type of activity; and
responsive to determining that the type of the activity is different than the other type of activity, outputting, by the at least one processor, an indication of the motion data to an edge computing device.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
determining, by the at least one processor, the type of activity is unknown in response to determining that the type of the activity is neither the first type of activity or the second type of activity;
responsive to determining that the type of activity is unknown, outputting, by the at least one processor an indication of the motion data to an edge computing device.

### Claim 17 (depends on 12)

The method of claim 12, wherein the type of activity is a type of activity associated with a first time period, the method further comprising:
determining, by the at least one processor, a type of activity performed by the user during a second time period that is within a threshold amount of time of the first time period;
determining, by the at least one processor, a type of activity performed by the user during a third time period that is within the threshold amount of time of the second time period; and
responsive to determining that the type of the activity performed by the user during the first time period is different than the type of the activity performed by the user during the second time period and that the type of the activity performed by the user during the second time period is different than the type of the activity performed by the user during the third time period, performing, by the at least one processor, an action to re-assign at least one of the type of activity performed by the user during the first time period, the second time period, or the third time period.

### Claim 18 (depends on 17)

The method of claim 17, wherein the type of activity is a type of activity associated with a first time period, and wherein performing the action to re-assign the at least one type of activity comprises:
responsive to determining that the type of the activity performed by the user during the first time period is the same as the type of the activity performed by the user during the third time period, assigning, by the at least one processor, the type of activity performed by the user during the second time period as the type of activity performed by the user during the first time period and third time period.

### Claim 19 (depends on 12)

The method of claim 12, further comprising determining, by the at least one processor, an updated hierarchy of the plurality of machine trained activity models.

### Claim 20

A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor of a computing device, cause the at least one processor to:
receive motion data indicative of motion of a hearing instrument;
determine, by applying a hierarchy of the plurality of machine trained activity models to the motion data, a type of activity performed by a user of the hearing instrument; and
responsive to determining the type of activity performed by the user, output data indicating the type of activity performed by the user.

### Claim 21

(canceled)
