# US 12,008,092 B2 · Supervised and unsupervised techniques for motion classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,092 B2
- Document ID: US12008092B2
- Status: Granted
- Kind: B2
- Application number: 17/884,456
- Filed: 2022-08-09
- Published: 2022-12-01
- Granted: 2024-06-11
- Earliest claimed: 2019-04-30
- Assignees: TruU, Inc.
- Applicants: TruU, Inc.
- Inventors: Lucas Allen Budman; Amitabh Agrawal; Andrew Weber Spott
- Web page: https://patentgrep.com/patent/US12008092B2

## CPC

- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06F 18/23213: with fixed number of clusters, e.g. K-means clustering
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/091: Active learning
- G06N 20/00: Machine learning
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 10/764: using classification, e.g. of video objects
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G06N 3/045: Combinations of networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system and method are disclosed for identifying a user based on the classification of user movement data. An identity verification system receives a sequence of motion data characterizing movements performed by a target user. The sequence of motion data is received as a point cloud of the motion data. The point cloud is input a machine-learned model trained based on manually labeled point clusters of a training set of motion data that each represent a movement. The machine-learned model identifies a movement represented by the point cloud of the motion data and assigns a label describing the movement the point cloud. The system generates a labeled representation of the sequence of motion data comprising the label identifying a portion of the sequence of motion data corresponding to the identified movement.

## Claims

### Claim 1

A non-transitory computer-readable medium comprising stored computer-readable instructions, the instructions when executed by a processor, cause the processor to:
receive, from at least one sensor, a sequence of motion data characterizing movements performed by a target user, the sequence of motion data encoded as a feature vector representation of the motion data;
input the feature vector representation of the motion data into a machine-learned model trained based on manually labeled groups of entries of a training set of motion data, each labeled group of the training set representative of a mapping from the entries in the labeled group to a movement;
identify, by the machine-learned model, a movement represented by the feature vector representation of the motion data, the identification determining a label to the feature vector describing the movement represented by the feature vector; and
generate a labeled representation of the sequence of motion data comprising the label describing the movement represented by the feature vector representation of the motion data and identifying a portion of the sequence of motion data corresponding to the identified movement.

### Claim 2 (depends on 1)

The non-transitory computer-readable medium of claim 1, further comprising stored instructions that when executed cause the processor to:
identify, from the labeled representation of the sequence of motion data, a plurality of identity blocks, each identity block representative of a different movement performed by the target user;
from each identity block, encode a set of signature sequences into the feature vector representation of the identity block;
input the feature vector representation of the identity block into a machine-learned identity confidence model to output an identity confidence value for the identity block, the identity confidence value descriptive of a confidence that the movement in the identity block was performed by the target user; and
grant the target user access to an operational context responsive to determining the identity confidence value is at least one of greater than or equal to an operational security threshold.

### Claim 3 (depends on 1)

The non-transitory computer-readable medium of claim 1, further comprising stored instructions that when executed cause the processor to:
receive, from at least one sensor, the sequence of motion data characterizing movements performed by the target user; and
encode the sequence of motion data into the feature vector representation of the motion data.

### Claim 4 (depends on 1)

The non-transitory computer-readable medium of claim 1, further comprising stored instructions that when executed cause the processor to:
encode a feature vector representation of the training set of motion data using a signal processing technique;
group entries of the feature vector representation of the training set of motion data; and
label each group of entries with a movement based on labels assigned by a manual operator.

### Claim 5 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the feature vector representation of the training set of motion data is a point cloud and the stored instructions cause the processor to:
generate, from the point cloud of the training set motion data, a simplicial complex by expanding each point of the point cloud;
generate, from the simplicial complex, a persistent homology of the point cloud of the training set of motion data; and
apply a clustering algorithm to the point cloud of the training set of motion data to cluster points based on the persistent homology of the point cloud.

### Claim 6 (depends on 5)

The non-transitory computer-readable medium of claim 5, further comprising stored instructions that when executed cause the processor to:
apply a clustering algorithm to the point cloud of the training set of motion data to generate a plurality of point clusters by clustering points based on a proximity to neighboring points of the point cloud of the training set of motion data;
receive, from a manual operator in response to presenting the plurality of point clusters to the manual operator, a label for each point cluster of the plurality, the label to identify a movement represented by the point cluster; and
train the machine-learned model using the labeled point clusters of the training set of motion data.

### Claim 7 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the stored instructions cause the processor to:
determine, responsive to inputting the feature vector to the machine-learned model, a movement label for the feature vector and a confidence value determined for the movement label;
classify, responsive to the confidence value being below a threshold confidence, the feature vector as an uncertain movement;
receive, from a manual operator, a label for the feature vector identifying the uncertain movement;
update the training set of motion data with the label received from the manual operator and the feature vector; and
re-train the machine-learned model using the updated training set of motion data.

### Claim 8 (depends on 1)

The non-transitory computer-readable medium of claim 1, further comprising stored instructions that when executed cause the processor to:
determine a movement classifier score for the machine-learned model characterizing an accuracy of the machine-learned model, wherein the movement classifier score is determined based on the accuracy of labels assigned to each entry of a test set by the machine-learned model;
identify, responsive to the movement classifier score for the determined machine-learned model being less than a threshold score, an alternate classifier with a higher movement classifier score; and
implement the alternate classifier to identify the movement represented by the sequence of motion data.

### Claim 9

An identity verification system comprising:
computing device configured to record a sequence of motion data characterizing movements performed by a target user;
a clustering module configured to:
receive, from at least one sensor, a sequence of motion data characterizing movements performed by a target user, the sequence of motion data encoded as a feature vector representation of the motion data; and
a motion classifier module configured to:
input the feature vector representation of the motion data into a machine-learned model trained based on manually labeled groups of entries of a training set of motion data, wherein each labeled group of the training set of motion data is representative of a mapping from the entries in the labeled group to a movement;
identify, by the machine-learned model, a movement represented by the feature vector representation of the motion data, the identification determining a label to the feature vector describing the movement represented by the feature vector; and
generate a labeled representation of the sequence of motion data.

### Claim 10 (depends on 9)

The identity verification system of claim 9, further comprising:
a labeled motion processor configured to:
identify, from the labeled representation of the sequence of motion data, a plurality of identity blocks, each identity block representative of a different movement performed by the target user; and
from each identity block, encode a set of signature sequences into the feature vector representation of the identity block;
an identify computation module configured to input the feature vector representation of the identity block into a machine-learned identity confidence model to output an identity confidence value for the identity block, the identity confidence value descriptive of a confidence that the movement in the identity block was performed by the target user; and
a confidence evaluation module configured to grant the target user access to an operational context responsive to determining the identity confidence value is at least one of greater than or equal to an operational security threshold.

### Claim 11 (depends on 9)

The identity verification system of claim 9, further comprising a feature vector generator configured to:
receive, from at least one sensor, the sequence of motion data characterizing movements performed by the target user; and
encode the sequence of motion data into the feature vector representation of the motion data.

### Claim 12 (depends on 9)

The identity verification system of claim 9, wherein the identity verification system is further configured to:
encode a feature vector representation of the training set of motion data using a signal processing technique;
group entries of the feature vector representation of the training set of motion data; and
label each group of entries with a movement based on labels assigned by a manual operator.

### Claim 13 (depends on 9)

The identity verification system of claim 9, wherein the feature vector representation of the training set of motion data is a point cloud and the feature vector generator is further configured to:
generate, from the point cloud of the training set of motion data, a simplicial complex by expanding each point of the point cloud;
generate, from the simplicial complex, a persistent homology of the point cloud of the training set of motion data; and
apply a clustering algorithm to the point cloud of the training set of motion data to cluster points based on the persistent homology of the point cloud.

### Claim 14 (depends on 13)

The identity verification system of claim 13, wherein the clustering module is further configured to:
apply a clustering algorithm to the point cloud of the training set of motion data to generate a plurality of point clusters by clustering points based on a proximity to neighboring points of the point cloud of the training set of motion data;
receive, from a manual operator in response to presenting the plurality of point clusters to the manual operator, a label for each point cluster of the plurality, wherein the label identifies a movement represented by the point cluster; and
train the machine-learned model using the labeled point clusters of the training set of motion data.

### Claim 15 (depends on 9)

The identity verification system of claim 9, wherein the identity verification system is further configured to:
determine, responsive to inputting the feature vector to the machine-learned model, a movement label for the feature vector and a confidence value determined for the movement label;
classify, responsive to the confidence value being below a threshold confidence, the feature vector as an uncertain movement;
receive, from a manual operator, a label for the feature vector identifying the uncertain movement;
update the training set of motion data with the label received from the manual operator and the feature vector; and
re-train the machine-learned model using the updated training set of motion data.

### Claim 16 (depends on 9)

The identity verification system of claim 9, wherein the motion classifier module is further configured to:
determine a movement classifier score for the machine-learned model characterizing an accuracy of the machine-learned model, wherein the movement classifier score is determined based on the accuracy of labels assigned to each entry of a test set by the machine-learned model;
identify, responsive to the movement classifier score for the determined machine-learned model being less than a threshold score, an alternate classifier with a higher movement classifier score; and
implement the alternate classifier to identify the movement represented by the sequence of motion data.

### Claim 17

A method comprising:
receiving, from at least one sensor, a sequence of motion data characterizing movements performed by a target user, the sequence of motion data encoded as a feature vector representation of the motion data;
inputting the feature vector representation of the motion data into a machine-learned model trained based on manually labeled groups of entries of a training set of motion data, each labeled group of the training set representative of a mapping from the entries in the labeled group to a movement;
identifying, by the machine-learned model, a movement represented by the feature vector representation of the motion data, the identification determining a label to the feature vector describing the movement represented by the feature vector; and
generating a labeled representation of the sequence of motion data.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
identifying, from the labeled representation of the sequence of motion data, a plurality of identity blocks, wherein each identity block represents a different movement performed by the target user;
from each identity block, encoding a set of signature sequences into the feature vector representation of the identity block;
inputting the feature vector representation of the identity block into a machine-learned identity confidence model to output an identity confidence value for the identity block, wherein the identity confidence value describes a confidence that the movement in the identity block was performed by the target user; and
granting the target user access to an operational context responsive to determining the identity confidence value is greater than an operational security threshold.

### Claim 19 (depends on 17)

The method of claim 17, wherein the feature vector representation of the training data is a point cloud, the method further comprising:
generating, from the point cloud of the training set motion data, a simplicial complex by expanding each point of the point cloud;
generating, from the simplicial complex, a persistent homology of the point cloud of the training set of motion data; and
applying a clustering algorithm to the point cloud of the training set of motion data to cluster points based on the persistent homology of the point cloud.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
applying a clustering algorithm to the point cloud of the training set of motion data to generate a plurality of point clusters by clustering points based on a proximity to neighboring points of the point cloud of the training set of motion data;
receiving, from a manual operator in response to presenting the plurality of point clusters to the manual operator, a label for each point cluster of the plurality, wherein the label identifies a movement represented by the point cluster; and
training the machine-learned model using the labeled point clusters of the training set of motion data.

### Claim 21 (depends on 17)

The method of claim 17, further comprising:
determining, responsive to inputting the feature vector to the machine learned model, a movement label for the feature vector and a confidence value determined for the movement label;
classifying, responsive to the confidence value being below a threshold confidence, the feature vector as an uncertain movement;
receiving, from a manual operator, a label for the feature vector identifying the uncertain movement;
updating the training set of motion data with the label received from the manual operator and the feature vector; and
re-training the machine-learned model using the updated training set of motion data.

### Claim 22 (depends on 17)

The method of claim 17, further comprising:
determining a movement classifier score for the machine-learned model characterizing an accuracy of the machine-learned model, wherein the movement classifier score is determined based on the accuracy of labels assigned to each entry of a test set by the machine-learned model;
identifying, responsive to the movement classifier score for the determined machine-learned model being less than a threshold score, an alternate classifier with a higher movement classifier score; and
implementing the alternate classifier to identify the movement represented by the sequence of motion data.