# US 2019/0172232 A1 · Method and system for sensor data recognition using data enrichment for the learning process

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0172232 A1
- Document ID: US20190172232A1
- Status: Published application
- Kind: A1
- Application number: 16/033,847
- Filed: 2018-07-12
- Published: 2019-06-06
- Earliest claimed: 2017-12-06
- Assignees: Samsung EletrôNica Da AmazôNia Ltda.; Universidade Federal De Minas Gerais - Ufmg
- Applicants: Samsung Eletrônica Da Amazônia Ltda.; Universidade Federal De Minas Gerais-Ufmg
- Inventors: Otávio A. B. Penatti; Milton F. S. Santos; Artur Jordão Lima Correia; William Robson Schwartz; Leonardo Antônio Borges Torres; Priscila Louise Ribeiro Aguirre
- Web page: https://patentgrep.com/patent/US20190172232A1

## CPC

- G06N 3/08: Learning methods
- G06T 11/26: Drawing of charts or graphs
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 7/00: Computing arrangements based on specific mathematical models
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method and system for the recognition of sensor data, like an accelerometer and a gyroscope. The method includes data/feature enrichment before a learning process. Thus, the learning process can benefit from better data, which includes both raw sensor data and enriched data/features. As a consequence, classification models may become less complex without harming accuracy and, therefore, can be more easily deployed to mobile devices, consuming less processing power and battery.

## Claims

### Claim 1

A method for recognition of sensor data using enriched data for a learning process based on mobile inertial sensors comprising:
receiving data from sensors as input;
enriching data/features from input raw sensor data;
learning features/classifier based on input raw data and on enriched data; and
outputting a classification information generated by the learned classifier.

### Claim 2 (depends on 1)

The method of claim 1, wherein input sensor data is obtained during a certain period of time.

### Claim 3 (depends on 1)

The method of claim 1, wherein the input sensor data comprises data from one of accelerometer, gyroscope, barometer, and magnetometer as inertial sensors.

### Claim 4 (depends on 1)

The method of claim 1, wherein the input sensor data is obtained from more than one device.

### Claim 5 (depends on 1)

The method of claim 1, wherein the input sensor data can be augmented by data augmentation.

### Claim 6 (depends on 1)

The method of claim 1, wherein the enriching data/features computes attitude estimation features and such attitude features are one of pitch, roll, and translation acceleration.

### Claim 7 (depends on 1)

The method of claim 1, wherein the enriching data/features generates images from sensor data.

### Claim 8 (depends on 7)

The method of claim 7, wherein the images generated from sensor data comprise recurrence plots, sensor signals in a matrix form and based on coefficients of autoregressive models.

### Claim 9 (depends on 7)

The method of claim 7, wherein the images generated by the enriching data/features are processed in order to obtain visual features to be used in a shallow learning process.

### Claim 10 (depends on 7)

The method of claim 7, wherein the images generated by the enriching data/features are processed in order to obtain visual features to be used in a deep learning process.

### Claim 11 (depends on 1)

The method of claim 1, wherein the enriching data/features to enrich input raw data uses autoregressive models.

### Claim 12 (depends on 1)

The method of claim 1, wherein the enriching data/features uses coefficients of bi-dimensional autoregressive models obtained from estimated wrist's attitude roll and pitch angles.

### Claim 13 (depends on 1)

The method of claim 1, wherein the enriching data/features uses time-domain autoregressive models

### Claim 14 (depends on 1)

The method of claim 1, wherein the enriching data/features uses phase plane autoregressive models.

### Claim 15 (depends on 1)

The method of claim 1, wherein the learning features/classifier is based on shallow classifiers, such classifiers comprising Support Vector Machine, Random Forest, Nearest Neighbors, Naïve Bayes and Neural Networks.

### Claim 16 (depends on 1)

The method of claim 1, wherein the learning features/classifier is based on deep classifiers, such classifiers comprising Convolutional Neural Networks, Recurrent Neural Networks, or other types of deep learning architectures.

### Claim 17 (depends on 1)

The method of claim 1, wherein results of the outputting classification information are stored in a user profile.

### Claim 18 (depends on 1)

The method of claim 1, wherein results of the outputting classification information are displayed to a user in a visual interface.

### Claim 19 (depends on 1)

A system for recognition of sensor data adapted to perform the method of claim 1 to obtain enriched data for learning based on mobile inertial sensors, the system comprising:
a data input module to receive data from inertial sensors stimulated by a user;
a feature/data enrichment module to enrich input raw data from the sensors;
a module to learn features and classifier based on the input raw and enriched data from the feature/data enrichment module; and
an output module to analyze the classification information generated by the learned classifier.