# US 11,908,453 B2 · Method and system for classifying a user of an electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,453 B2
- Document ID: US11908453B2
- Status: Granted
- Kind: B2
- Application number: 17/408,958
- Filed: 2021-08-23
- Published: 2022-08-11
- Granted: 2024-02-20
- Earliest claimed: 2021-02-10
- Assignees: Direct Cursus Technology L.L.C
- Applicants: Yandex Europe AG
- Inventors: Vladimir Andreevich Aliev; Stepan Aleksandrovich Kargaltsev; Artem Valerevich Babenko
- Web page: https://patentgrep.com/patent/US11908453B2

## CPC

- G10L 15/063: {Training}
- G06F 16/65: Clustering; Classification
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 17/04: Training, enrolment or model building
- G10L 17/18: Artificial neural networks; Connectionist approaches
- G06N 3/045: Combinations of networks
- G10L 2015/0638: {Interactive procedures}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method and a system for training a machine-learning algorithm (MLA) to determine a user class of a user of an electronic device are provided. The method comprises: receiving a training audio signal representative of a training user utterance; soliciting, by the processor, a plurality of assessor-generated labels for the training audio signal, the given one of the plurality of assessor-generated labels being indicative of whether the training user is perceived to be one of a first class and a second class; generating an amalgamated assessor-generated label for the training audio signal, the amalgamated assessor-generated label being indicative of a label distribution of the plurality of assessor-generated labels between the first class and the second class; generating a training set of data including the training audio signal and the amalgamated assessor-generated to train the MLA to determine the user class of the user producing an in-use user utterance.

## Claims

### Claim 1

A computer-implemented method for training a machine-learning algorithm (MLA) to determine a user class of a user producing a user utterance, the method being executable on a server including a processor, the method comprising:
receiving, by the processor, a training audio signal representative of a training user utterance,
the training user utterance having been produced by a training user;
soliciting, by the processor, a plurality of assessor-generated labels for the training audio signal,
a given one of the plurality of assessor-generated labels having been generated by a respective one of a plurality of human assessors;
the given one of the plurality of assessor-generated labels being indicative of whether the training user is perceived, by a respective human assessor, to be one of a first user class and a second user class;
generating, by the processor, an amalgamated assessor-generated label for the training audio signal,
the amalgamated assessor-generated label being indicative of a label distribution of the plurality of assessor-generated labels between the first user class and the second user class;
generating, by the processor, a training set of data including the training audio signal and the amalgamated assessor-generated label;
training, by the processor, based on the training set of data, the MLA to determine the user class of the user producing an in-use user utterance.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
segmenting, by the processor, the training audio signal into a plurality of training signal portions;
assigning, by the processor, the amalgamated assessor-generated label to each one of the plurality of training signal portions; and wherein the generating, by the processor, the training set of data comprises generating the training set of data based on the plurality of training signal portions.

### Claim 3 (depends on 2)

The method of claim 2, further comprising converting each one of the plurality of training signal portions in a respective time-frequency representation thereof.

### Claim 4 (depends on 3)

The method of claim 3, wherein the converting comprises applying, to each one of the plurality of training signal portions, a Fourier transform.

### Claim 5 (depends on 4)

The method of claim 4, wherein the converting further comprises generating, based on the respective time-frequency representation, a respective mel-frequency representation.

### Claim 6 (depends on 5)

The method of claim 5, wherein the respective mel-frequency representation includes a predetermined number of evenly spaced mel bands.

### Claim 7 (depends on 1)

The method of claim 1, wherein the amalgamating the plurality of assessor-generated labels comprises determining an average value thereof.

### Claim 8 (depends on 1)

The method of claim 1, further comprising, before soliciting the plurality of assessor-generated labels for the training audio signal, injecting noise into the training audio signal.

### Claim 9 (depends on 8)

The method of claim 8, wherein the injecting noise includes cancelling at least one predetermined frequency level of the training audio signal.

### Claim 10 (depends on 8)

The method of claim 8, wherein the injecting the noise includes modulating at least one predetermined frequency level of the training audio signal.

### Claim 11 (depends on 1)

The method of claim 1, further comprising using the MLA to determine the user class of the user producing the in-use user utterance, the in-use user utterance being captured by a speaker device communicatively coupled with the server, the using the MLA comprising:
generating, by the processor, based on the in-use user utterance, an in-use audio signal;
generating, by the processor, for the in-use audio signal, an in-use time-frequency representation thereof;
applying, by the processor, the MLA to the in-use time-frequency representation to generate a distribution parameter between likelihood values respectively indicative of the user producing the in-use user utterance being one of the first user class and the second user class, such that:
in response to the distribution parameter being equal to or greater than a predetermined distribution threshold, determining the user as being of the first user class; and
in response to the distribution parameter being lower than the predetermined distribution threshold, determining the user as being of the second user class.

### Claim 12 (depends on 11)

The method of claim 11, wherein the speaker device is configurable to operate in a first operation mode and a second operation mode, and the method further comprises:
in response to the determining the user as being of the first user class, causing the speaker device being in the second operation mode to switch into the first operation mode.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
the first operation mode is associated with retrieving, by the speaker device, content preselected for users of the first user class; and
the second operation mode is associated with retrieving, by the speaker device, content preselected for users of the second user class.

### Claim 14 (depends on 1)

The method of claim 1, wherein the first user class includes a child and the second user class includes an adult.

### Claim 15 (depends on 1)

The method of claim 1, wherein the MLA comprises a convolutional neural network.

### Claim 16

A server for training a machine-learning algorithm (MLA) to determine a user class of user producing a user utterance, the server including:
a processor;
a non-transitory computer-readable medium comprising instructions;
the processor, upon executing the instructions, being configured to:
receive a training audio signal representative of a training user utterance,
the training user utterance having been produced by a training user;
solicit a plurality of assessor-generated labels for the training audio signal,
a given one of the plurality of assessor-generated labels having been generated by a respective one of a plurality of human assessors;
the given one of the plurality of assessor-generated labels being indicative of whether the training user is perceived, by a respective human assessor, to be one of a first user class and a second user class;
generate an amalgamated assessor-generated label for the training audio signal,
the amalgamated assessor-generated label being indicative of a label distribution of the plurality of assessor-generated labels between the first user class and the second user class;
generate a training set of data including the training audio signal and the amalgamated assessor-generated label;
train, based on the training set of data, the MLA to determine the user class of the user producing an in-use user utterance.

### Claim 17 (depends on 16)

The server of claim 16, wherein the processor is further configured to:
segment the training audio signal into a plurality of training signal portions;
assign the amalgamated assessor-generated label to each one of the plurality of training signal portions; and
generate the training set of data based on the plurality of training signal portions.

### Claim 18 (depends on 17)

The server of claim 17, wherein the processor is further configured to convert each one of the plurality of training signal portions in a respective time-frequency representation thereof.

### Claim 19 (depends on 18)

The server of claim 18, wherein to convert each one of the plurality of training signal portions in the respective time-frequency representation thereof, the processor is configured to apply, to each one of the plurality of training signal portions, a Fourier transform.

### Claim 20 (depends on 16)

The server of claim 16, wherein the server is communicatively coupled, via a communication network, to a speaker device, and the processor is further configured to use the MLA to determine the user class of the user producing the in-use user utterance captured by the speaker device, by executing:
generating, based on the in-use user utterance, an in-use audio signal;
generating, for the in-use audio signal, an in-use time-frequency representation thereof;
applying the MLA to the in-use time-frequency representation to generate a distribution parameter between likelihood values respectively indicative of the user producing the in-use user utterance being one of the first user class and the second user class, such that:
in response to the distribution parameter being equal to or greater than a predetermined distribution threshold, determining the user as being of the first user class; and
in response to the distribution parameter being lower than the predetermined distribution threshold, determining the user as being of the second user class.