# US 11,443,750 B2 · User authentication method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,750 B2
- Document ID: US11443750B2
- Status: Granted
- Kind: B2
- Application number: 16/700,264
- Filed: 2019-12-02
- Published: 2020-06-04
- Granted: 2022-09-13
- Earliest claimed: 2018-11-30
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Keunseok Cho; Jaeyoung Roh; Donghan Jang; Jiwon Hyung; Jaewon Lee
- Web page: https://patentgrep.com/patent/US11443750B2

## CPC

- G10L 17/20: Pattern transformations or operations aimed at increasing system robustness, e.g. against channel noise or different working conditions
- G10L 17/24: the user being prompted to utter a password or a predefined phrase
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/04: Segmentation; Word boundary detection
- G10L 17/06: Decision making techniques; Pattern matching strategies
- G10L 15/16: using artificial neural networks
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method and apparatus for authenticating a user based on an utterance input includes obtaining an input audio signal based on the utterance input of the user; obtaining, from the input audio signal, at least one audio signal of an utterance section and at least one audio signal of a non-utterance section; generating environment information indicating an environment in which the utterance input is received, based on the at least one audio signal of the non-utterance section; obtaining a result of a comparison between the generated environment information and registration environment information indicating an environment in which a registration utterance input corresponding to a previously registered registration audio signal corresponding to the user is received; adjusting an authentication criterion for authenticating the user based on the result of the comparison; and authenticating the user based on the adjusted authentication criterion and the input audio signal.

## Claims

### Claim 1

A user authentication method, performed by an electronic apparatus, of authenticating a user based on an utterance input, the user authentication method comprising:
obtaining an input audio signal based on the utterance input of the user;
obtaining, from the input audio signal, at least one audio signal of an utterance section and at least one audio signal of a non-utterance section, wherein the at least one audio signal of the utterance section is generated based on a speech of the user, and the at least one audio signal of the non-utterance section is not generated based on the speech of the user;
generating environment information indicating an environment in which the utterance input is received, based on the at least one audio signal of the non-utterance section;
obtaining a result of a comparison between the generated environment information and registration environment information indicating an environment in which a registration utterance input included in a previously registered registration audio signal of the user is received, wherein the registration audio signal comprises at least one audio signal of a registration utterance section generated based on a previous speech of the user;
adjusting an authentication criterion for authenticating the user based on the result of the comparison between the generated environment information and registration environment information; and
authenticating the user based on the adjusted authentication criterion and the input audio signal,
wherein the adjusting of the authentication criterion comprises:
calculating a first average energy value of the at least one audio signal of the utterance section, wherein the first average energy value is related to a distance between the user and the electronic apparatus;
calculating a second average energy value of the at least one audio signal of the registration utterance input; and
adjusting a threshold corresponding to the authentication criterion based on a result of a comparison between the first average energy value and the second average energy value.

### Claim 2 (depends on 1)

The user authentication method of claim 1, wherein the obtaining of the at least one audio signal of the utterance section and the at least one audio signal of the non-utterance section comprises:
splitting the input audio signal into a plurality of frames;
extracting an audio feature of the plurality of frames; and
based on the extracted audio feature, distinguishing frames corresponding to the utterance section from frames corresponding to the non-utterance section among the plurality of frames.

### Claim 3 (depends on 2)

The user authentication method of claim 2, wherein the generated environment information is generated based on an audio feature of the frames corresponding to the non-utterance section.

### Claim 4 (depends on 1)

The user authentication method of claim 1, wherein the generated environment information comprises information about a plurality of situations, information indicating a situation in which the input audio signal is received, and information about a plurality of vectors corresponding to the plurality of situations.

### Claim 5 (depends on 1)

The user authentication method of claim 1, wherein the authenticating of the user comprises:
obtaining the previously registered registration audio signal;
obtaining the audio signal of the registration utterance section from the registration audio signal; and
authenticating the user by comparing the at least one audio signal of the utterance section and the audio signal of the registration utterance section.

### Claim 6 (depends on 5)

The user authentication method of claim 5, wherein the registration audio signal comprises at least one audio signal of the registration utterance section and at least one audio signal of a registration non-utterance section, and
wherein the at least one audio signal of the registration non-utterance section is used to generate the registration environment information indicating a situation in which the utterance input corresponding to the registration audio signal is received.

### Claim 7 (depends on 6)

The user authentication method of claim 6, wherein the adjusting of the authentication criterion further comprises adjusting the threshold corresponding to a similarity between the at least one audio signal of the utterance section and the at least one audio signal of the registration utterance section.

### Claim 8 (depends on 7)

The user authentication method of claim 7, wherein the adjusting of the threshold comprises:
selecting the threshold from a preset threshold table based on a length of the utterance section and a length of the registration utterance section; and
adjusting the threshold based on the result of the comparison between the generated environment information and the registration environment information.

### Claim 9 (depends on 8)

The user authentication method of claim 8, wherein the selected threshold is adjusted based on a similarity between a vector corresponding to the generated environment information and a vector corresponding to the registration environment information.

### Claim 10 (depends on 8)

The user authentication method of claim 8, wherein the threshold is adjusted based on at least one of a parameter value previously set based on a timbre of a voice of the user or a parameter value previously set based on a characteristic of the electronic apparatus.

### Claim 11

An electronic apparatus comprising:
a microphone;
a memory; and
at least one processor,
wherein the at least one processor is configured to:
obtain an input audio signal based on an utterance input of a user received through the microphone,
distinguish, from the input audio signal, at least one audio signal of an utterance section and at least one audio signal of a non-utterance section, wherein the at least one audio signal of the utterance section is generated based on a speech of the user, and the at least one audio signal of the non-utterance section is not generated based on the speech of the user,
generate environment information indicating an environment in which the utterance input is received,
obtain a result of a comparison between the generated environment information and registration environment information indicating an environment in which a registration utterance input included in a previously registered registration audio signal of the user is received, wherein the registration audio signal comprises at least one audio signal of a registration utterance section generated based on a previous speech of the user,
based on a result of the comparison between the generated environment information and registration environment information, adjust an authentication criterion for authenticating the user, and
authenticate the user based on the adjusted authentication criterion and the input audio signal,
wherein the at least one processor is further configured to calculate a first average energy value of the at least one audio signal of the utterance section, wherein the first average energy value is related to a distance between the user and the electronic apparatus, calculate a second average energy value of the at least one audio signal of the registration utterance input and adjust a threshold corresponding to the authentication criterion based on a result of a comparison between the first average energy value and the second average energy value.

### Claim 12 (depends on 11)

The electronic apparatus of claim 11, wherein the at least one processor is further configured to:
split the input audio signal into a plurality of frames,
extract an audio feature of the plurality of frames, and
based on the extracted audio feature, distinguish frames corresponding to the utterance section from frames corresponding to the non-utterance section among the plurality of frames.

### Claim 13 (depends on 12)

The electronic apparatus of claim 12, wherein the at least one processor is further configured to generate the environment information based on an audio feature of the frames corresponding to the non-utterance section.

### Claim 14 (depends on 11)

The electronic apparatus of claim 11, wherein the at least one processor is further configured to:
obtain the previously registered registration audio signal,
obtain the audio signal of the registration utterance section from the registration audio signal, and
authenticate the user by comparing the at least one audio signal of the utterance section and the audio signal of the registration utterance section.

### Claim 15 (depends on 14)

The electronic apparatus of claim 14, wherein the registration audio signal comprises at least one audio signal of the registration utterance section and at least one audio signal of a registration non-utterance section, and
wherein the at least one audio signal of the registration non-utterance section is used to generate the registration environment information indicating a situation in which the utterance input corresponding to the registration audio signal is received.

### Claim 16 (depends on 15)

The electronic apparatus of claim 15, wherein the at least one processor is further configured to adjust the threshold corresponding to a similarity between the at least one audio signal of the utterance section and the at least one audio signal of the registration utterance section.

### Claim 17 (depends on 16)

The electronic apparatus of claim 16, wherein the at least one processor is further configured to:
select the threshold from a preset threshold table based on a length of the utterance section and a length of the registration utterance section, and
adjust the threshold based on the result of the comparison between the generated environment information and the registration environment information.

### Claim 18 (depends on 17)

The electronic apparatus of claim 17, wherein the at least one processor is further configured to adjust the threshold based on a similarity between a vector corresponding to the generated environment information and a vector corresponding to the registration environment information.

### Claim 19

A non-transitory computer-readable recording medium having recorded thereon a program for executing a user authentication method, the user authentication method comprising:
obtaining an input audio signal based on an utterance input of a user;
obtaining, from the input audio signal, at least one audio signal of an utterance section and at least one audio signal of a non-utterance section, wherein the at least one audio signal of the utterance section is generated based on a speech of a user, and the at least one audio signal of the non-utterance section is not generated based on the speech of the user;
generating environment information indicating an environment in which the utterance input is received, based on the at least one audio signal of the non-utterance section;
obtaining a result of a comparison between the generated environment information and registration environment information indicating an environment in which a registration utterance input included in a previously registered registration audio signal of the user is received, wherein the registration audio signal comprises at least one audio signal of a registration utterance section generated based on a previous speech of the user;
adjusting an authentication criterion for authenticating the user based on the result the comparison between the generated environment information and registration environment information; and
authenticating the user based on the adjusted authentication criterion and the input audio signal,
wherein the adjusting of the authentication criterion further comprises:
calculating a first average energy value of the at least one audio signal of the utterance section, wherein the first average energy value is related to a distance between the user and the electronic apparatus;
calculating a second average energy value of the at least one audio signal of the registration utterance input; and
adjusting a threshold corresponding to the authentication criterion based on a result of a comparison between the first average energy value and the second average energy value.

### Claim 20

A user authentication method, performed by an electronic apparatus, of authenticating a user based on an utterance input, the user authentication method comprising:
obtaining an input audio signal based on the utterance input of the user;
obtaining, from the input audio signal, at least one audio signal of an utterance section and at least one audio signal of a non-utterance section;
generating environment information indicating an environment in which the utterance input is received, based on the at least one audio signal of the non-utterance section;
obtaining a result of a comparison between the generated environment information and registration environment information indicating an environment in which a registration utterance input included in a previously registered registration audio signal of the user is received;
adjusting an authentication criterion for authenticating the user based on the result of the comparison; and
authenticating the user based on the adjusted authentication criterion and the input audio signal,
wherein the generating of the environment information comprises:
identifying an amount of noise in the environment is input due to a distance between the user and the electronic apparatus; and
generating the environment information based on the amount of noise in the environment;
wherein the obtaining of the result of the comparison comprises comparing a first amount of noise of the environment information and a second amount of noise of the registration environment information; and
wherein the adjusting of the authentication criterion comprises adjusting the authentication criterion for authenticating the user based on the result of the comparison between the first amount of noise and the second amount of noise.

### Claim 21

An electronic apparatus comprising:
a microphone;
a memory; and
at least one processor,
wherein the at least one processor is configured to:
obtain an input audio signal based on an utterance input of a user received through the microphone,
distinguish, from the input audio signal, at least one audio signal of an utterance section and at least one audio signal of a non-utterance section,
generate environment information indicating an environment in which the utterance input is received,
obtain a result of a comparison between the generated environment information and registration environment information indicating an environment in which a registration utterance input included in a previously registered registration audio signal of the user is received,
based on a result of the comparison, adjust an authentication criterion for authenticating the user, and
authenticate the user based on the adjusted authentication criterion and the input audio signal,
wherein the at least one processor is further configured to:
identify an amount of noise in the environment is input due to a distance between the user and the electronic apparatus;
generate the environment information based on the amount of noise in the environment;
compare a first amount of noise of the environment information and a second amount of noise of the registration environment information; and
adjust the authentication criterion for authenticating the user based on the result of the comparison between the first amount of noise and the second amount of noise.

### Claim 22 (depends on 1)

The method of claim 1, wherein the adjusting of the authentication criterion further comprises calculating an energy ratio between the first average energy value and the second average energy value, and
wherein the threshold is adjusted based on the energy ratio.