# US 11,908,456 B2 · Azimuth estimation method, device, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,456 B2 - Document ID: US11908456B2 - Status: Granted - Kind: B2 - Application number: 17/006,440 - Filed: 2020-08-28 - Published: 2020-12-17 - Granted: 2024-02-20 - Earliest claimed: 2018-08-06 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Jimeng Zheng; Yi Gao; Meng Yu; Ian Ernan Liu - Web page: https://patentgrep.com/patent/US11908456B2 ## CPC - G10L 15/08: Speech classification or search - G01S 3/8006: {Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L 25/21: the extracted parameters being power information - G10L 2015/088: {Word spotting} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract Embodiments of this application discloses an azimuth estimation method performed at a computing device, the method including: obtaining, in real time, multi-channel sampling signals and buffering the multi-channel sampling signals; performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals; performing a spatial spectrum estimation on the buffered multi-channel sampling signals to obtain a spatial spectrum estimation result, when the wakeup word detection scores of the one or more sampling signals indicates that a wakeup word exists in the one or more sampling signals; and determining an azimuth of a target voice associated with the multi-channel sampling signals according to the spatial spectrum estimation result and a highest wakeup word detection score, thereby improving the accuracy of the azimuth estimation in a voice interaction process. ## Claims ### Claim 1 An azimuth estimation method performed at a computing device having one or more processors and memory storing a plurality of computer programs to be executed by the processor, the method comprising: obtaining, in real time, multi-channel sampling signals and buffering the multi-channel sampling signals, wherein the multi-channel sampling signals are respectively obtained through N1 channels corresponding to N1 beams in N1 different directions; performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals; computing, upon a determination that the wakeup word detection scores of the one or more sampling signals indicates that a wakeup word exists in the one or more sampling signals, a time period from appearance to disappearance of the wakeup word based on the wakeup word detection scores, further including: determining a time point of the appearance of the wakeup word as a starting time point at which a wakeup word detection score starts changing; determining a time point of the disappearance of the wakeup word as an ending time point corresponding to the highest wakeup word detection score among the wakeup word detection scores; and determining the time period from the appearance to the disappearance of the wakeup word according to the starting time point and the ending time point; extracting a target sampling signal within the time period from the buffered multi-channel sampling signals; performing a spatial spectrum estimation on the target sampling signal extracted within the time period to obtain a spatial spectrum estimation result indicating signal power strengths at N2 candidate azimuths, wherein N2 is different from N1; and determining, among the N2 candidate azimuths, an azimuth of a target voice associated with the multi-channel sampling signals based on the spatial spectrum estimation result and a highest wakeup word detection score. ### Claim 2 (depends on 1) The method according to claim 1, wherein the performing the spatial spectrum estimation on the target sampling signal to obtain the spatial spectrum estimation result comprises: calculating the signal power strengths at the N2 candidate azimuths based on the target sampling signal, wherein N2 is larger than N1. ### Claim 3 (depends on 2) The method according to claim 2, wherein the determining an azimuth of a target voice associated with the multi-channel sampling signals based on the spatial spectrum estimation result and a highest wakeup word detection score comprises: determining an azimuth of a target main beam, the target main beam being a main beam, among the N1 beams, of a sampling signal corresponding to the highest wakeup word detection score; determining at least one local maximum value point among the signal power strengths at the N2 candidate azimuths; and determining the azimuth of the target voice based on the azimuth of the target main beam and the at least one local maximum value point. ### Claim 4 (depends on 3) The method according to claim 3, wherein the determining the azimuth of the target voice based on the azimuth of the target main beam and the local maximum value point comprises: determining, among the N2 candidate azimuths, a candidate azimuth corresponding to a local maximum value point that is closest to the azimuth of the target main beam among the at least one local maximum value point; and determining the candidate azimuth as the azimuth of the target voice. ### Claim 5 (depends on 3) The method according to claim 3, wherein the determining the azimuth of the target voice based on the azimuth of the target main beam and the at least one local maximum value point comprises: when there are at least two local maximum value points closest to the azimuth of the target main beam, determining an average value of candidate azimuths respectively corresponding to the at least two local maximum value points as the azimuth of the target voice. ### Claim 6 (depends on 1) The method according to claim 1, wherein during the performing the spatial spectrum estimation, the method further comprises: stopping the wakeup word detection on the one or more sampling signals in the multi-channel sampling signals within a duration from determination of the existence of the wakeup word to reappearance of the wakeup word. ### Claim 7 (depends on 1) The method according to claim 1, wherein the performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals comprises: performing the wakeup word detection on the one or more sampling signals in the multi-channel sampling signals, and determining a confidence of a wakeup word of each channel of the one or more sampling signals, the confidence being a similarity between content in the sampling signal and a preconfigured wakeup word; and determining the wakeup word detection score for each channel of the one or more sampling signals according to the confidence of the wakeup word of the sampling signal. ### Claim 8 (depends on 1) The method according to claim 1, further comprising: determining that the wakeup word exists when the wakeup word detection score of any channel of the one or more sampling signals is greater than a score threshold. ### Claim 9 (depends on 1) The method according to claim 1, further comprising: reserving sampling signals within a latest duration (M+N), and deleting sampling signals beyond the duration (M+N) in the buffered multi-channel sampling signals, M being an occupation duration of the wakeup word, and N being a preset duration. ### Claim 10 A computing device, comprising one or more processors, memory and a plurality of computer programs stored in the memory that, when executed by the one or more processors, cause the computing device to perform a plurality of operations including: obtaining, in real time, multi-channel sampling signals and buffering the multi-channel sampling signals, wherein the multi-channel sampling signals are respectively obtained through N1 channels corresponding to N1 beams in N1 different directions; performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals; computing, upon a determination that the wakeup word detection scores of the one or more sampling signals indicates that a wakeup word exists in the one or more sampling signals, a time period from appearance to disappearance of the wakeup word based on the wakeup word detection scores, further including: determining a time point of the appearance of the wakeup word as a starting time point at which a wakeup word detection score starts changing; determining a time point of the disappearance of the wakeup word as an ending time point corresponding to the highest wakeup word detection score among the wakeup word detection scores; and determining the time period from the appearance to the disappearance of the wakeup word according to the starting time point and the ending time point; extracting a target sampling signal within the time period from the buffered multi-channel sampling signals; performing a spatial spectrum estimation on the target sampling signal extracted within the time period to obtain a spatial spectrum estimation result indicating signal power strengths at N2 candidate azimuths, wherein N2 is different from N1; and determining, among the N2 candidate azimuths, an azimuth of a target voice associated with the multi-channel sampling signals based on the spatial spectrum estimation result and a highest wakeup word detection score. ### Claim 11 (depends on 10) The computing device according to claim 10, wherein the performing the spatial spectrum estimation on the target sampling signal to obtain the spatial spectrum estimation result comprises: calculating the signal power strengths at the N2 candidate azimuths based on the target sampling signal, wherein N2 is larger than N1. ### Claim 12 (depends on 11) The computing device according to claim 11, wherein the determining an azimuth of a target voice associated with the multi-channel sampling signals based on the spatial spectrum estimation result and a highest wakeup word detection score comprises: determining an azimuth of a target main beam, the target main beam being a main beam, among the N1 beams, of a sampling signal corresponding to the highest wakeup word detection score; determining at least one local maximum value point among the signal power strengths at the N2 candidate azimuths; and determining the azimuth of the target voice based on the azimuth of the target main beam and the at least one local maximum value point. ### Claim 13 (depends on 10) The computing device according to claim 10, wherein during the performing the spatial spectrum estimation, the plurality of operations further comprise: stopping the wakeup word detection on the one or more sampling signals in the multi-channel sampling signals within a duration from determination of the existence of the wakeup word to reappearance of the wakeup word. ### Claim 14 (depends on 10) The computing device according to claim 10, wherein the performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals comprises: performing the wakeup word detection on the one or more sampling signals in the multi-channel sampling signals, and determining a confidence of a wakeup word of each channel of the one or more sampling signals, the confidence being a similarity between content in the sampling signal and a preconfigured wakeup word; and determining the wakeup word detection score for each channel of the one or more sampling signals according to the confidence of the wakeup word of the sampling signal. ### Claim 15 (depends on 10) The computing device according to claim 10, wherein the plurality of operations further comprise: determining that the wakeup word exists when the wakeup word detection score of any channel of the one or more sampling signals is greater than a score threshold. ### Claim 16 A non-transitory computer-readable storage medium storing a plurality of computer programs that, when executed by one or more processors of a computing device, cause the computing device to perform a plurality of operations including: obtaining, in real time, multi-channel sampling signals and buffering the multi-channel sampling signals, wherein the multi-channel sampling signals are respectively obtained through N1 channels corresponding to N1 beams in N1 different directions; performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals; computing, upon a determination that the wakeup word detection scores of the one or more sampling signals indicates that a wakeup word exists in the one or more sampling signals, a time period from appearance to disappearance of the wakeup word based on the wakeup word detection scores, further including: determining a time point of the appearance of the wakeup word as a starting time point at which a wakeup word detection score starts changing; determining a time point of the disappearance of the wakeup word as an ending time point corresponding to the highest wakeup word detection score among the wakeup word detection scores; and determining the time period from the appearance to the disappearance of the wakeup word according to the starting time point and the ending time point; extracting a target sampling signal within the time period from the buffered multi-channel sampling signals; performing a spatial spectrum estimation on the target sampling signal extracted within the time period to obtain a spatial spectrum estimation result indicating signal power strengths at N2 candidate azimuths, wherein N2 is different from N1; and determining, among the N2 candidate azimuths, an azimuth of a target voice associated with the multi-channel sampling signals based on the spatial spectrum estimation result and a highest wakeup word detection score. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium according to claim 16, wherein the plurality of operations further comprise: during the performing the spatial spectrum estimation, stopping the wakeup word detection on the one or more sampling signals in the multi-channel sampling signals within a duration from determination of the existence of the wakeup word to reappearance of the wakeup word. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium according to claim 16, wherein the performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals comprises: performing the wakeup word detection on the one or more sampling signals in the multi-channel sampling signals, and determining a confidence of a wakeup word of each channel of the one or more sampling signals, the confidence being a similarity between content in the sampling signal and a preconfigured wakeup word; and determining the wakeup word detection score for each channel of the one or more sampling signals according to the confidence of the wakeup word of the sampling signal. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium according to claim 16, wherein the plurality of operations further comprise: determining that the wakeup word exists when the wakeup word detection score of any channel of the one or more sampling signals is greater than a score threshold. ### Claim 20 (depends on 16) The non-transitory computer-readable storage medium according to claim 16, wherein the plurality of operations further comprise: reserving sampling signals within a latest duration (M+N), and deleting sampling signals beyond the duration (M+N) in the buffered multi-channel sampling signals, M being an occupation duration of the wakeup word, and N being a preset duration.