# US 11,308,936 B2 · Speech signal processing method and speech signal processing apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,936 B2 - Document ID: US11308936B2 - Status: Granted - Kind: B2 - Application number: 16/399,211 - Filed: 2019-04-30 - Published: 2019-08-22 - Granted: 2022-04-19 - Earliest claimed: 2014-11-07 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Tae-yoon Kim; Sang-ha Kim; Sung-Soo Kim; Jin-sik Lee; Chang-woo Han; Eun-kyoung Kim; Jae-won Lee - Web page: https://patentgrep.com/patent/US11308936B2 ## CPC - G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes} - G10L 15/063: {Training} - G10L 15/04: Segmentation; Word boundary detection - G10L 15/18: using natural language modelling - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems - G10L 2015/0635: {updating or merging of old and new templates; Mean values; Weighting} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A speech signal processing method of a user terminal includes: receiving a speech signal, detecting a personalized information section including personal information in the speech signal, performing data processing on the personalized information section of the speech signal by using a personalized model generated based on the personal information, and receiving, from a server, a result of the data processing performed by the server on a general information section of the speech signal that is different than the personalized information section of the speech signal. ## Claims ### Claim 1 A speech signal processing method of a user terminal, comprising: generating a mapping table by mapping an identification (ID) to personal information; generating an ID-based personalized model by using the mapping table, wherein the ID is an acoustic unit ID mapped to an acoustic unit, the acoustic unit ID represents a specific portion of a speech recognition (SR) model corresponding to a phonetic symbol or pronunciation, and the ID-based personalized model is the SR model; transmitting the ID-based personalized model to a server; receiving, from the server, a result of data processing performed on a speech signal by using the ID-based personalized model; and restoring the personal information corresponding to the acoustic unit ID by using a result of the data processing and the mapping table. ### Claim 2 (depends on 1) The speech signal processing method of claim 1, wherein the acoustic unit ID comprises an ID mapped to the phonetic symbol or the pronunciation according to prearrangement with the server. ### Claim 3 (depends on 1) The speech signal processing method of claim 1, further comprising generating the mapping table by mapping the acoustic unit ID to additional information generated from the personal information. ### Claim 4 (depends on 1) The speech signal processing method of claim 1, wherein generating the mapping table by mapping an ID to additional information generated from the personal information comprises: generating a phonetic dictionary by using a word list from the personal information; and mapping arbitrary IDs to at least one of the phonetic symbol and words in the word list by using the phonetic dictionary. ### Claim 5 (depends on 1) The speech signal processing method of claim 1, further comprising: obtaining the speech signal; and transmitting the obtained speech signal to the server. ### Claim 6 A user terminal comprising: a communication unit configured to communicate with a server; and a control unit configured to: generate a mapping table by associating personal information with an ID, generate an ID-based personalized model by using the mapping table, wherein the ID is an acoustic unit ID mapped to an acoustic unit, the acoustic unit ID represents a specific portion of a speech recognition (SR) model corresponding to a phonetic symbol or pronunciation, and the ID-based personalized model is the SR model, control the communication unit to transmit the ID-based personalized model the server, and restore the personal information corresponding to the ID by using the mapping table and a result of data processing that is performed on a speech signal by using the ID-based personalized model and received from the server. ### Claim 7 (depends on 6) The user terminal of claim 6, wherein the control unit is configured to perform control such that the mapping table is generated by mapping the acoustic unit ID to additional information generated from the personal information. ### Claim 8 (depends on 6) The user terminal of claim 6, wherein the control unit is further configured to: generate a phonetic dictionary by using a word list from the personal information; and map arbitrary IDs to at least one of the phonetic symbol and words in the word list by using the phonetic dictionary. ### Claim 9 (depends on 6) The user terminal of claim 6, wherein the control unit is further configured to: obtain the speech signal; and transmit the obtained speech signal to the server. ### Claim 10 (depends on 6) The user terminal of claim 6, wherein the acoustic unit ID comprises an ID mapped to the phonetic symbol or the pronunciation according to prearrangement with the server. ### Claim 11 A speech signal processing method of a server, comprising: receiving an ID-based personalized model from a user terminal; receiving a speech signal; performing data processing on the speech signal by using the ID-based personalized model, wherein the performing comprises representing an ID, wherein the ID is an acoustic unit ID mapped to an acoustic unit, the acoustic unit ID represents a specific portion of a speech recognition (SR) model corresponding to a phonetic symbol or pronunciation, and the ID-based personalized model is the SR model; and transmitting a result of the data processing to the user terminal. ### Claim 12 (depends on 11) The speech signal processing method of claim 11, wherein the performing of the data processing on the speech signal by using the ID-based personalized model, the acoustic unit ID is mapped to personal information. ### Claim 13 (depends on 12) The speech signal processing method of claim 12, wherein the acoustic unit ID is mapped to the phonetic symbol or the pronunciation according to a prearrangement with the server.