# US 2014/0207442 A1 · Protection of Private Information in a Client/Server Automatic Speech Recognition System > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0207442 A1 - Document ID: US20140207442A1 - Status: Published application - Kind: A1 - Application number: 13/748,878 - Filed: 2013-01-24 - Published: 2014-07-24 - Earliest claimed: 2013-01-24 - Assignees: Nuance Communications, Inc. - Applicants: Nuance Communications, Inc. - Inventors: William F. Ganong, III; Paul J. Vozila - Web page: https://patentgrep.com/patent/US20140207442A1 ## CPC - H04W 12/02: Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII] - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G10L 15/1822: {Parsing for meaning understanding} - G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems - G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence) - H04M 2201/40: using speech recognition - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract A mobile device is adapted for protecting private information on the mobile device in a hybrid automatic speech recognition arrangement. The mobile device includes a speech input component for receiving a speech input signal from a user. Additionally, the mobile device includes a local ASR arrangement for performing local ASR processing of the speech input signal and determining if private information is included within the speech input signal. A control unit on the mobile device obscures private information in the speech input signal if the local ASR arrangement identifies information within a speech recognition result as private information. The control unit releases the speech input signal with the obscured private information for transmission to a remote server for further ASR processing. Results from the remote server's ASR processing are integrated and combined with results from local ASR processing to display information on the mobile device. ## Claims ### Claim 1 A mobile device adapted for automatic speech recognition for protecting private information on the mobile device comprising: a speech input component for receiving a speech input signal from a user; a local ASR arrangement for performing local ASR processing of the speech input signal and determining if private information is included within the speech input signal; a control unit for obscuring private information in the speech input signal if the local ASR arrangement identifies private information and for sending the speech input signal with the obscured private information to a remote server for ASR processing. ### Claim 2 (depends on 1) A mobile device according to claim 1, wherein the control unit creates a hash code of identified private information and inserts the hash code into the speech input signal prior to sending the speech input signal to the remote server. ### Claim 3 (depends on 1) A mobile device according to claim 1, further including: a user interface allowing a user to identify private information not to be transmitted to a remote server. ### Claim 4 (depends on 1) A mobile device according to claim 1 wherein the control unit obscures the private information by removing a portion of the speech input signal associated with the private information prior to transmitting the speech input signal to the remote server. ### Claim 5 (depends on 2) A mobile device according to claim 2, wherein the control unit receives a recognized version of the speech input including an indicia of the hash code and the control unit inserts the private information into the recognized version of the speech input at the indicia. ### Claim 6 A mobile device wherein the control unit obscures the private information by identifying the private information within the speech input signal and replacing the private information with a different signal. ### Claim 7 (depends on 1) A mobile device according to claim 1 wherein a log including speech processing events is sent to the remote server by the control unit. ### Claim 8 (depends on 7) A mobile device according to claim 7 wherein the log includes a text presentation of the speech input signal as recognized by the local ASR and any private information within the log is obscured with a hash code. ### Claim 9 (depends on 1) A mobile device according to claim 1, wherein if the private information identified by the local ASR is contact information the control unit accesses a contact list on the mobile device and transmits only a portion of the contact list to the server. ### Claim 10 (depends on 10) A mobile device according to claim 10 wherein the portion of the contact list includes last names without first names. ### Claim 11 A mobile device adapted for automatic speech recognition for protecting private information on the mobile device comprising: a speech input for receiving a speech input signal from a user; a local ASR arrangement for performing local ASR processing of the speech input signal using identified private information sources on the mobile device; wherein if private information is identified in the result of the local ASR processing of the speech input signal, the control unit declassifies the private information; and a control unit for sending the speech input signal and declassified private information from the identified private information sources to a remote server for ASR processing. ### Claim 12 (depends on 11) A mobile device according to claim 11, wherein the identified private information sources include an address book. ### Claim 13 (depends on 11) A mobile device according to claim 11, wherein the result of the local ASR processing may include an N-best list and the declassified private information may include words or names in the N-best list and associated data. ### Claim 14 (depends on 11) A mobile device according to claim 11 wherein a subset of data from the private information sources is sent to the remote server for ASR processing. ### Claim 15 (depends on 14) A mobile device according to claim 14 wherein the subset of data may include a limited number of fields from a private information source database. ### Claim 16 A method for protecting private information on a mobile device during automatic speech recognition, the method comprising: performing automatic speech recognition on an unknown speech signal on the mobile device producing a local speech recognition result; identifying private information within the local speech recognition result based on a comparison with data within a local data store on the mobile device; obscuring private information within the speech signal; transmitting the speech signal with the obscured private information to server; receiving a remote speech recognition result from the server; and outputting a combined speech recognition result based upon the local speech recognition result and the remote speech recognition result, wherein the private information is provided in the combined speech recognition result. ### Claim 17 (depends on 16) A method according to claim 16, further comprising: creating a speech log including a speech recognition result. ### Claim 18 (depends on 18) A method according to claim 18, further comprising: obscuring private information within the speech log prior to transmitting the speech log to the server. ### Claim 19 (depends on 18) A method according to claim 18, wherein the combined speech recognition result is output to a display device providing one or more possible speech recognition results based upon probability data from both the local speech recognition result and the remote speech recognition result. ### Claim 20 A method for protecting private information on a mobile device during automatic speech recognition, the method comprising: performing automatic speech recognition on an unknown speech signal on the mobile device producing a local speech recognition result; identifying private contact information within the local speech recognition result based on a comparison with a contact list database within a local data store on the mobile device; obscuring a first portion of the private contact information within the speech signal; transmitting the speech signal with a second portion of the private contact information to a server, wherein the first portion of the private contact information is obscured; receiving a remote speech recognition result from the server; and outputting a combined speech recognition result based upon the local speech recognition result and the remote speech recognition result, wherein the private contact information is provided in the combined speech recognition result. ### Claim 21 (depends on 20) A method according to claim 20 further comprising: providing a user interface for selection of the second portion of the private contact information to transmit.