# US 11,442,601 B2 · Computer desktop that dynamically adapts to a live multi-channel audio stream, such as for use with wireless telecommunications customer service agents > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,601 B2 - Document ID: US11442601B2 - Status: Granted - Kind: B2 - Application number: 16/905,716 - Filed: 2020-06-18 - Published: 2021-12-23 - Granted: 2022-09-13 - Earliest claimed: 2020-06-18 - Assignees: T-Mobile USA, Inc. - Applicants: T-Mobile USA, Inc. - Inventors: Jonathan Soini - Web page: https://patentgrep.com/patent/US11442601B2 ## CPC - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 9/547: {Remote procedure calls [RPC]; Web services} - G06F 16/632: Query formulation - G06N 5/04: Inference or reasoning models - G06Q 30/016: After-sales - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/08: Speech classification or search - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04M 3/5133: {Operator terminal details} - H04M 3/5183: {Call or contact centers with computer-telephony arrangements} - H04S 1/00: Two-channel systems (H04S5/00, H04S7/00 take precedence) - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06N 20/00: Machine learning - G10L 2015/088: {Word spotting} - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04S: STEREOPHONIC SYSTEMS ## Abstract The disclosed technology includes at least one method performed by a system to dynamically adapt a computer desktop of a customer service agent to a live audio dialogue. The system can cause a speech feature analyzer to process a segment of the live audio dialogue to output multiple speech features and obtain search results by querying a database for the multiple speech features. The method can further include generating control signals based on the search results and cause an API to configure the computer desktop based on the control signals. The system can dynamically adapt the computer desktop in accordance with additional control signals that are generated based on next segments of the live audio dialogue. ## Claims ### Claim 1 A method performed by a system to dynamically adapt a computer desktop of a customer service agent to a live audio dialogue, wherein the customer service agent assists wireless telecommunications customers, the method comprising: causing a speech feature analyzer to process a segment of a two-channel audio signal including the live audio dialogue between a customer and the agent in accordance with a speech feature model to output multiple speech features, wherein the two-channel audio signal includes a first channel for customer speech and a second channel for agent speech, and wherein the customer is a subscriber of a wireless telecommunications network; obtaining one or more search results by querying a database for the multiple speech features, wherein the search results include multiple content items that are each relevant to the live audio dialogue; generating one or more control signals based on the search results, wherein the control signals are configured to dynamically control the multiple content items and placement of the multiple content items on the dynamic desktop during the live audio dialogue; causing an application programming interface (API) to configure the computer desktop based on the one or more control signals during the live audio dialogue between the customer and the agent, wherein the computer desktop includes multiple tabs and associated windows that are ordered such that any content items of a frontmost tab are more relevant to the segment of the two-channel audio signal compared to any content items of any remaining tabs; and dynamically adapting the computer desktop in accordance with one or more additional control signals that are generated based on a next segment of the live audio dialogue, wherein the computer desktop is adapted to the next segment of the live audio dialogue by adding or removing one of the multiple tabs, rearranging the order of the multiple tabs, or changing content of the multiple tabs. ### Claim 2 (depends on 1) The method of claim 1, wherein the speech feature analyzer includes a speech-to-text analyzer that extracts multiple keywords from two-channel audio signal, the method further comprising, prior to querying a database for search results: predicting a customer inquiry based on the multiple keywords; generating a search term based on the customer inquiry; and adding the search term to the search query. ### Claim 3 (depends on 1) The method of claim 1, wherein causing the speech feature analyzer to process the segment of the two-channel audio signal comprises: labeling at least some of the multiple search features as telecommunications terms, wherein any search feature that is labeled as a telecommunications term is weighted more than any search feature that is not labeled as a telecommunications term such that the search results are biased for telecommunications terms. ### Claim 4 (depends on 1) The method of claim 1, wherein causing the speech feature analyzer to process the segment of the two-channel audio signal comprises: labeling each of the multiple search features as either customer speech or agent speech, wherein any search feature that is labeled as customer speech is weighted more than any search feature that is labeled as agent speech such that the search results are biased for customer speech. ### Claim 5 (depends on 1) The method of claim 1, wherein the search results include, in addition to the multiple content items, a software tool, software application, or notification that is adapted to the segment of the live audio dialogue. ### Claim 6 (depends on 1) The method of claim 1 further comprising, prior to causing the API to configure the computer desktop: ranking the multiple content items based on a relevancy score, wherein the control signals are configured to cause the computer desktop to display only content items with a relevancy score that exceeds a threshold. ### Claim 7 (depends on 1) The method of claim 1 further comprising: periodically querying the database for additional speech features that are extracted from additional segments of the live audio dialogue; and in response to periodically querying the database, periodically obtaining one or more search results for dynamically adapting the computer desktop in accordance with additional control signals. ### Claim 8 (depends on 1) The method of claim 1 further comprising: continuously querying the database for additional speech features that are continuously generated from the live audio dialogue; and continuously generating control signals based on search results that are continuously retrieved. ### Claim 9 (depends on 1) The method of claim 1 further comprising: collecting input parameters indicative of an interaction by the agent with the computer desktop; generating a feedback signal based on the input parameters; and biasing a search engine that outputs search results based on speech features. ### Claim 10 (depends on 1) The method of claim 1, wherein a third-party service provider provides the speech feature analyzer to output the multiple speech features. ### Claim 11 (depends on 1) The method of claim 1, wherein the search results include multiple software tools, the method further comprising: causing display, on the computer desktop, of any of the multiple content items that exceed a first threshold; and causing launching, on the computer desktop, of any of the multiple software tools that exceed a second threshold. ### Claim 12 (depends on 1) The method of claim 1, wherein the one or more search results are ranked based on a statistical characteristic of the customer, a historical characteristic of the customer, an indication of a customer device, or service plan subscribed to by the customer.