# US 2019/0356701 A1 · Media optimization of browser-based real-time communications applications in a virtual desktop environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0356701 A1 - Document ID: US20190356701A1 - Status: Published application - Kind: A1 - Application number: 15/982,073 - Filed: 2018-05-17 - Published: 2019-11-21 - Earliest claimed: 2018-05-17 - Assignees: VMware, Inc. - Applicants: VMware, Inc. - Inventors: Sonia Prabhu; Vertika Singh; Anurag Katiyar; Sumeet Vohra - Web page: https://patentgrep.com/patent/US20190356701A1 ## CPC - H04L 65/60: Network streaming of media packets - H04L 65/1069: Session establishment or de-establishment - H04L 65/1073: Registration or de-registration - H04L 65/1104: Session initiation protocol [SIP] - H04L 65/1108: Web based protocols, e.g. webRTC - H04L 65/4061: Push-to services, e.g. push-to-talk or push-to-video - H04L 65/65: Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP] - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Techniques are described for offloading the encoding and decoding of multimedia content transmitted during a real-time communication (RTC) session from the virtual desktop to the client device so that the multimedia content can be communicated directly between the client device and the remote peer device without the involvement of the virtual desktop. The offload eliminates the additional network hop of the multimedia content to the virtual desktop that is present in conventional virtual desktop environments, thereby reducing network latency and improving performance, CPU utilization and network load on the virtual machine hosting the virtual desktop. In order to offload the multimedia encoding/decoding, the techniques described herein intercept a number of Application Programming Interface (API) calls to override the messages used to establish a multimedia communication session between the RTC application and the remote peer device. ## Claims ### Claim 1 A method for optimizing web-based real-time communication applications in a virtual desktop environment, the method comprising: establishing a virtual desktop session between a client device and a virtual desktop hosted on a server over a network connection; receiving a request to launch a real-time communication (RTC) application on a web browser operating on the virtual desktop, wherein the RTC application is configured to communicate multimedia data with a remote peer device; intercepting a session initiation message sent from the RTC application; overriding the session initiation message by replacing at least a portion of content of the session initiation message with information obtained from the client device; and forwarding the session initiation message including the information obtained from the client device to the remote peer device, wherein based on the session initiation message, a secured channel is established directly between the remote peer device and the client device and the multimedia data is directly communicated over the secured channel thereby bypassing network transmission of the multimedia data to the virtual desktop. ### Claim 2 (depends on 1) The method of claim 1, wherein the RTC application is a WebRTC application, wherein the session initiation message is a Session Description Protocol (SDP) offer, and wherein the remote peer device responds with an SDP answer. ### Claim 3 (depends on 2) The method of claim 2, further comprising: overriding the SDP offer and the SDP answer with Interactive Connectivity Establishment (ICE) candidates, a fingerprint for Datagram Transport Layer Security (DTLS) handshake and capabilities from the client device. ### Claim 4 (depends on 2) The method of claim 2, wherein the session initiation message is intercepted by a browser extension on the web browser on the virtual desktop, the browser extension configured to request the information from the client device and override the session initiation message with the information obtained from the client device and forward the session initiation message to the WebRTC application. ### Claim 5 (depends on 4) The method of claim 4, wherein the browser extension is configured to intercept calls between the WebRTC application and WebRTC framework of the browser by overriding WebRTC peer connection application programming interfaces (APIs) to establish the secured channel between the client device and the remote peer device. ### Claim 6 (depends on 4) The method of claim 4, wherein the browser extension employs a native messaging host application that runs as a process on the virtual desktop and communicates with the browser extension using standard input (STDIN) and standard output (STDOUT) streams. ### Claim 7 (depends on 1) The method of claim 1, wherein the client device includes a media client plugin having access to a camera and one or more audio devices of the client device, wherein the media client plugin is configured to handle encoding of audio and video captured on the client device before transmitting the audio and video to the remote peer device. ### Claim 8 A computing device, comprising: at least one processor; and memory including instructions that, when executed by the at least one processor, cause the computing device to perform the steps of: establishing a virtual desktop session between a client device and a virtual desktop hosted on a server over a network connection; receiving a request to launch a real-time communication (RTC) application on a web browser operating on the virtual desktop, wherein the RTC application is configured to communicate multimedia data with a remote peer device; intercepting a session initiation message sent from the RTC application; overriding the session initiation message by replacing at least a portion of content of the session initiation message with information obtained from the client device; and forwarding the session initiation message including the information obtained from the client device to the remote peer device, wherein based on the session initiation message, a secured channel is established directly between the remote peer device and the client device and the multimedia data is directly communicated over the secured channel thereby bypassing network transmission of the multimedia data to the virtual desktop. ### Claim 9 (depends on 8) The computing device of claim 8, wherein the RTC application is a WebRTC application, wherein the session initiation message is a Session Description Protocol (SDP) offer, and wherein the remote peer device responds with an SDP answer. ### Claim 10 (depends on 9) The computing device of claim 9, wherein the memory further includes instructions that, when executed by the at least one processor, cause the computing device to perform the steps of: overriding the SDP offer and the SDP answer with Interactive Connectivity Establishment (ICE) candidates, a fingerprint for Datagram Transport Layer Security (DTLS) handshake and capabilities from the client device. ### Claim 11 (depends on 9) The computing device of claim 9, wherein the session initiation message is intercepted by a browser extension on the web browser on the virtual desktop, the browser extension configured to request the information from the client device and override the session initiation message with the information obtained from the client device and forward the session initiation message to the WebRTC application. ### Claim 12 (depends on 11) The computing device of claim 11, wherein the browser extension is configured to intercept calls between the WebRTC application and WebRTC framework of the browser by overriding WebRTC peer connection application programming interfaces (APIs) to establish the secured channel between the client device and the remote peer device. ### Claim 13 (depends on 8) The computing device of claim 8, wherein the client device includes a media client plugin having access to a camera and one or more audio devices of the client device, wherein the media client plugin is configured to handle encoding of audio and video captured on the client device before transmitting the audio and video to the remote peer device. ### Claim 14 A non-transitory computer readable storage medium comprising one or more sequences of instructions, the instructions when executed by one or more processors causing the one or more processors to execute the operations of: establishing a virtual desktop session between a client device and a virtual desktop hosted on a server over a network connection; receiving a request to launch a real-time communication (RTC) application on a web browser operating on the virtual desktop, wherein the RTC application is configured to communicate multimedia data with a remote peer device; intercepting a session initiation message sent from the RTC application; overriding the session initiation message by replacing at least a portion of content of the session initiation message with information obtained from the client device; and forwarding the session initiation message including the information obtained from the client device to the remote peer device, wherein based on the session initiation message, a secured channel is established directly between the remote peer device and the client device and the multimedia data is directly communicated over the secured channel thereby bypassing network transmission of the multimedia data to the virtual desktop. ### Claim 15 (depends on 14) The non-transitory computer readable storage medium of claim 14, wherein the RTC application is a WebRTC application, wherein the session initiation message is a Session Description Protocol (SDP) offer, and wherein the remote peer device responds with an SDP answer. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, further comprising instructions which when executed by one or more processors cause the one or more processors to execute the operations of: overriding the SDP offer and the SDP answer with Interactive Connectivity Establishment (ICE) candidates, a fingerprint for Datagram Transport Layer Security (DTLS) handshake and capabilities from the client device. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the session initiation message is intercepted by a browser extension on the web browser on the virtual desktop, the browser extension configured to request the information from the client device and override the session initiation message with the information obtained from the client device and forward the session initiation message to the WebRTC application. ### Claim 18 (depends on 17) The non-transitory computer readable storage medium of claim 17, wherein the browser extension is configured to intercept calls between the WebRTC application and WebRTC framework of the browser by overriding WebRTC peer connection application programming interfaces (APIs) to establish the secured channel between the client device and the remote peer device. ### Claim 19 (depends on 17) The non-transitory computer readable storage medium of claim 17, wherein the browser extension employs a native messaging host application that runs as a process on the virtual desktop and communicates with the browser extension using standard input (STDIN) and standard output (STDOUT) streams. ### Claim 20 (depends on 14) The non-transitory computer readable storage medium of claim 14, wherein the client device includes a media client plugin having access to a camera and one or more audio devices of the client device, wherein the media client plugin is configured to handle encoding of audio and video captured on the client device before transmitting the audio and video to the remote peer device.