# US 2019/0379735 A1 · Method and apparatus for multimedia communication, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379735 A1
- Document ID: US20190379735A1
- Status: Published application
- Kind: A1
- Application number: 16/547,376
- Filed: 2019-08-21
- Published: 2019-12-12
- Earliest claimed: 2017-07-18
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Liuwen HUANG; Di Xue
- Web page: https://patentgrep.com/patent/US20190379735A1

## CPC

- H04L 9/3268: {using certificate validation, registration, distribution or revocation, e.g. certificate revocation list [CRL]}
- H04L 9/0819: {Key transport or distribution, i.e. key establishment techniques where one party creates or otherwise obtains a secret value, and securely transfers it to the other(s) (network architectures or network communication protocols for key distribution in a packet data network H04L63/062)}
- H04L 9/0825: {using asymmetric-key encryption or public key infrastructure [PKI], e.g. key signature or public key certificates}
- H04L 9/0841: {involving Diffie-Hellman or related key agreement protocols}
- H04L 9/14: using a plurality of keys or algorithms
- H04L 9/3263: {involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements (network architectures or network communication protocols for supporting authentication of entities using certificates in a packet data network H04L63/0823)}
- H04L 63/0823: {using certificates (cryptographic mechanisms or cryptographic arrangements for entity authentication involving certificates H04L9/3263)}
- H04L 65/1069: Session establishment or de-establishment
- H04L 65/1104: Session initiation protocol [SIP]
- H04L 65/80: Responding to QoS
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/1091: {Interfacing with client-server systems or between P2P systems}
- H04L 67/141: Setup of application sessions (admission control or resource allocation in data switching networks H04L47/70)
- H04L 67/34: {involving the movement of software or configuration parameters (network booting or remote initial program loading [RIPL] G06F9/4416)}
- H04L 69/24: Negotiation of communication capabilities
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

This application discloses a method and an apparatus for multimedia communication. A session page is loaded by using a browser kernel integrated in a local client, and a script on the session page is executed by using the browser kernel, to perform the following operations: exchanging a control parameter with a peer client by using a signaling server; establishing a data channel between the local client and the peer client; collecting multimedia data and transmitting the multimedia data to the peer client through the data channel, so that the peer client plays the multimedia data by using a media stream parameter of the local client; and receiving, through the data channel, the multimedia data collected by the peer client, and playing the multimedia data on the session page according to a media stream parameter of the peer client. In this way, cross-client multimedia communication is implemented.

## Claims

### Claim 1

A method for multimedia communication performed at a terminal having one or more processors and memory storing a plurality of programs to be executed by the one or more processors, and the method comprising:
loading a session page by using a browser kernel integrated in a local client, and executing a script on the session page by using the browser kernel, to perform the following operations:
exchanging a control parameter with a peer client by using a signaling server, the control parameter comprising a network parameter and a media stream parameter;
establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter;
collecting multimedia data and transmitting the multimedia data to the peer client through the data channel, so that the peer client plays the multimedia data by using a media stream parameter of the local client; and
receiving, through the data channel, the multimedia data collected by the peer client, and playing the multimedia data on the session page according to a media stream parameter of the peer client.

### Claim 2 (depends on 1)

The method for multimedia communication according to claim 1, wherein the exchanging a control parameter with a peer client by using a signaling server comprises:
obtaining the control parameter configured by the local client;
adding the control parameter to a signaling message, and transmitting the signaling message to the peer client by using the signaling server; and
receiving the signaling message transmitted by the peer client by using the signaling server, and obtaining, through extraction from the received signaling message, the control parameter configured by the peer client.

### Claim 3 (depends on 1)

The method for multimedia communication according to claim 1, wherein the establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter comprises:
when both the local client and the peer client are on a wide area network, establishing a direct data channel between the local client and the peer client by using a wide area network address and port of the local client and a wide area network address and port of the peer client that are comprised in the network parameter, wherein a predetermined protocol is used for a link carrying the direct data channel.

### Claim 4 (depends on 1)

The method for multimedia communication according to claim 1, wherein the establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter comprises:
when at least one of the local client and the peer client is located in a firewall or is configured with a network address translator device, correspondingly performing a traversal operation on the firewall or the network address translator device by using a wide area network address or port of the firewall or the network address translator device, to form a direct data channel between the local client and the peer client, wherein
the direct data channel passes through the firewall or the network address translator device, and a predetermined protocol is used for a link carrying the direct data channel.

### Claim 5 (depends on 1)

The method for multimedia communication according to claim 1, wherein the method further comprises:
before collecting multimedia data and transmitting the multimedia data to the peer client through the data channel:
detecting the data channel; and
instructing, when the detection succeeds, a transit server to record a binding relationship between an identifier of the data channel and the address and the port of the local client and the address and the port of the peer client; and
the transmitting the multimedia data to the peer client through the data channel comprises:
transmitting, to the transit server through the data channel, the multimedia data collected by the local client, so that the transit server forwards, to the peer client according to the binding relationship, the multimedia data collected by the local client.

### Claim 6 (depends on 4)

The method for multimedia communication according to claim 4, wherein the method further comprises:
comprising, by the network parameter, a wide area network address and port of the local client;
before the network information is exchanged with the peer client by using the signaling server, requesting, from the signaling server, authentication information used for logging in to a transit server;
receiving the authentication information returned by the signaling server after the signaling server successfully authenticates the local client; and
logging in to the transit server by using the authentication information, and obtaining the wide area network address or port of the firewall or the network address translator device that are detected by the transit server.

### Claim 7 (depends on 4)

The method for multimedia communication according to claim 4, wherein the method further comprises:
when establishing the direct data channel by performing the traversal operation fails, establishing, by using the transit server, a transit data channel that is between the local client and the peer client and that is used for data transit, wherein
the transit data channel comprises: a link between the local client and the transit server, and a link between the transit server and the peer client.

### Claim 8 (depends on 1)

The method for multimedia communication according to claim 1, wherein the method further comprises:
before collecting multimedia data and transmitting the multimedia data to the peer client through the data channel:
performing an encrypted negotiation operation with the peer client through the data channel, to determine an encryption algorithm and a session key of the multimedia data transmitted on the data channel.

### Claim 9 (depends on 8)

The method for multimedia communication according to claim 8, wherein the performing an encrypted negotiation operation with the peer client through the data channel, to determine an encryption algorithm and a session key of the multimedia data transmitted on the data channel comprises:
receiving an encryption algorithm set and a peer client digital certificate that are selected by the peer client and that are sent by the peer client through the data channel, wherein the encryption algorithm set comprises at least a certificate authentication algorithm and an encryption algorithm;
verifying the peer client digital certificate according to the certificate authentication algorithm;
determining a key exchange algorithm by performing a negotiation operation with the peer client, and exchanging a key with the peer client; and
encrypting, according to the key exchange algorithm, the key generated by the local client and the peer client, to form the session key used for encrypting the multimedia data.

### Claim 10 (depends on 1)

The method for multimedia communication according to claim 1, wherein the method further comprises:
when a quantity of peer clients joining the session page exceeds a predetermined quantity, establishing a transit data channel to the peer client by using the transit server for data transit, and transmitting the multimedia data to the peer client through the transit data channel.

### Claim 11 (depends on 10)

The method for multimedia communication according to claim 10, wherein the method further comprises:
releasing the direct data channel between the local client and the peer client when the direct data channel is already established between the local client and the peer client; and
establishing the transit data channel to the peer servers by using the transit server for data transit.

### Claim 12 (depends on 1)

The method for multimedia communication according to claim 1, wherein the method further comprises:
when a quantity of peer clients joining the session page is less than a predetermined quantity, establishing a direct data channel to the peer client, and transmitting the multimedia data to the peer client through the direct data channel.

### Claim 13 (depends on 12)

The method for multimedia communication according to claim 12, wherein the method further comprises:
releasing a transit data channel between the local client and the peer client when the transit data channel used for data transit is already established between the local client and the peer client.

### Claim 14

An apparatus for multimedia communication, comprising: one or more processors, memory coupled to the one or more processors and a plurality of programs stored in the memory that, when executed by the one or more processors, cause the transcoding capability configuration device to perform operations including:
loading a session page by using a browser kernel integrated in a local client, and executing a script on the session page by using the browser kernel, to perform the following operations:
exchanging a control parameter with a peer client by using a signaling server, the control parameter comprising a network parameter and a media stream parameter;
establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter;
collecting multimedia data and transmitting the multimedia data to the peer client through the data channel, so that the peer client plays the multimedia data by using a media stream parameter of the local client; and
receiving, through the data channel, the multimedia data collected by the peer client, and playing the multimedia data on the session page according to a media stream parameter of the peer client.

### Claim 15 (depends on 14)

The apparatus for multimedia communication according to claim 14, wherein the exchanging a control parameter with a peer client by using a signaling server comprises:
obtaining the control parameter configured by the local client;
adding the control parameter to a signaling message, and transmitting the signaling message to the peer client by using the signaling server; and
receiving the signaling message transmitted by the peer client by using the signaling server, and obtaining, through extraction from the received signaling message, the control parameter configured by the peer client.

### Claim 16 (depends on 14)

The apparatus for multimedia communication according to claim 14, wherein the establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter comprises:
when both the local client and the peer client are on a wide area network, establishing a direct data channel between the local client and the peer client by using a wide area network address and port of the local client and a wide area network address and port of the peer client that are comprised in the network parameter, wherein a predetermined protocol is used for a link carrying the direct data channel.

### Claim 17 (depends on 14)

The apparatus for multimedia communication according to claim 14, wherein the establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter comprises:
when at least one of the local client and the peer client is located in a firewall or is configured with a network address translator device, correspondingly performing a traversal operation on the firewall or the network address translator device by using a wide area network address or port of the firewall or the network address translator device, to form a direct data channel between the local client and the peer client, wherein
the direct data channel passes through the firewall or the network address translator device, and a predetermined protocol is used for a link carrying the direct data channel.

### Claim 18 (depends on 14)

The apparatus for multimedia communication according to claim 14, wherein the operations further comprise:
when a quantity of peer clients joining the session page exceeds a predetermined quantity:
establishing a transit data channel to the peer client by using the transit server for data transit; and
transmitting the multimedia data to the peer client through the transit data channel.

### Claim 19 (depends on 14)

The apparatus for multimedia communication according to claim 14, wherein the operations further comprise:
releasing the direct data channel between the local client and the peer client when the direct data channel is already established between the local client and the peer client; and
establishing the transit data channel to the peer servers by using the transit server for data transit.

### Claim 20

A non-transitory computer readable storage medium storing a plurality of machine readable instructions in connection with an apparatus having one or more processors, wherein the plurality of machine readable instructions, when executed by the one or more processors, cause the apparatus to perform operations including:
loading a session page by using a browser kernel integrated in a local client, and executing a script on the session page by using the browser kernel, to perform the following operations:
exchanging a control parameter with a peer client by using a signaling server, the control parameter comprising a network parameter and a media stream parameter;
establishing a data channel between the local client and the peer client by using an address and a port of the local client and an address and a port of the peer client that are comprised in the network parameter;
collecting multimedia data and transmitting the multimedia data to the peer client through the data channel, so that the peer client plays the multimedia data by using a media stream parameter of the local client; and
receiving, through the data channel, the multimedia data collected by the peer client, and playing the multimedia data on the session page according to a media stream parameter of the peer client.
