# US 10,375,354 B2 · Video communication using subtractive filtering > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,354 B2 - Document ID: US10375354B2 - Status: Granted - Kind: B2 - Application number: 15/863,713 - Filed: 2018-01-05 - Published: 2019-07-11 - Granted: 2019-08-06 - Earliest claimed: 2018-01-05 - Assignees: Facebook, Inc. - Applicants: Facebook, Inc. - Inventors: Sai Sri Sathya; Pritesh Sankhe; Myshkin Ingawale - Web page: https://patentgrep.com/patent/US10375354B2 ## CPC - H04L 65/1069: Session establishment or de-establishment - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/73: using feature-based methods - G06V 10/87: {using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system} - G06V 40/165: {using facial parts and geometric relationships} - G06V 40/176: {Dynamic expression} - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - H04L 65/80: Responding to QoS - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04N 7/15: Conference systems - G06T 2207/10016: Video; Image sequence - G06T 2207/20072: Graph-based image processing - G06T 2207/20081: Training; Learning - G06T 2207/20132: Image cropping - G06T 2207/20224: Image subtraction - G06T 2207/30201: Face - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In one embodiment, a method includes receiving a request to establish an audiovisual streaming session between a first device and a second device; determining a level of network connectivity for a network associated with the first device or the second device; receiving data from the first device. The data may the data represent facial expressions of a user in a video captured by the first device. The data may be generated by the first device using the video based on: detections of a face and facial features of the user in the video and application of one or more subtractive filters using the detected face and facial features. The subtractive filters may be selected based on the level of network connectivity. The method may also include sending the data to the second device with instructions to process the data and display a representation of the user's facial expressions. ## Claims ### Claim 1 A method comprising, by one or more computing devices: receiving a request to establish a live audiovisual streaming session between a first device and a second device; determining a level of network connectivity for a network associated with the first device or the second device; receiving data generated by the first device, wherein the data represents facial expressions of a user in a video captured by the first device, wherein the data is generated by the first device using the video based on: a detection of a face or a facial feature of the user in the video; and an application of one or more subtractive filters on the video using the detected face or facial feature, wherein the one or more subtractive filters are selected based on the level of network connectivity, wherein the data, relative to the video, is reduced in size based on the application of the one or more subtractive filters; and sending the data to the second device with instructions to process the data and display a representation of the user's facial expressions on a display screen associated with the second device. ### Claim 2 (depends on 1) The method of claim 1, wherein one of the subtractive filters is a facial-landmark filter, wherein the facial-landmark filter removes substantially all data from the video except for a set of facial landmarks for the face, and wherein the facial-landmark filter is applied based on a determination that the level of network connectivity is below a threshold level. ### Claim 3 (depends on 1) The method of claim 1, wherein the representation of the user's facial expressions comprises one or more animated features that have been added to the data. ### Claim 4 (depends on 1) The method of claim 1, wherein one of the subtractive filters is a dominant-feature filter, wherein the dominant-feature filter removes substantially all video data from the original data stream except for one or more dominant features of each of the isolated faces, and wherein the dominant-feature filter is applied based on a determination that the level of network connectivity is moderately poor. ### Claim 5 (depends on 4) The method of claim 4, wherein the dominant features comprise eyes and a mouth of the first user. ### Claim 6 (depends on 1) The method of claim 1, wherein the representation of the user's facial expressions comprises one or more animated features that have been added to the data, and wherein the animated features are generated at least in part based on a machine-learning model that takes as input the data received from the first device and audio data and outputs the representation of the user's facial expressions. ### Claim 7 (depends on 6) The method of claim 6, wherein the machine-learning model is trained with an audiovisual recording of the user. ### Claim 8 (depends on 6) The method of claim 6, wherein the machine-learning model is trained with one or more images or audiovisual recordings stored in association with a social-networking system. ### Claim 9 (depends on 1) The method of claim 1, wherein the detection of the face or the facial feature is accomplished using Haar Cascading. ### Claim 10 One or more computer-readable non-transitory storage media embodying software that is operable when executed to: receive a request to establish a live audiovisual streaming session between a first device and a second device; determine a level of network connectivity for a network associated with the first device or the second device; receive data generated by the first device, wherein the data represents facial expressions of a user in a video captured by the first device, wherein the data is generated by the first device using the video based on: a detection of a face or a facial feature of the user in the video; and an application of one or more subtractive filters on the video using the detected face or facial feature, wherein the one or more subtractive filters are selected based on the level of network connectivity, wherein the data, relative to the video, is reduced in size based on the application of the one or more subtractive filters; and send the data to the second device with instructions to process the data and display a representation of the user's facial expressions on a display screen associated with the second device. ### Claim 11 (depends on 10) The media of claim 10, wherein one of the subtractive filters is a facial-landmark filter, wherein the facial-landmark filter removes substantially all data from the video except for a set of facial landmarks for the face, and wherein the facial-landmark filter is applied based on a determination that the level of network connectivity is below a threshold level. ### Claim 12 (depends on 10) The media of claim 10, wherein the representation of the user's facial expressions comprises one or more animated features that have been added to the data. ### Claim 13 (depends on 10) The media of claim 10, wherein one of the subtractive filters is a dominant-feature filter, wherein the dominant-feature filter removes substantially all video data from the original data stream except for one or more dominant features of each of the isolated faces, and wherein the dominant-feature filter is applied based on a determination that the level of network connectivity is moderately poor. ### Claim 14 (depends on 13) The media of claim 13, wherein the dominant features comprise eyes and a mouth of the first user. ### Claim 15 (depends on 10) The media of claim 10, wherein the representation of the user's facial expressions comprises one or more animated features that have been added to the data, and wherein the animated features are generated at least in part based on a machine-learning model that takes as input the data received from the first device and audio data and outputs the representation of the user's facial expressions. ### Claim 16 A system comprising: one or more processors; and one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to: receive a request to establish a live audiovisual streaming session between a first device and a second device; determine a level of network connectivity for a network associated with the first device or the second device; receive data generated by the first device, wherein the data represents facial expressions of a user in a video captured by the first device, wherein the data is generated by the first device using the video based on: a detection of a face or a facial feature of the user in the video; and an application of one or more subtractive filters on the video using the detected face or facial feature, wherein the one or more subtractive filters are selected based on the level of network connectivity, wherein the data, relative to the video, is reduced in size based on the application of the one or more subtractive filters; and send the data to the second device with instructions to process the data and display a representation of the user's facial expressions on a display screen associated with the second device. ### Claim 17 (depends on 16) The system of claim 16, wherein one of the subtractive filters is a facial-landmark filter, wherein the facial-landmark filter removes substantially all data from the video except for a set of facial landmarks for the face, and wherein the facial-landmark filter is applied based on a determination that the level of network connectivity is below a threshold level. ### Claim 18 (depends on 16) The system of claim 16, wherein the representation of the user's facial expressions comprises one or more animated features that have been added to the data. ### Claim 19 (depends on 16) The system of claim 16, wherein one of the subtractive filters is a dominant-feature filter, wherein the dominant-feature filter removes substantially all video data from the original data stream except for one or more dominant features of each of the isolated faces, and wherein the dominant-feature filter is applied based on a determination that the level of network connectivity is moderately poor. ### Claim 20 (depends on 19) The system of claim 19, wherein the dominant features comprise eyes and a mouth of the first user.