# US 12,537,994 B2 · Selective content masking for collaborative computing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,537,994 B2 - Document ID: US12537994B2 - Status: Granted - Kind: B2 - Application number: 18/263,597 - Filed: 2022-06-02 - Published: 2024-03-14 - Granted: 2026-01-27 - Earliest claimed: 2022-06-02 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Dhandapani Shanmugam; Sreenivas Makam - Web page: https://patentgrep.com/patent/US12537994B2 ## CPC - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/45455: {applied to a region of the image} - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - H04N 7/15: Conference systems - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)} - H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/44029: {for generating different versions} - H04N 21/4542: {Blocking scenes or portions of the received content, e.g. censoring scenes} - H04N 21/4665: {involving classification methods, e.g. Decision trees} - H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} - G06N 3/084: Backpropagation, e.g. using gradient descent - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A machine-learned sharing system and methods are provided for sharing content with users while masking sensitive information. The system receives a content stream for display to one or more users, converts the content stream into image data representative of at least a portion of the content stream, inputs the image data into a machine-learned model configured for masking sensitive content within shared content, receives from the machine-learned model a first mask indicative of a region within the first content stream that contains sensitive content, and renders a display of the content stream that masks the sensitive content based at least in part on the first mask indicative of the region of the first content stream having the sensitive content. ## Claims ### Claim 1 A computer-implemented method for sharing content within a videoconferencing application, comprising: receiving, by a computing system comprising one or more computing devices, a request from a first participant in a video conference to share with one or more additional participants of the video conference a first content stream within the videoconferencing application; converting, by the computing system, at least a portion of the first content stream into image data representative of a display of the first content stream; inputting, by the computing system, the image data representative of the display of the first content stream into a machine-learned model configured for masking sensitive content within shared content; processing, by using the machine-learned model, the image data representative of the display of the first content stream to generate a first mask indicative of a region within the first content stream that contains sensitive content; and rendering, by the computing system, for the video conference a display of the first content stream that masks the sensitive content based at least in part on the first mask indicative of the region of the first content stream having the sensitive content. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein: said converting the first content stream into image data comprises converting the first content stream into a plurality of image frames, each image frame including image data representing at least a portion of the first content stream; said inputting the image data representative of the display of the first content stream into the machine-learned model comprises inputting the plurality of image frames into the machine-learned model; and the first mask is indicative of a first region within at least a first frame of the plurality of image frames that includes sensitive content. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, further comprising: partitioning, by the computing system using the machine-learned model, each frame into a set of logical partitions; wherein the first mask identifies at least one logical partition of the first frame of the plurality of image frames as containing sensitive content; wherein rendering for the video conference the display of the first content stream comprises masking the at least one logical partition of the first frame. ### Claim 4 (depends on 2) The computer-implemented method of claim 2, further comprising: determining, by the computing system with the machine-learned model, that a second frame of the plurality of image frames includes sensitive content; and receiving, by the computing system from the machine-learned model, a second mask indicative of a second region within the second frame that contains sensitive content; wherein the first region and the second region are at different locations within a display area of the first content stream. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: analyzing, by the computing system, the image data representative of the display of the first content stream to determine representative content of the first content stream; wherein rendering for the video conference the display of the first content stream that masks the sensitive content comprises rendering the representative content for the region within the first content stream that contains the sensitive content. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein: the computing system includes a host computing device, a first client computing device associated with the first participant and at least one additional client computing device associated with the one or more additional participants; and the machine-learned model is configured at the first client computing device. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein: the videoconferencing application is implemented at least partially by a web browser at the first client computing device. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein: the computing system includes a host computing device, a first client computing device associated with the first participant and at least one additional client computing device associated with the at least one additional participant; and the machine-learned model is configured at the host computing device. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein: the first content stream is generated by a first application of a client device associated with the first participant. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein: the first content stream is a multimedia content stream including a plurality of content types; said converting at least a portion of the first content stream into image data representative of the display of the first content stream comprises converting the multimedia content stream into image data representative of the plurality of content types. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, further comprising: obtaining, by the computing system, data descriptive of the machine-learned model; obtaining, by the computing system, one or more sets of training data comprising image data labeled to indicate sensitive content; and training, by the computing system, the machine-learned model based on the one or more sets of training data, wherein training the machine-learned model comprises: determining one or more parameters of a loss function based on the one or more sets of training data; and modifying at least a portion of the machine-learned model based at least in part on the one or more parameters of the loss function. ### Claim 12 A computing system, comprising: one or more processors; and one or more non-transitory, computer-readable media that store instructions that when executed by the one or more processors cause the computing system to perform operations, the operations comprising: receiving a request from a first participant in a video conference to share with one or more additional participants of the video conference a first content stream within a videoconferencing application; converting at least a portion of the first content stream into image data representative of a display of the first content stream; inputting the image data representative of the display of the first content stream into a machine-learned model configured for masking sensitive content within shared content; processing, by using the machine-learned model, the image data representative of the display of the first content stream to generate a first mask indicative of a region within the first content stream that contains sensitive content; and receiving from the machine-learned model a first mask indicative of a region within the first content stream that contains sensitive content; and rendering for the video conference a display of the first content stream that masks the sensitive content based at least in part on the first mask indicative of the region of the first content stream having the sensitive content. ### Claim 13 (depends on 12) The computing system of claim 12, wherein: said converting the first content stream into image data comprises converting the first content stream into a plurality of image frames, each image frame including image data representing at least a portion of the first content stream; said inputting the image data representative of the display of the first content stream into the machine-learned mode comprises inputting the plurality of image frames into the machine-learned model; and the first mask is indicative of a first region within at least a first frame of the plurality of image frames that includes sensitive content. ### Claim 14 (depends on 13) The computing system of claim 13, wherein the operations further comprise: partitioning, by the machine-learned model, each frame into a set of logical partitions; wherein the first mask identifies at least one logical partition of the first frame of the plurality of image frames as containing sensitive content; and wherein rendering for the video conference the display of the first content stream comprises masking the at least one logical partition of the first frame. ### Claim 15 (depends on 13) The computing system of claim 13, wherein the operations further comprise: determining with the machine-learned model that a second frame of the plurality of image frames includes sensitive content; and receiving from the machine-learned model a second mask indicative of a second region within the second frame that contains sensitive content; wherein the first region and the second region are at different locations within a display area of the first content stream. ### Claim 16 (depends on 15) The computing system of claim 15, wherein: the videoconferencing application is implemented at least partially by a web browser at a client computing device. ### Claim 17 One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations comprising: receiving a first content stream for display to one or more users; converting the first content stream into a plurality of image frames including image data representative of at least a portion of the first content stream; inputting the plurality of image frames into a machine-learned model configured for masking sensitive content within shared content; processing, by using the machine-learned model, the plurality of image frames to generate a first mask indicative of a region within the first content stream that contains sensitive content; and receiving from the machine-learned model a first mask indicative of a region within the first content stream that contains sensitive content; and rendering a display of the first content stream that masks the sensitive content based at least in part on the first mask indicative of the region of the first content stream having the sensitive content. ### Claim 18 (depends on 17) The one or more non-transitory computer-readable media of claim 17, further comprising: partitioning, by the machine-learned model, each frame into a set of logical partitions; wherein the first mask identifies at least one logical partition of a first frame of the plurality of image frames as containing sensitive content; and wherein rendering for the display of the first content stream comprises masking the at least one logical partition of the first frame. ### Claim 19 (depends on 17) The one or more non-transitory computer-readable media of claim 17, wherein the first mask is indicative of a first region within at least a first frame of the plurality of image frames that includes sensitive content, the operations further comprising: determining with the machine-learned model that a second frame of the plurality of image frames includes sensitive content; and receiving from the machine-learned model a second mask indicative of a second region within the second frame that contains sensitive content; wherein the first region and the second region are at different locations within a display area of the first content stream. ### Claim 20 (depends on 17) The one or more non-transitory computer-readable media of claim 17, wherein: the first content stream is a multimedia content stream including a plurality of content types; and said converting the first content stream into a plurality of image frames representative comprises converting the multimedia content stream into image data representative of the plurality of content types.