Real-time replacement of policy-violating content within voice chat communication

A system comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
• obtaining a real-time audio stream of voice chat communication;
• dividing the real-time audio stream into a plurality of chunks, wherein each chunk comprises audio from the real-time audio stream from a respective time window;
• for each chunk,
• providing the audio of the chunk to a machine-learning based detection model;
• obtaining, as output of the detection model, an indication whether policy-violating content is present in the audio of the chunk, wherein the detection model comprises an encoder that encodes the audio of the chunk into a feature vector and a classifier that generates the indication of whether policy-violating content is present in the audio of the chunk based on the feature vector;
• and
• in response to the indication that policy-violating content is present in the audio of the chunk, modifying the audio of the chunk to at least one of:
• mute at least a portion of the audio of the chunk or replace the at least a portion of the audio of the chunk with a different audio;
• and
• providing the audio stream to a client device for playback at the client device, wherein the audio stream is provided with a delay.

A non-transitory computer-readable medium containing instructions comprising:

• obtaining a real-time audio stream of voice chat communication;
• dividing the real-time audio stream into a plurality of chunks, wherein each chunk comprises audio from the real-time audio stream from a respective time window;
• for each chunk,
• providing the audio of the chunk to a machine-learning based detection model;
• obtaining, as output of the detection model, an indication whether policy-violating content is present in the audio of the chunk, wherein the detection model comprises an encoder that encodes the audio of the chunk into a feature vector and a classifier that generates the indication of whether policy-violating content is present in the audio of the chunk based on the feature vector;
• and
• in response to the indication that policy-violating content is present in the audio of the chunk, modifying the audio of the chunk to at least one of:
• mute at least a portion of the audio of the chunk or replace the at least a portion of the audio of the chunk with different audio;
• and
• providing the audio stream to a client device for playback at the client device, wherein the audio stream is provided with a delay.