Back to results
US 11,706,176 B2Granted
Detecting messages with offensive content
OwnerTrisha N. Prabhuinventor
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage medium, to facilitate interception of messages that include offensive content. In one aspect, a method includes actions of receiving input on a user device that includes message content, determining, on the user device, whether the message content includes offensive content, and in response to determining, on the user device, that the message content includes offensive content, generating an alert message for display on the user device that provides an indication that the message includes offensive content.
Claims21
A method comprising:
- receiving, by one or more computers, first data that represents message content obtained from a first user device;
- determining, by one or more computers and prior to communication of second data to a second user device that, when processed by the second user device, causes the second user device to display the message content on a display of the second user device, whether the first data represents message content that is likely offensive content, wherein determining whether the received first data represents message content that is likely offensive content comprises:
- providing, by one or more computers, at least a portion of the received first data as an input to a machine learning model that has been trained to detect a likelihood that a message includes offensive content;
- and
- obtaining, by one or more computers, output data that was generated by the machine learning model based on the machine learning model processing the provided first data;
- in response to determining, by one or more computers and based on the obtained output data, that the first data represents message content that is likely offensive content, generating, by one or more computers, alert data that, when processed by the first user device, causes the first user device to display a visualization that provides a user of the first user device with an opportunity to reconsider whether the message content is to be communicated to the second user device;
- and
- providing, by one or more computers, the generated alert data to the first user device.
5 more elements
A system comprising:
- one or more computers;
- and
- one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations, the operations comprising:
- receiving, by one or more computers, first data that represents message content obtained from a first user device;
- determining, by one or more computers and prior to communication of second data to a second user device that, when processed by the second user device, causes the second user device to display the message content on a display of the second user device, whether the first data represents message content that is likely offensive content, wherein determining whether the received first data represents message content that is likely offensive content comprises:
- providing, by one or more computers, at least a portion of the received first data as an input to a machine learning model that has been trained to detect a likelihood that a message includes offensive content;
- and
- obtaining, by one or more computers, output data that was generated by the machine learning model based on the machine learning model processing the provided first data;
- in response to determining, by one or more computers and based on the obtained output data, that the first data represents message content that is likely offensive content, generating, by one or more computers, alert data that, when processed by the first user device, causes the first user device to display a visualization that provides a user of the first user device with an opportunity to reconsider whether the message content is to be communicated to the second user device;
- and
- providing, by one or more computers, the generated alert data to the first user device.
8 more elements
One or more computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations, the operations comprising:
- receiving first data that represents message content obtained from a first user device;
- determining, prior to communication of second data to a second user device that, when processed by the second user device, causes the second user device to display the message content on a display of the second user device, whether the first data represents message content that is likely offensive content, wherein determining whether the received first data represents message content that is likely offensive content comprises:
- providing at least a portion of the received first data as an input to a machine learning model that has been trained to detect a likelihood that a message includes offensive content;
- and
- obtaining, output data that was generated by the machine learning model based on the machine learning model processing the provided first data;
- in response to determining, based on the obtained output data, that the first data represents message content that is likely offensive content, generating alert data that, when processed by the first user device, causes the first user device to display a visualization that provides a user of the first user device with an opportunity to reconsider whether the message content is to be communicated to the second user device;
- and
- providing the generated alert data to the first user device.
5 more elements
Dependencies are inferred from the claim text.