Generating encoded text based on spoken utterances using machine learning systems and methods

A mobile device for generating encoded text to convey nonverbal meaning based on audio inputs, the mobile device comprising:

• at least one hardware processor;
• at least one wireless transceiver;
• at least one hardware display screen;
• and
• at least one non-transitory memory carrying instructions that, when executed by the at least one hardware processor, cause the mobile device to:
• receive audio data for a spoken utterance;
• analyze the audio data using a text encoding model to identify a nonverbal characteristic of the spoken utterance,
• wherein the nonverbal characteristic includes a sentiment and a speaking rate or a volume;
• generate, by the text encoding model, an encoded text representation of the spoken utterance,
• wherein the encoded text representation comprises a text transcription of the spoken utterance and a visual representation of the nonverbal characteristic of the spoken utterance, and
• wherein the visual representation comprises a geometric element or a variation in a text attribute;
• generate, based on the nonverbal characteristic of the spoken utterance, a prompt to input a second spoken utterance comprising at least one suggestion for changes to one or more different nonverbal characteristics indicative of a different sentiment;
• and
• cause display, on the at least one hardware display screen, of the generated encoded text representation and the prompt.

A method for generating encoded text representations to convey nonverbal information based on audio inputs, the method comprising:

• receiving audio data for a spoken utterance;
• analyzing the audio data using a text encoding model to identify a nonverbal characteristic of the spoken utterance,
• wherein the nonverbal characteristic includes a sentiment and a speaking rate or a volume;
• generating, by the text encoding model, an encoded text representation of the spoken utterance,
• wherein the encoded text representation comprises a text transcription of the spoken utterance and a visual representation of the nonverbal characteristic of the spoken utterance, and
• wherein the visual representation comprises a geometric element or a variation in a text attribute;
• generating, based on the identified nonverbal characteristic of the spoken utterance, a prompt to input a second spoken utterance comprising at least one suggestion for changes to one or more different nonverbal characteristics indicative of a different sentiment;
• and
• causing display, via a user interface, of the generated encoded text representation and the prompt.

At least one computer-readable medium, excluding transitory signals, carrying instructions that, when executed by a computing system, cause the computing system to perform operations to generate encoded text to convey nonverbal information based on audio inputs, the operations comprising:

• receiving audio data for a spoken utterance;
• analyzing the audio data using a text encoding model to identify a nonverbal characteristic of the spoken utterance,
• wherein the nonverbal characteristic includes a sentiment and a speaking rate or a volume;
• generating, by the text encoding model, an encoded text representation of the spoken utterance,
• wherein the encoded text representation comprises a text transcription of the spoken utterance and a visual representation of the nonverbal characteristic of the spoken utterance, and
• wherein the visual representation comprises a geometric element or a variation in a text attribute;
• generate, based on the identified nonverbal characteristic of the spoken utterance, a prompt to input a second spoken utterance comprising at least one suggestion for changes to one or more different nonverbal characteristics indicative of a different sentiment;
• and
• cause display of the generated encoded text representation and the prompt.