Artificial intelligence and machine learning for transcription and translation for media editing

A computer program product comprising:

• a non-transitory computer-readable medium with computer-readable instructions encoded thereon, wherein the computer-readable instructions, when processed by a processing device instruct the processing device to perform a method of editing media, the method comprising:
• receiving a media composition that includes video and audio, wherein the audio includes dialog in a first language spoken by a first voice;
• converting the spoken dialog in the first language to first language text using automatic speech recognition;
• using machine-learning-based text-to-text translation to translate the first language text into second language text;
• using text-to-speech synthesis on the second language text to generate a spoken dialog in the second language;
• generating a language proxy by linking the spoken dialog in the second language to the media composition, such that the spoken dialog in the second language is in temporal synchrony with the video of the media composition;
• enabling an editor to edit the audio and video of the media composition by editing the language proxy;
• relinking a version of the media composition that has been edited using the language proxy to source media of the media composition, wherein the source media includes the spoken dialog in the first language;
• and
• generating an edited version of the media composition with dialog spoken in the first language.

A system comprising:

• a memory for storing computer-readable instructions;
• and
• a processor connected to the memory, wherein the processor, when executing the computer-readable instructions, causes the system to perform a method of editing media, the method comprising:
• receiving a media composition that includes video and audio, wherein the audio includes dialog in a first language spoken by a first voice;
• converting the spoken dialog in the first language to first language text using automatic speech recognition;
• using machine-learning-based text-to-text translation to translate the first language text into second language text;
• using text-to-speech synthesis on the second language text to generate a spoken dialog in the second language;
• generating a language proxy by linking the spoken dialog in the second language to the media composition, such that the spoken dialog in the second language is in temporal synchrony with the video of the media composition;
• enabling an editor to edit the audio and video of the media composition by editing the language proxy;
• relinking a version of the media composition that has been edited using the language proxy to source media of the media composition, wherein the source media includes the spoken dialog in the first language;
• and
• generating an edited version of the media composition with dialog spoken in the first language.