# US 12,010,399 B2 · Generating revoiced media streams in a virtual reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,399 B2 - Document ID: US12010399B2 - Status: Granted - Kind: B2 - Application number: 18/097,900 - Filed: 2023-01-17 - Published: 2023-05-18 - Granted: 2024-06-11 - Earliest claimed: 2019-03-10 - Applicants: Ben Avi Ingel; Ron Zass - Inventors: Ben Avi Ingel; Ron Zass - Web page: https://patentgrep.com/patent/US12010399B2 ## CPC - H04N 21/8106: {involving special audio data, e.g. different tracks for different languages} - H04N 21/8126: {involving additional data, e.g. news, sports, stocks, weather forecasts} - G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation - G10L 13/00: Speech synthesis; Text to speech systems - G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser - G10L 13/086: {Detection of language} - G10L 13/10: Prosody rules derived from text; Stress or intonation - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/26603: {for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques} - H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/458: Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules {; time-related management operations (arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10)} - H04N 21/4755: {for defining user preferences, e.g. favourite actors or genre (retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Methods, systems, and computer-readable media for generating videos with characters indicating regions of images are provided. For example, an image containing a first region may be received. At least one characteristic of a character may be obtained. A script containing a first segment of the script may be received. The first segment of the script may be related to the first region of the image. The at least one characteristic of a character and the script may be used to generate a video of the character presenting the script and at least part of the image, where the character visually indicates the first region of the image while presenting the first segment of the script. ## Claims ### Claim 1 A computer program product for generating a revoiced media stream in a virtual reality system, the computer program product embodied in a non-transitory computer-readable medium and including instructions for causing at least one processor to execute a method comprising: receiving a media stream from an individual speaking in an origin language, wherein the individual is associated with a particular voice; obtaining a transcript of the media stream in the origin language; translating the transcript of the media stream to a target language, wherein the translated transcript includes at least one word in the target language for each word spoken in the origin language; analyzing the media stream to determine a voice profile for the individual, wherein the voice profile corresponds with the particular voice of the individual; determining at least one characteristic of a personalized avatar that represents the individual; determining a synthesized voice for the personalized avatar based on the voice profile, wherein the synthesized voice sounds substantially identical to the particular voice; and enabling the virtual reality system to generate a revoiced media stream that includes a visualization of the personalized avatar speaking the translated transcript in the target language using the synthesized voice. ### Claim 2 (depends on 1) The computer program product of claim 1, wherein the method further includes: determining a desired level of origin language accent to introduce in the synthesized voice of the personalized avatar; and enabling the virtual reality system to generate a revoiced media stream that includes a visualization of the personalized avatar that speaks in the target language with the desired level of origin language accent. ### Claim 3 (depends on 1) The computer program product of claim 1, wherein the method further includes: based on at least one rule for revising transcripts of media streams, automatically revising a first part of the transcript and avoid from revising a second part of the transcript; and enabling the virtual reality system to generate a revoiced media stream that includes a visualization of the personalized avatar that speaks the first revised part the translated transcript in the target language and the second unrevised part the translated transcript in the target language. ### Claim 4 (depends on 1) The computer program product of claim 1, wherein the method further includes: based on a determined user category indicative of a vocabulary level of associated with the individual, revising the transcript of the media stream; and enabling the virtual reality system to generate the revoiced media stream that includes the visualization of the personalized avatar that speaks the revised transcript in the target language. ### Claim 5 (depends on 1) The computer program product of claim 1, wherein the method further includes: translating the transcript of the media stream to the target language based on preferred language characteristics of the individual; and enabling the virtual reality system to generate the revoiced media stream that includes the visualization of the personalized avatar that speaks the translated the transcript in the target language. ### Claim 6 (depends on 1) The computer program product of claim 1, wherein the method further includes: analyzing the transcript to determine a set of language characteristics associated with the individual; translating the transcript to the target language based on the determined set of language characteristics; and enabling the virtual reality system to generate the revoiced media stream that includes the visualization of the personalized avatar that speaks the translated the transcript in the target language. ### Claim 7 (depends on 1) The computer program product of claim 1, wherein the method further includes: based on at least one rule for translating transcripts of media streams, automatically translating a first part of the transcript to the target language and avoid from translating a second part of the transcript to the target language; and enabling the virtual reality system to generate the revoiced media stream that includes the visualization of the personalized avatar that speaks the first part of the transcript in the target language and the second part of the transcript in the origin language. ### Claim 8 (depends on 1) The computer program product of claim 1, wherein the method further includes: analyzing the transcript to determine that the individual discusses a subject likely to be unfamiliar with at least one individual that would listen to the personalized avatar; and enabling the virtual reality system to provide an explanation in the target language to the subject discussed by the individual in the origin language. ### Claim 9 (depends on 1) The computer program product of claim 1, wherein the method further includes: determining metadata information for the translated transcript, wherein the metadata information includes desired volume levels for different words; and enabling the virtual reality system to generate the revoiced media stream in which a ratio of the volume levels between words spoken by the personalized avatar in the target language is substantially identical to the ratio of volume levels between different words spoken by the individual in the origin language. ### Claim 10 (depends on 1) The computer program product of claim 1, wherein the received media stream is associated with a real-time conversation between the individual and at least one other individual, and the method further includes processing visual data to identify text written in the origin language; determining relevancy of the identified text to the particular user; and providing a translation in the target language for the identified text, when the content of the identified text is determined to be relevant. ### Claim 11 (depends on 1) The computer program product of claim 1, wherein the personalized avatar that speaks in the target language is a realistic avatar or a semi-realistic avatar associated with a depiction of the individual that speaks in the origin language. ### Claim 12 (depends on 1) The computer program product of claim 1, wherein the method further includes causing the personalized avatar to visually point to an object while presenting in the target language a segment of the translated transcript that relates to the object. ### Claim 13 (depends on 12) The computer program product of claim 12, wherein the object is a graphic presentation of a weather forecast, a graphic presentation of a calendar event, or a graphic presentation of a past event associated with the individual. ### Claim 14 (depends on 1) The computer program product of claim 1, wherein the method further includes: receiving user selection from the individual to determine the target language for the personalized avatar; and enabling the virtual reality system to generate a revoiced media stream that includes a visualization of the personalized avatar that speaks the selected target language. ### Claim 15 (depends on 1) The computer program product of claim 1, wherein the received media stream is associated with a real-time conversation between the individual and at least one other individual, and the method further includes determining a preferred target language for the personalized avatar based on an identity at least one other individual; and enabling the virtual reality system to generate a revoiced media stream that includes a visualization of the personalized avatar that speaks in the preferred target language. ### Claim 16 (depends on 1) The computer program product of claim 1, wherein the method further includes selecting the at least one characteristic of the personalized avatar based on a profile of the individual associated, at least in part, on a geographical location associated with the individual. ### Claim 17 (depends on 1) The computer program product of claim 1, wherein the method further includes receiving user selection from the individual to enable selective manipulation of the at least one visual characteristic of the personalized avatar. ### Claim 18 (depends on 1) The computer program product of claim 1, wherein the method further includes receiving user selection from the individual to enable selective manipulation of the at least one voice characteristic of the personalized avatar. ### Claim 19 A method for artificially generating a revoiced media stream in a virtual reality system, the method comprising: receiving a media stream from an individual speaking in an origin language, wherein the individual is associated with a particular voice; obtaining a transcript of the media stream in the origin language; translating the transcript of the media stream to a target language, wherein the translated transcript includes at least one word in the target language for each word spoken in the origin language; analyzing the media stream to determine a voice profile for the individual, wherein the voice profile corresponds with the particular voice of the individual; determining at least one characteristic of a personalized avatar that represents the individual; determining a synthesized voice for the personalized avatar based on the voice profile, wherein the synthesized voice sounds substantially identical to the particular voice; and enabling the virtual reality system to generate a revoiced media stream that includes a visualization of the personalized avatar speaking the translated transcript in the target language using the synthesized voice. ### Claim 20 A virtual reality system for artificially generating a revoiced media stream, the system comprising at least one processing device configured to: receive a media stream from an individual speaking in an origin language, wherein the individual is associated with a particular voice; obtain a transcript of the media stream in the origin language; translate the transcript of the media stream to a target language, wherein the translated transcript includes at least one word in the target language for each word spoken in the origin language; analyze the media stream to determine a voice profile for the individual, wherein the voice profile corresponds with the particular voice of the individual; determine at least one characteristic of a personalized avatar that represents the individual; determine a synthesized voice for the personalized avatar based on the voice profile, wherein the synthesized voice sounds substantially identical to the particular voice; and generate a revoiced media stream that includes a visualization of the personalized avatar speaking the translated transcript in the target language using the synthesized voice.