On demand interactive content generation in audiobooks through a natural language interface
AssigneeApple Inc.
InventorsKin Wai Tsui, Yuchia Chang, Junhan Chen
Life of this patent
Estimating the term…
Abstract
Disclosed are systems, methods, and computer-readable media for generating interactive content of audiobooks through a natural language interface. The disclosed technology generates a system prompt based on a combination of a user prompt and personal information of the user or of the user's environment. This combination can then be input into a multimodal machine learning model or multiple unimodal machine learning models to create both text and image outputs corresponding to the requested story line. The story can then be presented to the user and edited as needed, in some embodiments using the same content generation service that produced the system prompt to begin with.
Claims21
A method for generating a narrative with corresponding visuals from a user prompt, the method comprising:
- receiving, by a virtual assistant bot, a request to create a story, wherein the request to create the story includes a user-provided prompt that includes a narrative seed from which to create the story;
- generating, by the virtual assistant bot, a narrative generation prompt from a combination of a system prompt and the user-provided prompt;
- sending, by the virtual assistant bot, the narrative generation prompt to a generative content generation service, wherein the generative content generation service generates narrative text corresponding to the story based on the narrative generation prompt, wherein the generative content generation service further generates visual media prompts corresponding to the narrative text, and wherein the visual media prompts include text that represents instructions to generate visuals;
- generating, by the generative content generation service, the visuals in response to the visual media prompts;
- receiving, by a story application and from the generative content generation service, the narrative text and the visuals in a style defined by the narrative generation prompt;
- synchronizing, by the story application, the visuals with the narrative text to create the story;
- and
- presenting, by the story application, the story to a user, wherein the story is presented as a series of segments.
A system for generating a narrative with corresponding visuals from a user prompt, the system comprising:
- one or more processors;
- and
- at least one computer-readable storage medium having stored therein instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
- receiving, by a virtual assistant bot, a request to create a story, wherein the request to create the story includes a user-provided prompt that includes a narrative seed from which to create the story;
- generating, by the virtual assistant bot, a narrative generation prompt from a combination of a system prompt and the user-provided prompt;
- sending, by the virtual assistant bot, the narrative generation prompt to a generative content generation service, wherein the generative content generation service generates narrative text corresponding to the story based on the narrative generation prompt, wherein the generative content generation service further generates visual media prompts corresponding to the narrative text, and wherein the visual media prompts include text that represents instructions to generate visuals;
- generating, by the generative content generation service, the visuals in response to the visual media prompts;
- receiving, by a story application and from the generative content generation service, the narrative text and the visuals in a style defined by the narrative generation prompt;
- synchronizing, by the story application, the visuals with the narrative text to create the story;
- and
- presenting, by the story application, the story to a user, wherein the story is presented as a series of segments.
A non-transitory computer-readable storage medium having stored therein instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- receiving, by a virtual assistant bot, a request to create a story from a user, wherein the request to create the story includes a user-provided prompt that includes a narrative seed from which to create the story;
- generating, by the virtual assistant bot, a narrative generation prompt from a combination of a system prompt and the user-provided prompt;
- sending, by the virtual assistant bot, the narrative generation prompt to a generative content generation service, wherein the generative content generation service generates narrative text corresponding to the story based on the narrative generation prompt, wherein the generative content generation service further generates visual media prompts corresponding to the narrative text, and wherein the visual media prompts include text that represents instructions to generate visuals;
- generating, by the generative content generation service, the visuals in response to the visual media prompts;
- receiving, by a story application and from the generative content generation service, the narrative text and the visuals in a style defined by the narrative generation prompt;
- synchronizing, by the story application, the visuals with the narrative text to create the story;
- and
- presenting, by the story application, the story to a user, wherein the story is presented as a series of segments.
Dependencies are inferred from the claim text.