Automated multimodal adaptation of multimedia content
AssigneeInternational Business Machines Corporation
InventorsSai Krishna Reddy Gudimetla, Aaron K. Baughman, Micah Forster, Craig M. Trim
Life of this patent
Estimating the term…
Abstract
Using a first trained generative adversarial network, a first multimedia content is transformed into a text description of the first multimedia content. The text description is adjusted according to a constraint using a trained attention layer, the adjusting creating an adjusted text description. Using a trained model, the adjusted text description is transformed into a second multimedia content, the second multimedia content comprising an adjustment of the first multimedia content according to the constraint.
Claims20
A computer-implemented method comprising:
- transforming, using a first trained generative adversarial network, a first multimedia content into a text description of the first multimedia content;
- adjusting, according to a constraint using a trained attention layer, the text description, the adjusting creating an adjusted text description;
- and
- transforming, using a trained model, the adjusted text description into a second multimedia content, the second multimedia content comprising an adjustment of the first multimedia content according to the constraint.
A computer program product for automated multimodal adaptation of multimedia content, the computer program product comprising:
- one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising:
- program instructions to transform, using a first trained generative adversarial network, a first multimedia content into a text description of the first multimedia content;
- program instructions to adjust, according to a constraint using a trained attention layer, the text description, the adjusting creating an adjusted text description;
- and
- program instructions to transform, using a trained model, the adjusted text description into a second multimedia content, the second multimedia content comprising an adjustment of the first multimedia content according to the constraint.
A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:
- program instructions to transform, using a first trained generative adversarial network, a first multimedia content into a text description of the first multimedia content;
- program instructions to adjust, according to a constraint using a trained attention layer, the text description, the adjusting creating an adjusted text description;
- and
- program instructions to transform, using a trained model, the adjusted text description into a second multimedia content, the second multimedia content comprising an adjustment of the first multimedia content according to the constraint.
Dependencies are inferred from the claim text.