Automated digital document generation from digital videos
AssigneeAdobe Inc.
InventorsNiyati Himanshu Chhaya, Tripti Shukla, Jeevana Kruthi Karnuthala, Bhanu Prakash Reddy Guda, Ayudh Saxena, Abhinav Bohra, Abhilasha Sancheti, Aanisha Bhattacharyya
Life of this patent
Estimating the term…
Abstract
Techniques are described that support automated generation of a digital document from digital videos using machine learning. The digital document includes textual components that describe a sequence of entity and action descriptions from the digital video. These techniques are usable to generate a single digital document based on a plurality of digital videos as well as incorporate user-specified constraints in the generation of the digital document.
Claims20
A method implemented by a computing device, the method comprising:
- locating, by the computing device, action clips including frames that depict actions from a plurality of digital videos;
- generating, by the computing device, an action graph based on the action clips;
- selecting, by the computing device, a path based on the action graph;
- finding, by the computing device, frames from the plurality of digital videos by mapping nodes of the path to the action clips;
- forming, by the computing device, textual components that include a sequence of entity and respective action descriptions based on the frames using a model trained using machine-learning;
- generating, by the computing device, a digital document based on the textual components;
- and
- rendering the digital document by a display device.
A system comprising:
- a processing system;
- a non-transitory computer readable media communicatively coupled to the processing system;
- an action detection module implemented by the processing system to locate action clips including frames that depict actions from a digital video;
- an action graph generation module implemented by the processing system to generate an action graph based on the action clips;
- a path selection module implemented by the processing system to select a path based on the action graph;
- a frame location module implemented by the processing system to locate frames from the digital video based on a mapping nodes of the path to the action clips;
- and
- a decoding module implemented by the processing system to form textual components that include a sequence of entity and respective action descriptions based on the located frames using a model trained using machine-learning.
A system comprising:
- means for locating action clips from a plurality of digital videos;
- means for generating an action graph based on the action clips;
- means for selecting a path based on the action graph;
- means for locating frames from the plurality of digital videos based on the path;
- means for forming textual components that include a sequence of entity and respective action descriptions based on the located frames using a model trained using machine-learning;
- and
- means for generating a digital document based on the textual components.
Dependencies are inferred from the claim text.