Automatic generation of training data for scientific paper summarization using videos

A system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

• collecting a plurality of video and audio recordings of presentations of documents;
• collecting a plurality of documents corresponding to the video and audio recordings;
• converting the plurality of video and audio recordings of presentations of documents into transcripts of the plurality of presentations;
• generating a summary of each document by selecting a plurality of sentences from each document using the transcript of the that document;
• generating a dataset comprising a plurality of the generated summaries;
• and
• training a machine learning model using the generated dataset.

A computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

• collecting, at the computer system, a plurality of video and audio recordings of presentations of documents;
• collecting, at the computer system, a plurality of documents corresponding to the video and audio recordings;
• converting, at the computer system, the plurality of video and audio recordings of presentations of documents into transcripts of the plurality of presentations;
• generating, at the computer system, a summary of each document by selecting a plurality of sentences from each document using the transcript of the that document;
• generating, at the computer system, a dataset comprising a plurality of the generated summaries;
• and
• training, at the computer system, a machine learning model using the generated dataset.