Back to results
US 10,832,734 B2Granted
Dynamic audiovisual segment padding for machine learning
AssigneeInternational Business Machines Corporation
InventorsAaron K. Baughman, Stephen C. Hammer, Gray Cannon
Life of this patent
Estimating the term…
Abstract
Techniques for padding audiovisual clips (for example, audiovisual clips of sporting events) for the purpose of causing the clip to have a predetermined duration so that the padded clip can be evaluated for viewer interest by a machine learning (ML) algorithm. The unpadded clip is padded with audiovisual segment(s) that will cause the padded clip to have a level of viewer interest that it would have if the unpadded clip had been longer. In some embodiments the padded segments are synthetic images generated by a generative adversarial network such that the synthetic images would have the same level of viewer interest (as adjudged by an ML algorithm) as if the unpadded clip had been shot to be longer.
Claims15
A method comprising:
- receiving a first unpadded audiovisual segment data set including information indicative of a first unpadded audiovisual segment;
- determining a set of padding time interval(s) occurring before and/or after the first unpadded segment;
- for each given padding time interval of the set of padding time interval(s):
- determining a respectively corresponding viewer interest value that would characterize the given padding time interval if the first unpadded audiovisual segment continued through the given padding time interval and had its viewer interest value determined by a machine learning (ML) algorithm, and
- generating a synthetic audiovisual segment for the given padding time interval so that the synthetic audiovisual segment for the given padding time interval is characterized by the viewer interest value determined for the given padding time interval;
- assembling the first unpadded audiovisual segment with the synthetic audiovisual segment(s) corresponding to each padding time interval of the set of padding time interval(s) to obtain a first padded audiovisual segment data set including information indicative of a first padded audiovisual segment;
- and
- determining, by the ML algorithm, a viewer interest value for the first padded audiovisual segment considered as a whole;
- wherein the generation of the synthetic audiovisual segment for each given padding time interval is performed by a generative adversarial network (GAN).
6 more elements
A computer program product (CPP) comprising:
- a computer readable storage medium;
- and computer code stored on the machine readable storage device, with the computer code including instructions for causing a processor(s) set to perform operations including the following:
- receiving a first unpadded audiovisual segment data set including information indicative of a first unpadded audiovisual segment, determining a set of padding time interval occurring before and/or after the first unpadded segment, for each given padding time interval of the set of padding time interval(s):
- determining a respectively corresponding viewer interest value that would characterize the given padding time interval if the first unpadded audiovisual segment continued through the given padding time interval and had its viewer interest value determined by a machine learning (ML) algorithm, and generating a synthetic audiovisual segment for the given padding time interval so that the synthetic audiovisual segment for the given padding time interval is characterized by the viewer interest value determined for the given padding time interval, assembling the first unpadded audiovisual segment with the synthetic audiovisual segment(s) corresponding to each padding time interval of the set of padding time interval(s) to obtain a first padded audiovisual segment data set including information indicative of a first padded audiovisual segment, and determining, by the ML algorithm, a viewer interest value for the first padded audiovisual segment considered as a whole, wherein the generation of the synthetic audiovisual segment for each given padding time interval is performed by a generative adversarial network (GAN).
1 more element
A computer system (CS) comprising:
- a processor(s) set;
- a machine readable storage device;
- and
- computer code stored on the machine readable storage device, with the computer code including instructions for causing the processor(s) set to perform operations including the following:
- receiving a first unpadded audiovisual segment data set including information indicative of a first unpadded audiovisual segment,
- determining a set of padding time interval(s) occurring before and/or after the first unpadded segment,
- for each given padding time interval of the set of padding time interval(s):
- determining a respectively corresponding viewer interest value that would characterize the given padding time interval if the first unpadded audiovisual segment continued through the given padding time interval and had its viewer interest value determined by a machine learning (ML) algorithm, and
- generating a synthetic audiovisual segment for the given padding time interval so that the synthetic audiovisual segment for the given padding time interval is characterized by the viewer interest value determined for the given padding time interval,
- assembling the first unpadded audiovisual segment with the synthetic audiovisual segment(s) corresponding to each padding time interval of the set of padding time interval(s) to obtain a first padded audiovisual segment data set including information indicative of a first padded audiovisual segment, and
- determining, by the ML algorithm, a viewer interest value for the first padded audiovisual segment considered as a whole,
- wherein the generation of the synthetic audiovisual segment for each given padding time interval is performed by a generative adversarial network (GAN).
9 more elements
Dependencies are inferred from the claim text.