Back to results
US 11,636,681 B2Granted
Anticipating future video based on present video
AssigneeMeta Platforms, Inc.
InventorsHeng Wang, Du Le Hong Tran, Antoine Miech, Lorenzo Torresani
Life of this patent
Estimating the term…
Abstract
In one embodiment, a method includes accessing a first set of images of multiple images of a scene, wherein the first set of images show the scene during a time period. The method includes generating, by processing the first set of images using a first machine-learning model, one or more attributes representing observed actions performed in the scene during the time period. The method includes predicting, by processing the generated one or more attributes using a second machine-learning model, one or more actions that would happen in the scene after the time period.
Claims19
A method comprising:
- accessing a first set of images of a plurality of images of a scene, wherein the first set of images show the scene during a time period;
- generating, by processing the first set of images using a first machine-learning model, one or more attributes representing one or more of observed actions performed in the scene during the time period or visual attributes appearing in the scene during the time period, wherein the one or more attributes further includes probabilities indicating one or more of likelihoods that the observed actions are accurately represented or likelihoods that the visual attributes are accurately represented, wherein the first machine-learning model receives the first set of images as input and wherein the one or more attributes represent at least one of observed characteristics of the scene during the time period, a description of the scene during the time period, a description of an action performed in the scene, or a description of entities in the scene, and
- predicting, with a second machine-learning model, one or more actions that would happen in the scene after the time period based on the one or more of the observed actions or the visual attributes, and the probabilities generated by the first machine-learning model,
- wherein the first machine-learning model and the second machine-learning model are different types of learning models.
1 more element
One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
- access a first set of images of a plurality of images of a scene, wherein the first set of images show the scene during a time period;
- generate, by processing the first set of images using a first machine-learning model, one or more attributes representing one or more of observed actions performed in the scene during the time period or visual attributes appearing in the scene during the time period, wherein the one or more attributes further includes probabilities indicating one or more of likelihoods that the observed actions are accurately represented or likelihoods that the visual attributes are accurately represented, wherein the first machine-learning model receives the first set of images as input and wherein the one or more attributes represent at least one of observed characteristics of the scene during the time period, a description of the scene during the time period, a description of an action performed in the scene, or a description of entities in the scene;
- and
- predict, with a second machine-learning model, one or more actions that would happen in the scene after the time period based on the one or more of the observed actions or the visual attributes, and the probabilities generated by the first machine-learning model,
- wherein the first machine-learning model and the second machine-learning model are different types of learning models.
2 more elements
A system comprising:
- one or more processors;
- and
- one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
- access a first set of images of a plurality of images of a scene, wherein the first set of images show the scene during a time period;
- generate, by processing the first set of images using a first machine-learning model, one or more attributes representing one or more of observed actions performed in the scene during the time period or visual attributes appearing in the scene during the time period, wherein the one or more attributes further includes probabilities indicating one or more of likelihoods that the observed actions are accurately represented or likelihoods that the visual attributes are accurately represented, wherein the first machine-learning model receives the first set of images as input and wherein the one or more attributes represent at least one of observed characteristics of the scene during the time period, a description of the scene during the time period, a description of an action performed in the scene, or a description of entities in the scene;
- and
- predict, with a second machine-learning model, one or more actions that would happen in the scene after the time period based on the one or more of the observed actions or the visual attributes, and the probabilities generated by the first machine-learning model,
- wherein the first machine-learning model and the second machine-learning model are different types of learning models.
5 more elements
Dependencies are inferred from the claim text.