Forecasting multiple poses based on a graphical image
AssigneeAdobe Inc.
InventorsJimei Yang, Yu-Wei Chao, Scott Cohen, Brian Price
Life of this patent
Estimating the term…
Abstract
A forecasting neural network receives data and extracts features from the data. A recurrent neural network included in the forecasting neural network provides forecasted features based on the extracted features. In an embodiment, the forecasting neural network receives an image, and features of the image are extracted. The recurrent neural network forecasts features based on the extracted features, and pose is forecasted based on the forecasted features. Additionally or alternatively, additional poses are forecasted based on additional forecasted features.
Claims20
A method of forecasting a position of a subject depicted by an image, the method comprising:
- receiving, by a forecasting neural network, an image depicting a subject, wherein the forecasting neural network comprises an encoder neural network, a recurrent neural network, and a decoder neural network;
- extracting, by the encoder neural network, a feature of the received image;
- providing the extracted feature to the recurrent neural network;
- determining, by the recurrent neural network, a modification to the extracted feature;
- generating, by the recurrent neural network, a forecasted feature based on memory information describing the determined modification;
- providing the forecasted feature to the decoder neural network;
- and
- generating, by the decoder neural network, a set of keypoints, wherein each keypoint in the set of keypoints indicates a forecasted position of a respective portion of the image subject.
A non-transitory computer-readable medium embodying program code for producing a pose from an input image, the program code comprising instructions which, when executed by a processor, cause the processor to perform operations comprising:
- receiving, by a forecasting neural network, an image depicting a subject, wherein the forecasting neural network comprises an encoder neural network, a recurrent neural network, and a decoder neural network;
- extracting, by the encoder neural network, a feature of the received image;
- providing the extracted feature to the recurrent neural network;
- determining, by the recurrent neural network, a modification to the extracted feature;
- generating, by the recurrent neural network, a forecasted feature based memory information describing on the determined modification;
- providing the forecasted feature to the decoder neural network;
- and
- generating, by the decoder neural network, a set of keypoints, wherein each keypoint in the set of keypoints indicates a forecasted position of a respective portion of the image subject.
A system for producing a pose from an input image, the system comprising:
- a means for receiving, by a forecasting neural network, an image depicting a subject, wherein the forecasting neural network comprises an encoder neural network, a recurrent neural network, and a decoder neural network;
- a means for extracting, by the encoder neural network, a feature of the received image;
- a means for providing the extracted feature to the recurrent neural network;
- a means for determining, by the recurrent neural network, a modification to the extracted feature;
- a means for generating, by the recurrent neural network, a forecasted feature based on memory information describing the determined modification;
- a means for providing the forecasted feature to the decoder neural network;
- and
- a means for generating, by the decoder neural network, a set of keypoints, wherein each keypoint in the set of keypoints indicates a forecasted position of a respective portion of the image subject.
Dependencies are inferred from the claim text.