Virtual object machine learning
AssigneeSnap Inc.
InventorsXuehan Xiong, Zehao Xue
Life of this patent
Estimating the term…
Abstract
A machine learning scheme can be trained on a set of labeled training images of a subject in different poses, with different textures, and with different background environments. The label or marker data of the subject may be stored as metadata to a 3D model of the subject or rendered images of the subject. The machine learning scheme may be implemented as a supervised learning scheme that can automatically identify the labeled data to create a classification model. The classification model can classify a depicted subject in many different environments and arrangements (e.g., poses).
Claims18
A method comprising:
- storing, on a client device, a convolutional neural network trained to detect different gestures of an object, the convolutional neural network trained on a plurality of images of a three-dimensional model of the object, each of the plurality of images being a render of the three-dimensional model of the object in a different gesture;
- generating, using an image sensor of the client device, an image depicting a physical object;
- classifying, using the convolutional neural network, the physical object as being in a gesture by applying the convolution neural network to the generated image depicting the physical object;
- displaying with the image, on a display device of the client device, additional content associated with the identified gesture of the physical object;
- and
- publishing, to a network platform, the additional content and the image as an ephemeral message.
A client device comprising:
- one or more processors;
- an image sensor;
- a display device;
- and
- a memory storing instructions that, when executed by the one or more processors, cause the client device to perform operations comprising:
- store a convolutional neural network trained to detect different gestures of an object, the convolutional neural network trained on a plurality of images of a three-dimensional model of the object, each of the plurality of images being a render of the three-dimensional model of the object in a different gesture;
- generate, using the image sensor, an image depicting a physical object;
- classify, using the convolutional neural network, the physical object as being in a gesture by applying the convolution neural network to the generated image depicting the physical object;
- display with the image, on the display device, additional content associated with the identified gesture of the physical object;
- and
- provide, to a network platform, the additional content and the image as an ephemeral message.
A non-transitory machine-readable storage device embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
- store a convolutional neural network trained to detect different gestures of an object, the convolutional neural network trained on a plurality of images of a three-dimensional model of the object, each of the plurality of images being a render of the three-dimensional model of the object in a different gesture;
- generate, using the image sensor, an image depicting a physical object;
- classify, using the convolutional neural network, the physical object as being in a gesture by applying the convolution neural network to the generated image depicting the physical object;
- display with the image, on the display device, additional content associated with the identified gesture of the physical object;
- and
- provide, to a network platform, the additional content and the image as an ephemeral message.
Dependencies are inferred from the claim text.