Back to results
US 11,448,884 B2Granted
Image based finger tracking plus controller tracking
AssigneeSony Interactive Entertainment Inc.
InventorYutaka Yokokawa
Life of this patent
Estimating the term…
Abstract
An image from a camera on, e.g., a VR HMD is identified of a hand grasping a computer game controller. The image may be cropped to a region containing the controller and the hand to simplify processing, and a virtual representation of the hand is presented on a display such as the HMD, with the virtual representation being generated based on both image analysis of the region and a touch signal from the controller.
Claims19
An apparatus comprising:
- at least one processor programmed with instructions to:
- identify an image from at least one camera of a hand grasping a computer game controller;
- crop the image to a region containing the controller and the hand;
- and
- present on a computerized display a virtual representation of the hand generated at least in part based on image analysis of the region and at least one touch signal from the controller, wherein the instructions are executable by the processor to:
- execute a machine learning (ML) module to generate the virtual representation by modifying keypoints of a template image based on the image in the region and the touch signal from the controller.
3 more elements
A method, comprising:
- identifying an image of a hand grasping a computer simulation controller;
- receiving from the computer simulation controller at least one touch signal;
- and
- based on both the image of the hand and the touch signal, generating an image of a virtual hand for display thereof.
1 more element
A device comprising:
- at least one computer storage that is not a transitory signal and that comprises instructions executable by at least one processor to:
- receive at least one image of a human hand grasping a computer game controller;
- receive at least one touch signal from the controller;
- and
- generate, based on both the image and the touch signal, a virtual hand representing the human hand for presentation of the virtual hand on at least one computerized display.
2 more elements
Dependencies are inferred from the claim text.