Method, mobile device, head-mounted display, and system for estimating hand pose

A method performed by a head-mounted display (HMD), comprising:

• receiving, by at least one self-owned or agent processor of the HMD, information indicating which hand is used to perform a gesture to be estimated from first image data, and not detecting, by the at least one self-owned or agent processor of the HMD, the information indicating which hand is used to perform the gesture by an image-based detection method;
• wherein the information indicating which hand is used to perform the gesture is detected by a mobile device from first data sensed by a sensing device of the mobile device when the HMD is tethered to the mobile device and when the mobile device is held by a first hand;
• and
• wherein the gesture is performed by a second hand and the information indicating which hand is used to perform the gesture is an updated indication with respect to the gesture considering a hand switch state during a first elapsed duration from a sensing time of the first data to a sensing time of the first image data;
• and
• performing, by the at least one self-owned or agent processor of the HMD, side-adapted hand pose estimation on the first image data using the information indicating which hand is used to perform the gesture.

A method performed by a mobile device and a head-mounted display (HMD), comprising:

• sensing, by a sensing device of the mobile device when the mobile device is tethered to the HMD and when the mobile device is held by a first hand, first data;
• detecting, by at least one processor of the mobile device, information indicating which hand is used to perform a gesture to be estimated from first image data by at least one self-owned or agent processor of the HMD, wherein the information indicating which hand is used to perform the gesture is detected from the first data;
• sending, by the at least one processor of the mobile device, the information indicating which hand is used to perform the gesture to the at least one self-owned or agent processor of the HMD;
• receiving, by the at least one self-owned or agent processor of the HMD, the information indicating which hand is used to perform the gesture, and not detecting, by the at least one self-owned or agent processor of the HMD, the information indicating which hand is used to perform the gesture by an image-based detection method;
• wherein the gesture is performed by a second hand and the information indicating which hand is used to perform the gesture is an updated indication with respect to the gesture considering a hand switch state during a first elapsed duration from a sensing time of the first data to a sensing time of the first image data;
• and
• performing, by the at least one self-owned or agent processor of the HMD, side-adapted hand pose estimation on the first image data using the information indicating which hand is used to perform the gesture.