Real-time hand-held markerless human motion recording and avatar rendering in a mobile platform
AssigneeInnopeak Technology, Inc.
InventorsZhong Li, Shuxue Quan, Yi Xu
Life of this patent
Estimating the term…
Abstract
A computer system obtains an image of a scene captured by a camera and identifies a two-dimensional (2D) pose of the person in the image. The 2D pose includes a plurality of 2D joint positions in the image. The 2D pose is converted to a three-dimensional (3D) pose of the person including a plurality of 3D joint positions. The computer system determines a rotation angle of each joint relative to a T-pose of the person based on the plurality of 3D joint positions. The rotation angle of each joint is applied to a skeleton template of the avatar. The computer system renders the skeleton template of the avatar having the rotation angle for each joint.
Claims20
A method for driving an avatar, comprising:
- obtaining an image of a scene captured by a camera;
- identifying a two-dimensional (2D) pose of a person in the image, the 2D pose including a plurality of 2D joint positions in the image;
- converting the 2D pose to a three-dimensional (3D) pose of the person, the 3D pose including a plurality of 3D joint positions;
- determining a rotation angle of each joint relative to a T-pose of the person based on the plurality of 3D joint positions;
- applying the rotation angle of each joint to a skeleton template of an avatar;
- and
- rendering the skeleton template of the avatar having the rotation angle for each joint;
- wherein determining the rotation angle of each joint relative to the T-pose of the person, comprises:
- building a kinematic skeleton tree starting from a hip key point based on the plurality of 3D joint positions;
- in accordance with the kinematic skeleton tree, associating each joint with a parent key point and a child key point to form a respective plane and generate a respective normal vector;
- and
- for each joint, determining the rotation angle from an angle between the respective vector associated with the kinematic skeleton tree and a corresponding vector associated with the T-pose of the person.
A computer system, comprising:
- one or more processors;
- and
- a memory having instructions stored thereon, which when executed by the one or more processors cause the processors to perform a method for driving an avatar, wherein the method comprises:
- obtaining an image of a scene captured by a camera;
- identifying a two-dimensional (2D) pose of a person in the image, the 2D pose including a plurality of 2D joint positions in the image;
- converting the plurality of 2D joint positions to a plurality of three-dimensional (3D) joint positions;
- determining, based on the plurality of 3D joint positions, a rotation angle of each joint relative to a T-pose of the person;
- applying the rotation angle of each joint to a skeleton template of an avatar;
- and
- rendering the skeleton template of the avatar having the rotation angle for each joint;
- wherein determining, based on the plurality of 3D joint positions, the rotation angle of each joint relative to the T-pose of the person, comprises:
- building a kinematic skeleton tree starting from a hip key point based on the plurality of 3D joint positions;
- in accordance with the kinematic skeleton tree, associating each joint with a parent key point and a child key point to form a respective plane and generate a respective normal vector;
- and
- for each joint, determining the rotation angle from an angle between the respective vector associated with the kinematic skeleton tree and a corresponding vector associated with the T-pose of the person.
A non-transitory computer-readable medium, having instructions stored thereon, which when executed by one or more processors cause the processors to perform a method for driving an avatar, wherein the method comprises:
- obtaining an image of a scene captured by a camera;
- identifying a two-dimensional (2D) pose of a person in the image, the 2D pose including a plurality of 2D joint positions in the image;
- determining a plurality of three-dimensional (3D) joint positions based on the plurality of 2D joint positions;
- determining a rotation angle of each joint relative to a T-pose of the person, based on the plurality of 3D joint positions;
- obtaining a skeleton template of an avatar having the rotation angle for each joint;
- and
- rendering, at a global position of the avatar in a camera coordinate, the skeleton template of the avatar having the rotation angle of each joint;
- wherein determining the rotation angle of each joint relative to the T-pose of the person based on the plurality of 3D joint positions, comprises:
- building, based on the plurality of 3D joint positions, a kinematic skeleton tree starting from a hip key point;
- in accordance with the kinematic skeleton tree, associating each joint with a parent key point and a child key point to form a respective plane and generate a respective normal vector;
- and
- for each joint, determining the rotation angle from an angle between the respective vector associated with the kinematic skeleton tree and a corresponding vector associated with the T-pose of the person.
Dependencies are inferred from the claim text.