Method and system for matching 2D human poses from multiple views
AssigneeHinge Health, Inc.
InventorWenxin Zhang
Life of this patent
Estimating the term…
Abstract
This disclosure is directed to a method and system for matching human pose data in the form of 2D skeletons for the purposes of 3D reconstruction. The system may comprise a scoring module that assigns an affinity score to each pair of cross-view 2D skeletons, a matching module that assigns optimal pairwise matches based on the affinity scores, a grouping module that assigns each 2D skeleton to a group such that each group corresponds to a unique person, based on the pairwise matches; and a temporal consistency module that assigns each group an ID that maintains correspondence to the same person over the multi-video sequence.
Claims10
A method of identifying humans between two or more camera views from two-dimensional (2D) skeletons of the humans of each view, the method comprising:
- a) for each 2D skeleton in each of the two or more camera views, performing a pairwise scoring with each of the 2D skeletons in another of the two or more camera views and assigning an affinity score to each pair,
- wherein the pairwise scoring of a pair of 2D skeletons from a pair of camera views comprises:
- for each element of multiple elements,
- modelling a pair of rays from the pair of camera views to that element of the 2D skeleton,
- determining a distance between the pair of rays, and
- generating the affinity score that is representative of a weighted sum of the multiple distances determined for the multiple elements;
- b) identifying a best match of a first 2D skeleton in a first camera view to a second 2D skeleton in a second camera view by maximizing the affinity score of the pair;
- and
- c) grouping the 2D skeletons by identifying a first set of 2D skeletons in the first camera view, the first set of 2D skeletons relating to the humans in the first camera view, with a second set of 2D skeletons in the second camera view using the best match.
A non-transitory, computer-readable medium with instructions that, when executed by a system, cause the system to perform operations comprising:
- obtaining two-dimensional (2D) skeletons of humans in multiple camera views of a given environment;
- for each 2D skeleton in each of the multiple camera views, performing a pairwise scoring with each of the 2D skeletons in another of the multiple camera views and assigning an affinity score to each pair,
- wherein the pairwise scoring of a pair of 2D skeletons from a pair of camera views comprises:
- for each element of multiple elements of the 2D skeleton,
- modelling a ray from each camera view to that element of the 2D skeleton,
- determining a distance between the pair of rays, and
- generating the affinity score that is representative of a weighted sum of the multiple distances determined for the multiple elements;
- identifying a best match of a first 2D skeleton in a first camera view to a second 2D skeleton in a second camera view by maximizing the affinity score of the pair;
- and
- grouping the 2D skeletons by identifying a first set of skeletons in the first camera view, the first set of skeletons relating to humans in the first camera view, with a second set of skeletons in the second camera view using the best match.
Dependencies are inferred from the claim text.