Method and system for matching 2d human poses from multiple views
AssigneeHinge Health, Inc.
InventorWenxin ZHANG
Life of this patent
Estimating the term…
Abstract
This disclosure is directed to a method and system for matching human pose data in the form of 2D skeletons for the purposes of 3D reconstruction. The system may comprise a scoring module that assigns an affinity score to each pair of cross-view 2D skeletons, a matching module that assigns optimal pairwise matches based on the affinity scores, a grouping module that assigns each 2D skeleton to a group such that each group corresponds to a unique person, based on the pairwise matches; and a temporal consistency module that assigns each group an ID that maintains correspondence to the same person over the multi-video sequence.
Claims22
A method of identifying humans across first and second views corresponding to different cameras, the method comprising:
- for each of multiple skeletons in the first view,
- performing a pairwise scoring with each of multiple skeletons in the second view to produce multiple affinity scores, wherein each affinity score is associated with a corresponding one of the multiple skeletons in the second view;
- identifying a matching skeleton from among the multiple skeletons in the second view based on the affinity scores;
- and
- forming multiple skeleton sets by assigning each of the multiple skeletons in the first view and the matching skeleton identified in the second view to a different grouping that corresponds to a unique one of the humans.
A non-transitory medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:
- obtaining (i) a first video stream that comprises a first series of frames and is generated by a first camera and (ii) a second video stream that comprises a second series of frames and is generated by a second camera,
- wherein the first and second video streams are synchronized, such that each frame in the first series of frames is associated with a corresponding frame in the second series of frames;
- generating (i) first skeletons for humans that are viewable in the first video stream and (ii) second skeletons for humans that are viewable in the in the second video stream;
- and
- forming matches between the first and second skeletons, such that each of the first skeletons is matched with one of the second skeletons in the corresponding frame.
A method of identifying humans across first and second views corresponding to different cameras, the method comprising:
- for each of multiple skeletons in the first view,
- performing a pairwise comparison with each of multiple skeletons in the second view;
- identifying, based on the pairwise comparison, a matching skeleton from among the multiple skeletons in the second view;
- forming multiple skeleton sets by assigning each of the multiple skeletons in the first view and the matching skeleton identified in the second view to a different grouping that corresponds to a unique one of the humans;
- and
- for each of the multiple skeleton sets,
- assigning an identifier to each skeleton in that skeleton set across a sequence of frames corresponding to the first and second views.
A method for identifying humans across first and second views corresponding to different cameras, the method comprising:
- for each of multiple skeletons in the first view,
- identifying a matching skeleton from among multiple skeletons in the second view;
- forming multiple skeleton sets by assigning each of the multiple skeletons in the first view and the matching skeleton identified in the second view to a different grouping that corresponds to a unique one of the humans;
- and
- for each of the multiple skeleton sets,
- assigning an identifier to each skeleton in that skeleton set across a sequence of frames corresponding to the first and second views.
Dependencies are inferred from the claim text.