System for estimating a three dimensional pose of one or more persons in a scene

A system for estimating a three dimensional pose of one or more persons in a scene, the system comprising:

• a plurality of cameras, the plurality of cameras including a first camera and a second camera, the first camera configured to capture a first image of the scene and the second camera configured to capture a second image of the scene;
• and
• a data processor including at least one hardware component, the data processor configured to execute computer executable instructions, the computer executable instructions comprising instructions for:
• receiving the first image of the scene from the first camera;
• receiving the second image of the scene from the second camera;
• during training of the system, extracting features from the first and second images of the scene for providing inputs to a first branch pose estimation network, the first branch pose estimation network comprising a first convolutional neural network configured to generate respective three dimensional poses of one or more persons in the scene from the first and second images;
• during the training of the system, extracting features from the first and second images of the scene for providing inputs to a second branch pose estimation network, the second branch pose estimation network comprising a second convolutional neural network configured to generate respective two dimensional poses of one or more persons in the scene from the first and second images;
• during the training of the system, generating a training signal from the second branch pose estimation network for input into the first branch pose estimation network;
• during inference of the system, extracting features from a particular image of an inference scene for providing inputs to the first convolutional neural network;
• during the inference of the system, generating a volumetric heatmap using the first convolutional neural network;
• and
• during the inference of the system, applying a maximization function to the volumetric heatmap to obtain a three dimensional pose of one or more persons in the inference scene.

A system for estimating a three dimensional pose of one or more persons in a scene, the system comprising:

• a plurality of cameras, the plurality of cameras including a first camera and a second camera, the first camera configured to capture a first image of the scene and the second camera configured to capture a second image of the scene;
• and
• a data processor including at least one hardware component, the data processor configured to execute computer executable instructions, the computer executable instructions comprising instructions for:
• receiving the first image of the scene from the first camera;
• receiving the second image of the scene from the second camera;
• during training of the system, extracting features from the first and second images of the scene for providing inputs to a first branch pose estimation network, the first branch pose estimation network comprising a first convolutional neural network configured to generate respective three dimensional poses of one or more persons in the scene from the first and second images;
• during the training of the system, extracting features from the first and second images of the scene for providing inputs to a second branch pose estimation network;
• during the training of the system, generating a training signal from the second branch pose estimation network for input into the first branch pose estimation network;
• during inference of the system, extracting features from a particular image of an inference scene for providing inputs to the first convolutional neural network;
• during the inference of the system, generating a volumetric heatmap using the first branch pose estimation network;
• and
• during the inference of the system, applying a maximization function to the volumetric heatmap to obtain a three dimensional pose of one or more persons in the inference scene.