System and method for reconstructing a 3D human body using compact kit of depth cameras

A system for 3D human reconstruction using a compact kit of depth cameras, including four blocks:

• a data collection block including a module of two depth cameras aims to capture 3D images of a digitized person;
• output of the data collection block is a raw point cloud data which is captured by the depth cameras, depth data obtained by the two cameras is passed to a deep learning network to extract positions of human joints in the depth data;
• then using this data to interpolate 3D positions of human joints in 3D space, a distance between cameras and the digitized person is set up carefully to ensure capturing full image of the person (from head to toe) (about ≥ 1.5 m), an angle between two cameras can be adjusted flexibly, after completing system installation, positions of two depth cameras are fixed for a 3D human reconstruction process;
• a point cloud standardization block includes two modules:
• a point cloud filtering module and a point cloud calibrating module, in which:
• the point cloud filtering module is responsible for removing redundant point clouds in the 3D image (for example:
• point clouds of the ground, ceiling . . . ) and removing noise caused by camera devices;
• the point cloud filtering module comprises a pre-filter and a fine filter, In particular:
• the pre-filter is accountable for removing data of objects appearing around the digitized person and the ground;
• the fine filter is accountable for removing points or clusters of points which stay close or stuck to human's point cloud, these points are caused by errors of point cloud receivers;
• the point cloud calibrating module performs mathematical calculations of spatial transformation to move the obtained point cloud to a pre-defined coordinate system, being ready for a human digitization block, specifically, in the coordinate system, the origin is placed at the top of a head of the human point cloud;
• the y-axis is coincident with vertical direction of the digitized person, the x-axis is perpendicular to the y-axis and coincident with a standing direction of the digitized person, the z-axis is determined based on a Cartesian coordinate system rule;
• the human digitization block includes two modules:
• a human parametric model module and a point cloud optimization module, in which:
• the human parametric model module is a model comprising shape control parameters such as height, weight, chest circumference, waist circumference parameters;
• the human parametric model module includes three main components:
• shape parameters;
• pose parameters;
• skeletal simulation, in particular:
• shape parameters are 10 parameters controlling and transforming the human body shape into different types (such as tall, short, fat, thin, . . . );
• pose parameters are joints parameters of a skeletal system which simulates the human body, the number of joints in the parametric model is 24 and the number of pose parameters is 72;
• skeletal simulation is to simulate a bones system of the human body, including 24 joints (head, neck, right shoulder, left shoulder, . . . ), these joints are accountable for ensuring the movement of the human parametric model is similar to that of a real person;
• the point cloud optimization module calculates shape control parameters obtained by the human parametric model module so that the shape generated after the optimization process approximates the shape of the point cloud, the point cloud optimization module includes two main components:
• pose optimization, shape optimization, firstly, length of bones and angle between joints of the human body are calculated by pose optimization, the main idea of this step is to minimize errors between length of bones, angle of joints in the parametric model and those in the calibrated point cloud, next, shape parameters of the parametric model are similarly calculated by shape optimization to find a set of parameters according to the calibrated point cloud, by this way, the optimization process is implemented by minimizing the objective function which represents errors between the parametric model and point cloud data, the output of this module is a set of model parameters of the human parametric model that approximates the real model shape, this set of parameters is passed to an output block to generate a movable 3D model representing the digitized person;
• the output block:
• displaying the human body in 3D space with a pre-defined file format.