Real-time motion capture system using single LiDAR device

A method of obtaining a motion of an object according to a frame rate of a Light Detection And Ranging (LiDAR) sensor based on a point cloud data output from the LiDAR sensor, the method comprising:

• obtaining, via a processor of a motion analysis device that controls an artificial intelligence (AI) based inference model which is stored in a memory, at least two first point cloud data related to the object at time points from a first time point to a second time point and second point cloud data related to the object at a third time point, wherein the third time point is later than the first time point and the second time point, and a time gap between adjacent two time points from the first time point to the third time point is determined according to the frame rate of the LiDAR sensor;
• and
• inferring, via the processor of the motion analysis device, present joint data and present orientation information for the object corresponding to the third time point by inputting data obtained from the at least two first point cloud data, the second point cloud data, at least two past orientation information corresponding to the time points from the first time point to the second time point, and at least two past joint data corresponding to time points from the first time point to the second time point to the AI based inference model,
• wherein the present joint data and the at least two past joint data each comprises at least one of position information, angle information, velocity information, and angular velocity information of each predetermined reference joint, corresponding to one of the time points from the first time point to the third time point,
• wherein the position information comprises relative position of the each predetermined reference joint with respect to a corresponding parent joint,
• wherein the angle information comprises a rotated angle of the each predetermined reference joint with respect to a corresponding parent joint,
• wherein the velocity information is calculated by dividing a change in position of each predetermined reference joint in a past frame and a present frame by a time interval between the past frame and the present frame,
• wherein the angular velocity information is calculated by dividing a change in angle information of the each predetermined reference joint in the past frame and the present frame by the time interval between the past frame and the present frame,
• wherein the present orientation information and the at least two past orientation information each indicates a direction in which the object faces within a sensing area of the LiDAR sensor,
• wherein the direction in which the object faces is obtained based on a relative relationship between a reference axis defining a reference direction in the sensing area of the LiDAR sensor and an orientation axis defined at a representative point of the object,
• wherein the AI based inference model is a pre-trained model that is trained by using a set of previous-frame point cloud data, current-frame point cloud data, previous-frame joint data, previous-frame orientation information, and first current-frame joint data and first current-frame orientation information,
• wherein the current-frame joint data and the current-frame orientation information are ground-truth to compare with second current-frame joint data and second current-frame orientation information each inferred using the AI based inference model which is training and to training the AI based inference model using the compare result,
• wherein the current-frame joint data and the current-frame orientation information are obtained by at least one predetermined sensor operating while a LiDAR sensor that obtains the previous-frame point cloud data and the current-frame point cloud data is operating, and
• wherein the AI based inference model is configured to, by using the at least two past joint data, the at least two past orientation information, the at least two first point cloud data, and the second point cloud data, distinguish and infer a joint that is overlapped with or hidden by another body part by considering a change of the at least two past joint data, a change of the at least two past orientation information and a change of the at least two first point cloud data and the second point cloud data that is continuous in time series between the first time point and the third time point.