Back to results
US 10,705,216 B2Granted
Three-dimensional point cloud tracking apparatus and method using recurrent neural network
AssigneeInstitute for Information Industry
InventorsChih-Wei Wang, Ywe-Ting Tsai
Life of this patent
Estimating the term…
Abstract
The embodiments of the present invention provide a three-dimensional point cloud tracking apparatus and method using a recurrent neural network. The three-dimensional point cloud tracking apparatus and method can track the three-dimensional point cloud of the entire environment and model the entire environment by using a recurrent neural network model. Therefore, the three-dimensional point cloud tracking apparatus and method can be used to reconstruct the three-dimensional point cloud of the entire environment at the current moment and also can be used to predict the three-dimensional point cloud of the entire environment at a later moment.
Claims12
A three-dimensional point cloud tracking apparatus using a recurrent neural network, comprising:
- an input/output interface, configured to receive different observed three-dimensional point clouds at different moments in an environment, wherein the observed three-dimensional point clouds are obtained by a scanning of at least one LiDAR;
- a memory, configured to store at least one memory three-dimensional point cloud related to the environment;
- and
- a processor, electrically connected to the input/output interface and the memory respectively and configured to receive the observed three-dimensional point clouds and the at least one memory three-dimensional point cloud, wherein when receiving the observed three-dimensional point cloud of the environment at a first moment, the processor utilizes at least one recurrent neural network model to perform an environment reconstruction operation on the observed three-dimensional point cloud and the at least one memory three-dimensional point cloud to obtain a reconstructed three-dimensional point cloud of the environment at the first moment, and then utilizes the recurrent neural network model again to perform an environment prediction operation on the at least one memory three-dimensional point cloud and a blank three-dimensional point cloud to obtain a predicted three-dimensional point cloud of the environment at a second moment, wherein the second moment is later than the first moment;
- wherein the at least one memory three-dimensional point cloud comprises a first memory three-dimensional point cloud and a second memory three-dimensional point cloud, and the step of utilizing the recurrent neural network model to perform the environment reconstruction operation on the observed three-dimensional point cloud and the at least one memory three-dimensional point cloud to obtain the reconstructed three-dimensional point cloud of the environment at the first moment comprises:
- performing a first sparse convolution operation on the observed three-dimensional point cloud to obtain a first calculated three-dimensional point cloud;
- performing a second sparse convolution operation on the first calculated three-dimensional point cloud and the first memory three-dimensional point cloud to obtain a second calculated three-dimensional point cloud and updating the first memory three-dimensional point cloud with the second calculated three-dimensional point cloud;
- and
- performing a third sparse convolution operation on the second calculated three-dimensional point cloud and the second memory three-dimensional point cloud to obtain the reconstructed three-dimensional point cloud of the environment at the first moment and updating the second memory three-dimensional point cloud with the reconstructed three-dimensional point cloud.
6 more elements
A three-dimensional point cloud tracking method using a recurrent neural network and executed in a three-dimensional point cloud tracking apparatus, the three-dimensional point cloud tracking apparatus comprising an input/output interface, a memory, and a processor, the three-dimensional point cloud tracking method comprising:
- configuring the input/output interface to receive different observed three-dimensional point clouds at different moments in an environment, wherein the observed three-dimensional point clouds are obtained by a scanning of at least one LiDAR;
- configuring the memory to store at least one memory three-dimensional point cloud related to the environment;
- and
- configuring the processor to receive the observed three-dimensional point clouds and the at least one memory three-dimensional point cloud, configuring, when receiving the observed three-dimensional point cloud of the environment at a first moment, the processor to utilize at least one recurrent neural network model to perform an environment reconstruction operation on the observed three-dimensional point cloud and the at least one memory three-dimensional point cloud to obtain a reconstructed three-dimensional point cloud of the environment at the first moment, and configuring the processor to utilize the recurrent neural network model again to perform an environment prediction operation on the at least one memory three-dimensional point cloud and a blank three-dimensional point cloud to obtain a predicted three-dimensional point cloud of the environment at a second moment, wherein the second moment is later than the first moment;
- wherein the at least one memory three-dimensional point cloud comprises a first memory three-dimensional point cloud and a second memory three-dimensional point cloud, and the step of utilizing the recurrent neural network model to perform the environment reconstruction operation on the observed three-dimensional point cloud and the at least one memory three-dimensional point cloud to obtain the reconstructed three-dimensional point cloud of the environment at the first moment comprises:
- performing a first sparse convolution operation on the observed three-dimensional point cloud to obtain a first calculated three-dimensional point cloud;
- performing a second sparse convolution operation on the first calculated three-dimensional point cloud and the first memory three-dimensional point cloud to obtain a second calculated three-dimensional point cloud and updating the first memory three-dimensional point cloud with the second calculated three-dimensional point cloud;
- and
- performing a third sparse convolution operation on the second calculated three-dimensional point cloud and the second memory three-dimensional point cloud to obtain the reconstructed three-dimensional point cloud of the environment at the first moment and updating the second memory three-dimensional point cloud with the reconstructed three-dimensional point cloud.
6 more elements
Dependencies are inferred from the claim text.