Method of predicting occupancy of unseen areas for path planning, associated device, and network training method

A method of predicting occupancy of unseen areas in a region of interest (ROI) comprising:

• obtaining a depth image of the ROI, the depth image being captured from a first height;
• generating an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI, each cell having an occupancy score;
• and
• generating an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, each cell having an occupancy score, and
• wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

A method of path planning and navigating to a goal position comprising:

• capturing a depth image of a region of interest (ROI) from a first height;
• generating an occupancy map corresponding to the ROI based on the captured depth image;
• generating an inpainted map by inputting the occupancy map into a trained inpainting network;
• updating a global map based on the inpainted map;
• planning a path to the goal position based on the updated global map;
• and
• traveling to the goal position based on the planned path.

A device for training an inpainting network to predict occupancy of unseen areas in a region of interest (ROI), the device comprising:

• a memory configured to store instructions;
• and
• a processor configured to execute the instructions to:
• obtain an upper depth image of the ROI from an upper three-dimensional (3D) mapping sensor positioned at a first height and a lower depth image of the ROI from a lower 3D mapping sensor positioned at a second height lower than the first height;
• generate an upper occupancy map based on the upper depth image and a lower occupancy map based on the lower depth image;
• generate a ground truth map by combining the upper occupancy map and the lower occupancy map;
• generate an inpainted map by inputting the lower occupancy map into the inpainting network;
• and
• train the inpainting network by comparing the inpainted map to the ground truth map.

A device for predicting occupancy of unseen areas in a region of interest (ROI), the device comprising:

• a memory configured to store instructions;
• and
• a processor configured to execute the instructions to:
• obtain a depth image of the ROI, the depth image being captured from a first height;
• generate an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI, each cell having an occupancy score;
• and
• generate an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, each cell having an occupancy score, and
• wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

A device for path planning and navigating to a goal position, the device comprising:

• a memory configured to store instructions;
• and
• a processor configured to execute the instructions to:
• capture a depth image of a region of interest (ROI) from a first height;
• generate an occupancy map corresponding to the ROI based on the captured depth image;
• generate an inpainted map by inputting the occupancy map into a trained inpainting network;
• update a global map based on the inpainted map;
• plan a path to the goal position based on the updated global map;
• and
• travel to the goal position based on the planned path.

A non-transitory computer-readable medium storing instructions, the instructions comprising:

• one or more instructions that, when executed by one or more processors of a mobile device, cause the one or more processors to:
• obtain an upper depth image of a region of interest (ROI) from an upper three-dimensional (3D) mapping sensor positioned at a first height and a lower depth image of the ROI from a lower 3D mapping sensor positioned at a second height lower than the first height;
• generate an upper occupancy map based on the upper depth image and a lower occupancy map based on the lower depth image;
• generate a ground truth map by combining the upper occupancy map and the lower occupancy map;
• generate an inpainted map by inputting the lower occupancy map into the inpainting network;
• and
• train the inpainting network by comparing the inpainted map to the ground truth map.

A non-transitory computer-readable medium storing instructions, the instructions comprising:

• one or more instructions that, when executed by one or more processors of a mobile device, cause the one or more processors to:
• obtain a depth image of a region of interest (ROI), the depth image being captured from a first height;
• generate an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI, each cell having an occupancy score;
• and
• generate an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, each cell having an occupancy score, and
• wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

A non-transitory computer-readable medium storing instructions, the instructions comprising:

• one or more instructions that, when executed by one or more processors of a mobile device, cause the one or more processors to:
• capture a depth image of a region of interest (ROI) from a first height;
• generate an occupancy map corresponding to the ROI based on the captured depth image;
• generate an inpainted map by inputting the occupancy map into a trained inpainting network;
• update a global map based on the inpainted map;
• plan a path to the goal position based on the updated global map;
• and
• travel to the goal position based on the planned path.