Systems and methods for visuo-tactile object pose estimation
AssigneeHonda Motor Co., Ltd.
InventorsNawid Jamali, Huckleberry Febbo, Karankumar Patel, Soshi Iba, Akinobu Hayashi, Itoshi Naramura
Life of this patent
Estimating the term…
Abstract
Systems and methods for visuo-tactile object pose estimation are provided. In one embodiment, a method includes receiving image data about an object and receiving depth data about the object. The method also includes generating a visual estimate of the object based on the image data and the depth data. The method further includes receiving tactile data about the object. The method yet further includes generating a tactile estimate of the object based on the tactile data. The method includes estimating a pose of the object based on the visual estimate and the tactile estimate.
Claims20
A system for visuo-tactile object pose estimation, comprising:
- a sensor module, implemented via a processor, configured to:
- receive image data about an object in an environment;
- receive depth data about the object in the environment;
- and
- receive tactile data about the object in the environment;
- a visual module, implemented via the processor, configured to generate a visual estimate of the object by processing a combined dataset of the image data and the depth data into a reconstructed fused image;
- a tactile module, implemented via the processor, configured to separately generate a tactile estimate of the object by processing the tactile data;
- and
- a pose module, implemented via the processor, configured to estimate a pose of the object based on the visual estimate and the tactile estimate, wherein
- the visual module employs a first convolutional neural network (CNN) to process the combined dataset of the image data and the depth data.
A method for visuo-tactile object pose estimation, comprising:
- receiving image data about an object in an environment;
- receiving depth data about the object;
- generating a visual estimate of the object by processing a combined dataset of the image data and the depth data into a reconstructed fused image;
- receiving tactile data about the object;
- generating, separately, a tactile estimate of the object by processing the tactile data;
- and
- estimating a pose of the object based on the visual estimate and the tactile estimate, wherein
- the visual estimate is generated based on a first convolutional neural network (CNN) to process the combined dataset of the image data and the depth data.
A non-transitory computer readable storage medium storing instructions that when executed by a computer having a processor to perform a method for visuo-tactile object pose estimation, the method comprising:
- receiving image data about an object in an environment;
- receiving depth data about the object;
- generating a visual estimate of the object by processing a combined dataset of the image data and the depth data into a reconstructed fused image;
- receiving tactile data about the object;
- generating, separately, a tactile estimate of the object by processing the tactile data;
- and
- estimating a pose of the object based on the visual estimate and the tactile estimate, wherein
- the visual estimate is generated based on a first convolutional neural network (CNN) to process the combined dataset of the image data and the depth data.
Dependencies are inferred from the claim text.