Training methods for deep networks
AssigneeToyota Research Institute, Inc.
InventorsKevin Stone, Krishna Shankar, Michael Laskey
Life of this patent
Estimating the term…
Abstract
A method for training a deep neural network of a robotic device is described. The method includes constructing a 3D model using images captured via a 3D camera of the robotic device in a training environment. The method also includes generating pairs of 3D images from the 3D model by artificially adjusting parameters of the training environment to form manipulated images using the deep neural network. The method further includes processing the pairs of 3D images to form a reference image including embedded descriptors of common objects between the pairs of 3D images. The method also includes using the reference image from training of the neural network to determine correlations to identify detected objects in future images.
Claims18
A method for training a deep neural network of a robotic device, comprising:
- constructing a 3D model using images captured via a 3D camera of the robotic device in a training environment;
- generating pairs of 3D images from the 3D model by artificially adjusting parameters of the training environment to form manipulated images using the deep neural network;
- processing the pairs of 3D images to form a reference image including embedded descriptors of common objects between the pairs of 3D images;
- and
- using the reference image from training of the neural network to determine correlations to identify detected objects in future images by:
- overlaying the corresponding reference image over a captured image of a scene, and
- determining an identity of the detected object based on a point correspondence between the corresponding reference image and the captured image and the embedded descriptors of the corresponding reference image, in which the embedded descriptors encode information into a series of numbers to provide a numerical fingerprint to differentiate one feature from another.
A method for controlling a robotic device based on identification of detected objects in an unknown environment, comprising:
- detecting an object in the unknown environment;
- selecting a corresponding reference image including embedded descriptors corresponding to trained objects manipulated according to artificially adjusting parameters of an image capture environment;
- and
- identifying the detected object according to the embedded descriptors of the corresponding reference image by:
- overlaying the corresponding reference image over a captured image of a scene, and
- determining an identity of the detected object based on a point correspondence between the corresponding reference image and the captured image and the embedded descriptors of the corresponding reference image, in which the embedded descriptors encode information into a series of numbers to provide a numerical fingerprint to differentiate one feature from another.
A non-transitory computer-readable medium having program code recorded thereon for training a deep neural network of a robotic device, the program code being executed by a processor and comprising:
- program code to generate pairs of 3D images from the 3D model by artificially adjusting parameters of a training environment to form manipulated images using the deep neural network;
- program code to process the pairs of 3D images to form a reference image including embedded descriptors of common objects between the pairs of 3D images;
- program code to use the reference image from training of the neural network to determine correlations to identify detected objects in future images by:
- program code to overlay the corresponding reference image over a captured image of a scene, and
- program code to determine an identity of a detected object based on a point correspondence between the corresponding reference image and the captured image and the embedded descriptors of the corresponding reference image, in which the embedded descriptors encode information into a series of numbers to provide a numerical fingerprint to differentiate one feature from another.
A system for controlling a robotic device based on identification of detected objects in an unknown environment, the system comprising:
- a pre-trained, object identification module to select a corresponding reference image to identify a detected object in a captured image, the corresponding reference image including embedded descriptors based on trained objects manipulated according to artificially adjusted parameters of an image capture environment, the pre-trained, object identification module to overlay the corresponding reference image over the captured image of a scene, and to determine an identity of the detected object based on a point correspondence between the corresponding reference image and the captured image and the embedded descriptors of the corresponding reference image, in which the embedded descriptors encode information into a series of numbers to provide a numerical fingerprint to differentiate one feature from another;
- and
- a controller to select an autonomous behavior of the robotic device based on an identity of the detected object.
Dependencies are inferred from the claim text.