Self-driving mobile robots using human-robot interactions
AssigneeFord Global Technologies, LLC
InventorsJustin Miller, Sanghyun Hong, Jianbo Lu
Life of this patent
Estimating the term…
Abstract
Systems, methods, and computer-readable media are disclosed for enhanced human-robot interactions. A device such as a robot may send one or more pulses. The device may identify one or more reflections associated with the one or more pulses. The device may determine, based at least in part on the one or more reflections, a cluster. The device may associate the cluster with an object identified in an image. The device may determine, based at least in part on an image analysis of the image, a gesture associated with the object. The device may determine, based at least in part on the gesture, a command associated with an action. The device may to perform the action.
Claims20
A device comprising storage and processing circuitry configured to:
- determine, based at least in part on one or more reflections associated with one or more light detection and ranging (LIDAR) pulses captured by the device, a LIDAR cluster, the LIDAR cluster comprising LIDAR data associated with the one or more reflections;
- identify an object represented by a 2D image captured by the device, the 2D image comprising image data different than the LIDAR data;
- determine that the LIDAR cluster is associated with the object represented by the 2D image;
- determine a gesture associated with the object represented by the 2D image;
- determine, based at least in part on the gesture, a command associated with an action;
- convert the image data associated with the 2D image into a 3D coordinate frame of the device using the LIDAR data;
- and
- cause the device to perform the action using the 3D coordinate frame of the device.
A non-transitory computer-readable medium storing computer-executable instructions which when executed by one or more processors result in performing operations comprising:
- determining, based at least in part on one or more reflections associated with one or more light detection and ranging (LIDAR) pulses captured by a device, a LIDAR cluster, the LIDAR cluster comprising LIDAR data associated with the one or more reflections;
- identifying an object represented by a 2D image captured by the device, the 2D image comprising image data different than the LIDAR data;
- determining that the LIDAR cluster is associated with the object represented by the 2D image;
- determining a gesture associated with the object represented by the 2D image;
- determining, based at least in part on the gesture, a command associated with an action;
- converting the image data associated with the 2D image into a 3D coordinate frame of the device using the LIDAR data;
- and
- causing the device to perform the action using the 3D coordinate frame of the device.
A method, comprising:
- determining, by processing circuitry of a device, based at least in part on one or more reflections associated with one or more light detection and ranging (LIDAR) pulses captured by the device, a LIDAR cluster, the LIDAR cluster comprising LIDAR data associated with the one or more reflections;
- identifying, by the processing circuitry, an object represented by a 2D image captured by the device, the 2D image comprising image data different than the LIDAR data;
- determining, by the processing circuitry, that the LIDAR cluster is associated with the object represented by the 2D image;
- determining, by the processing circuitry, a gesture associated with the object represented by the 2D image;
- determining, by the processing circuitry, based at least in part on the gesture, a command associated with an action;
- converting, by the processing circuitry, the image data associated with the 2D image into a 3D coordinate frame of the device using the LIDAR data;
- and
- causing, by the processing circuitry, the device to perform the action using the 3D coordinate frame of the device.
Dependencies are inferred from the claim text.