Back to results
US 11,079,767 B2Granted
Lidar based recognition of ride hailing gestures for autonomous vehicles
AssigneeGM Cruise Holdings LLC
InventorRajaditya Mukherjee
Life of this patent
Estimating the term…
Abstract
An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a gesture recognition component to determine, based upon lidar sensor data, whether a pedestrian in a driving environment of the AV is performing a hailing gesture. The AV can be configured to initiate a pickup maneuver to approach the pedestrian in response to determining that the pedestrian is performing a hailing gesture.
Claims20
An autonomous vehicle (AV) comprising:
- a lidar sensor;
- and
- a computing system that is in communication with the lidar sensor, wherein the computing system comprises:
- a processor;
- and
- memory that stores instructions that, when executed by the processor, cause the processor to perform acts comprising:
- receiving lidar data from the lidar sensor, the lidar data being data generated by the lidar sensor responsive to performing a lidar scan of a driving environment of the AV, the lidar data being indicative of a point cloud that includes a first plurality of points, each of the first plurality of points of the point cloud comprising data indicative of a coordinate triple corresponding to a location of an object in the driving environment;
- determining that a second plurality of points is representative of a pedestrian in the driving environment, the second plurality of points being a subset of the first plurality of points;
- identifying, based upon the second plurality of points, a skeletal model for the pedestrian in the driving environment;
- generating a pose similarity score based upon the skeletal model;
- determining that the pedestrian is making a hailing signal based upon the pose similarity score;
- and
- controlling operation of the AV in the driving environment based upon determining that the pedestrian is making the hailing signal.
10 more elements
A method for controlling an autonomous vehicle (AV), the method comprising:
- receiving a lidar point cloud from a lidar sensor mounted on the AV, the point cloud generated by the lidar sensor responsive to performing a lidar scan of a driving environment of the AV, the point cloud comprising a first plurality of points, each of the points of the point cloud being assigned data indicative of a coordinate triple corresponding to a location of an object in the driving environment;
- identifying a second plurality of points in the point cloud that are representative of a pedestrian;
- generating a skeletal model that is representative of a pose of the pedestrian based upon the second plurality of points;
- computing a pose similarity score based upon the skeletal model;
- determining that the pedestrian is performing a hailing gesture based upon the pose similarity score;
- and
- controlling operation of the AV in the driving environment based upon determining that the pedestrian is making the hailing signal.
4 more elements
A computing system comprising:
- a processor;
- and
- memory that stores instructions that, when executed by the processor, cause the processor to perform acts comprising:
- computing a skeletal model for a pedestrian in a driving environment of an autonomous vehicle (AV) based upon a lidar point cloud generated by a lidar sensor included on the AV, the pedestrian represented in the lidar point cloud;
- providing the skeletal model as input to a convolutional neural network;
- outputting, by way of the convolutional neural network, a probability that the pedestrian is making the hailing signal;
- determining that the pedestrian is making the hailing signal based upon the probability output by the convolutional neural network;
- and
- causing the AV to initiate a pickup maneuver to approach the pedestrian responsive to determining that the pedestrian is making the hailing signal.
6 more elements
Dependencies are inferred from the claim text.