Flagman traffic gesture recognition
AssigneeGM Global Technology Operations LLC
InventorsEran Kishon, Weijing Shi, Ragunathan Rajkumar
Life of this patent
Estimating the term…
Abstract
A vehicle and a system and method of operating the vehicle based on a gesture made by a traffic director. The system includes a camera and at least one neural network. The camera obtains an image of a flag operator. The at least one neural network is to generates an encoded hand vector based on a configuration of a hand of the traffic director from the image, combines a skeleton of the traffic director generated from the image and the encoded hand vector to generate a representation vector, and predicts a gesture of the traffic director from the representation vector.
Claims14
A method of operating a vehicle based on a gesture made by a traffic director, comprising:
- inputting an image of the traffic director into at least one neural network, wherein the traffic director is using a traffic prop;
- generating a skeleton of the traffic direction from the image;
- locating a hand of the traffic director in the image from a ray drawn through an elbow key point of the skeleton and a wrist key point of the skeleton;
- locating a bounding box in the image that indicates the traffic prop;
- generating, at the at least one neural network, an encoded hand vector based on a configuration of the hand of the traffic director from the image;
- combining, at the at least one neural network, the skeleton of the traffic director in the bounding box, and the encoded hand vector to generate a representation vector;
- predicting, at the at least one neural network, the gesture from the representation vector;
- and
- operating the vehicle based on the meaning of the gesture.
A system for operating a vehicle, comprising:
- a camera for obtaining an image of a traffic director, wherein the traffic director is using a traffic prop;
- and
- at least one neural network configured to:
- generate a skeleton of the traffic direction from the image;
- determine a location of a hand of the traffic director in the image from a ray drawn through an elbow key point of the skeleton and a wrist key point of the skeleton;
- locate a bounding box in the image that indicates the traffic prop;
- generate an encoded hand vector based on a configuration of a hand of the traffic director from the image;
- combine a skeleton of the traffic director generated from the image and the encoded hand vector to generate a representation vector;
- and
- predict a gesture of the traffic director from the representation vector.
A vehicle, comprising:
- a camera for obtaining an image of a traffic director, wherein the traffic director is using a traffic prop;
- and
- at least one neural network configured to:
- generate a skeleton of the traffic direction from the image;
- determine a location of a hand of the traffic director in the image from a ray drawn through an elbow key point of the skeleton and a wrist key point of the skeleton;
- locate a bounding box in the image that indicates the traffic prop;
- generate an encoded hand vector based on a configuration of a hand of the traffic director from the image;
- combine a skeleton of the traffic director and the encoded hand vector to generate a representation vector;
- and
- predict a gesture of the traffic director from the representation vector.
Dependencies are inferred from the claim text.