# US 2020/0394393 A1 · Gesture Control for Communication with an Autonomous Vehicle on the Basis of a Simple 2D Camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394393 A1
- Document ID: US20200394393A1
- Status: Published application
- Kind: A1
- Application number: 16/641,828
- Filed: 2018-09-10
- Published: 2020-12-17
- Earliest claimed: 2017-09-11
- Assignees: Conti Temic microelectronic GmbH
- Applicants: Conti Temic Microelectronic GmbH
- Inventors: Erwin KRAFT; Nicolai HARICH; Sascha SEMMLER; Pia DREISEITEL
- Web page: https://patentgrep.com/patent/US20200394393A1

## CPC

- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06F 18/23: Clustering techniques
- G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/113: {Recognition of static hand signs}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The invention relates to a method and a device for recognizing gestures by means of a monocular camera and can be used in particular in vehicle cameras of a partly autonomously driven vehicle. The method for recognizing gestures of a person from at least one image from a monocular camera comprises the steps: a) detecting key points of a person in the at least one image from the monocular camera, b) connecting key points to a skeleton-like representation of parts of the person, wherein the skeleton-like representation reflects the relative position and orientation of individual body parts of the person, c) recognizing a gesture from the skeleton-like representation of the person, and d) outputting the gesture recognized.

## Claims

### Claim 1

A method for recognizing gestures of a person from at least one image from a monocular camera, comprising the steps:
a) detecting key points of a person in the at least one image from the monocular camera,
b) connecting key points to a skeleton-like representation of parts of the person, wherein the skeleton-like representation reflects the relative position and orientation of individual body parts of the person,
c) recognizing a gesture from the skeleton-like representation of the person, and
d) outputting the gesture recognized.

### Claim 2 (depends on 1)

The method according to claim 1, wherein groups are formed from one or more related body parts.

### Claim 3 (depends on 2)

The method according to claim 2, wherein a body part belongs to multiple groups.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the number N of the groups is adjusted.

### Claim 5 (depends on 2)

The method according to claim 2, wherein a feature vector is assigned to each group, said feature vector being based on coordinates of the key points which are combined in the respective group.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the feature vectors of the groups of a person are merged with the aid of a clustered pose directory to produce a final feature vector.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the gesture is recognized on the basis of a classification of the final feature vector.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the viewing direction of the person is estimated based on the skeleton-like representation.

### Claim 9 (depends on 8)

The method according to claim 8, wherein it is checked whether the viewing direction of the person is directed towards the monocular camera.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the gesture is recognized by a gesture classification which has previously been trained.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the number of key points which are assigned to a person is a maximum of 20.

### Claim 12 (depends on 8)

The method according to claim 8, wherein a person who has lowered his/her head and who is looking at his/her hand is classified as a distracted road user.

### Claim 13

A device for recognizing gestures of a person from images from a monocular camera, wherein the device is configured
a) to detect key points of a person in the at least one image from the monocular camera,
b) to connect key points to a skeleton-like representation of parts of the person, wherein the skeleton-like representation reflects the relative position and orientation of individual body parts of the person,
c) to recognize a gesture from the skeleton-like representation of the person, and
d) to produce and output an output signal representing the recognized gesture.

### Claim 14 (depends on 13)

A vehicle having a monocular camera and a device according to claim 13.
