# US 2024/0199062 A1 · Steering wheel hands-on/off detection device and method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0199062 A1
- Document ID: US20240199062A1
- Status: Published application
- Kind: A1
- Application number: 18/376,709
- Filed: 2023-10-04
- Published: 2024-06-20
- Earliest claimed: 2022-12-16
- Assignees: Hyundai Motor Company; Kia Corporation
- Applicants: Hyundai Motor Company; Kia Corporation
- Inventors: Joon-Sang Park
- Web page: https://patentgrep.com/patent/US20240199062A1

## CPC

- B60W 50/16: Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
- B60W 40/08: related to drivers or passengers
- B60W 2040/0818: {Inactivity or incapacity of driver}
- B60W 2050/0029: {Mathematical model of the driver}
- B60W 2540/18: Steering angle
- B60W 2540/223: Posture, e.g. hand, foot, or seat position, turned or inclined
- B60W 2540/227: Position in the vehicle
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

A steering wheel hands-on/off detection device includes: an imaging part provided in a vehicle and configured to capture a motion of a driver; a dynamic sensor configured to measure a dynamics motion of the vehicle; an image processor configured to process an image obtained from the imaging part; and a vehicle safety controller. The image processor includes a driver monitoring part configured to derive a behavior type of the driver from the image and a keypoint extraction part configured to extract keypoints of the driver. An optimal torque and an optimal contact force are applied to the keypoints of the driver so that a driver dynamics model optimally controlled to follow a position of the driver is applied. The vehicle safety controller configured to determine steering wheel hands-on/off on the basis of a magnitude of a force applied to the steering wheel, which is estimated from the driver dynamics model.

## Claims

### Claim 1

A steering wheel hands-on/off detection device, comprising:
an imaging part provided in a vehicle and configured to capture a motion of a driver;
a dynamic sensor configured to measure a dynamic motion of the vehicle; and
an image processor configured to process an image obtained from the imaging part, wherein
the image processor includes a keypoint extraction part configured to extract keypoints of the driver, and
an optimal torque and an optimal contact force are applied to the keypoints of the driver so that a driver dynamics model following a position of the driver is applied; and
a vehicle safety controller configured to determine steering wheel hands-on/off based on the driver dynamics model and a magnitude of a force applied to the steering wheel, which is estimated from the dynamic sensor.

### Claim 2 (depends on 1)

The steering wheel hands-on/off detection device of claim 1, further comprising:
a seat sensor assembly configured to measure information on at least one seat among a seat position in the vehicle, a seat incline angle, and a seat swivel angle,
wherein the seat information is added to the driver dynamics model of the vehicle safety controller.

### Claim 3 (depends on 1)

The steering wheel hands-on/off detection device of claim 1, wherein a cost function in a convex quadratic form, which receives a joint torque and a contact force as inputs and receives the optimal torque and the optimal contact force, which are applied to the keypoints of the driver, is used for optimal control of the driver dynamics model of the vehicle safety controller.

### Claim 4 (depends on 1)

The steering wheel hands-on/off detection device of claim 1, wherein:
the image processor further includes a driver monitoring part configured to derive a behavior type of the driver from the image; and
when the behavior type of the driver is a behavior type in which one hand of the driver corresponds to steering wheel hands-off, the driver monitoring part updates the driver dynamics model of the vehicle safety controller as another hand of the driver comes into contact with the steering wheel.

### Claim 5 (depends on 4)

The steering wheel hands-on/off detection device of claim 4, wherein, when the force applied to the steering wheel estimated from the driver dynamics model is greater than or equal to a threshold, it is determined that the driver is holding the steering wheel.

### Claim 6

A steering wheel hands-on/off detection device, comprising:
an imaging part provided in a vehicle and configured to capture a motion of a driver;
a dynamic sensor configured to measure a dynamics motion of the vehicle; and
an image processor configured to process an image obtained from the imaging part, wherein the image processor includes a driver monitoring part configured to derive a behavior type of the driver from the image, and a keypoint extraction part configured to extract keypoints of the driver; and
a vehicle safety controller configured to determine steering wheel hands-on/off from a driver dynamics model which is set from a behavior type and the key points of the driver.

### Claim 7

A steering wheel hands-on/off detection device, comprising:
an imaging part provided in a vehicle and configured to capture a motion of a driver;
a dynamic sensor configured to measure a dynamic motion of the vehicle;
an image processor configured to process an image obtained from the imaging part, wherein the image processor includes a driver monitoring part configured to derive a behavior type of the driver from the image, and a keypoint extraction part configured to extract keypoints of the driver; and
a vehicle safety controller configured to determine steering wheel hands-on/off by predicting a motion of the driver through optimal control to allow the driver dynamics model, which is set from the behavior type and the keypoints of the driver, to follow a position of the driver.

### Claim 8

A method of detecting steering wheel hands-on/off, the method comprising:
extracting, by an image processor, a behavior type and keypoint coordinates of a driver from an image acquired from an imaging part in a vehicle;
determining whether the driver has a behavior type capable of holding a steering wheel;
determining whether a hand of the driver is within a predetermined distance from the steering wheel;
adding a steering wheel contact condition to a constraint condition of a driver dynamics model of a vehicle safety controller;
estimating a force applied to the steering wheel from the driver dynamics model through optimal control; and
determining steering wheel hands-on/off by predicting a motion of the driver through the optimal control to allow the driver dynamics model, which is set from the behavior type and the keypoints of the driver, to follow a position of the driver.

### Claim 9 (depends on 8)

The method of claim 8, wherein, in determining steering wheel hands-on/off, when the force applied to the steering wheel is greater than or equal to a threshold for a specific time interval, the driver is determined as holding the steering wheel.

### Claim 10 (depends on 8)

The method of claim 8, further comprising:
before estimating the force applied to the steering wheel through the optimal control, adding a seat constraint condition from a seat sensor assembly.

### Claim 11 (depends on 10)

The method of claim 10, wherein one or more among a seat position, a seat incline angle, or a seat swivel angle from the seat sensor assembly are added to a constraint condition of the driver dynamics model.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
when the behavior type of the driver is determined not to be a behavior type in which the driver is capable of holding the steering wheel, adding one or more among a seat position, a seat incline angle, or a seat swivel angle from the seat sensor assembly to a constraint condition of the driver dynamics model without adding the steering wheel contact condition of the driver.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
when the hand of the driver is not within a predetermined distance from the steering wheel, adding one or more among a seat position, a seat incline angle, or a seat swivel angle from the seat sensor assembly to a constraint condition of the driver dynamics model without adding the steering wheel contact condition of the driver.
