# US 12,602,949 B2 · System and method for detecting human presence based on depth sensing and inertial measurement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,949 B2
- Document ID: US12602949B2
- Status: Granted
- Kind: B2
- Application number: 18/335,046
- Filed: 2023-06-14
- Published: 2023-10-12
- Granted: 2026-04-14
- Earliest claimed: 2020-08-28
- Assignees: STMicroelectronics S.r.l.
- Applicants: Stmicroelectronics, Inc.
- Inventors: Xiaoyong Yang; Kalyan-Kumar Vadlamudi-Reddy
- Web page: https://patentgrep.com/patent/US12602949B2

## CPC

- G01B 21/22: for measuring angles or tapers; for testing the alignment of axes
- G06V 40/166: {using acquisition arrangements}
- G01B 7/30: for measuring angles or tapers; for testing the alignment of axes
- G01B 11/22: for measuring depth
- G01C 19/00: Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects
- G01P 15/18: in two or more dimensions
- G01S 17/08: for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48)
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/25: Fusion techniques
- G06V 10/82: using neural networks
- G06V 20/64: Three-dimensional [3D] objects
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G01B 11/026: {by measuring distance between sensor and object (G01B11/0608 takes precedence)}
- G06F 1/3231: Monitoring the presence, absence or movement of users
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G})
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An electronic device includes a depth sensor and an inertial measurement unit. The electronic device detects a presence of the user of the electronic device by analyzing a combination of inertial sensor signals from the inertial measurement unit and depth sensor signals from the depth sensor.

## Claims

### Claim 1

An electronic device, comprising:
a top lid;
a bottom lid;
a hinge coupled to the top lid and the bottom lid;
a display in the top lid;
a depth sensor in the top lid and including a reflectance sensor and configured to sense a distance between the display and objects in view of the depth sensor and to generate a depth map including a plurality of zones;
a first inertial measurement unit in the top lid and configured to generate first inertial sensor data;
a second inertial measurement unit in the bottom lid and configured to generate second inertial sensor data; and
a sensor processing unit configured to:
receive the first inertial sensor data, the second inertial sensor data, and the depth map;
to determine a heading of the electronic device based on one or both of the first inertial sensor data and the second inertial sensor data;
to determine a rotational angle of the top lid and the bottom lid;
to identify an expected zone from the plurality of zones of the depth map in which a user of the electronic device is expected to be located based on the heading and the rotational angle, wherein identifying an expected zone includes identifying the expected zone based on rotation motion indicated by first and second inertial sensor data;
to detect a presence of the user within the expected zone by analyzing the expected zone; and
tracking a face of the user based on the first and second inertial sensor data indicating rotational motion.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the depth sensor includes a time of flight sensor.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein at least one of the first and second inertial measurement units includes an accelerometer.

### Claim 4 (depends on 1)

The electronic device of claim 1, wherein at least one of the first and second inertial measurement units includes a gyroscope.

### Claim 5 (depends on 1)

The electronic device of claim 1, wherein the sensor processing unit is configured to detect the presence of the user based on whether or not reflectance sensor signals of the reflectance sensor indicate a reflectance less than a threshold reflectance.

### Claim 6

A method, comprising:
generating depth sensor data with a depth sensor in a top lid of an electronic device and including a reflectance sensor;
generating first inertial sensor data with a first inertial sensor in the top lid of the electronic device;
generating second inertial sensor signals with a second inertial sensor in a bottom lid of the electronic device, the bottom lid being rotatably coupled to the top lid by a hinge;
generating a depth map based on the depth sensor signals;
receiving, with a sensor processing unit of the electronic device, the first inertial sensor data, the second inertial sensor data, and the depth map;
determining, with the sensor processing unit, a heading of the electronic device based on one or both of the first inertial sensor data and the second inertial sensor data;
determining, with the sensor processing unit, a rotational angle of the top lid and the bottom lid;
identifying, with the sensor processing unit, an expected zone from a plurality of zones of the depth map in which a user of the electronic device is expected to be located based on the heading and the rotational angle, wherein identifying an expected zone includes identifying the expected zone based on rotation motion indicated by first and second inertial sensor data;
detecting, with the sensor processing unit, a presence of the user within the expected zone by analyzing the expected zone; and
tracking a face of the user based on the first and second inertial sensor data indicating rotational motion.

### Claim 7 (depends on 6)

The method of claim 6, wherein the electronic device is a laptop computer.

### Claim 8 (depends on 6)

The method of claim 6, wherein the sensor processing unit includes a first neural network.

### Claim 9 (depends on 8)

The method of claim 8, wherein the sensor processing unit includes a second neural network.

### Claim 10 (depends on 8)

The method of claim 8, further comprising generating combined sensor data with the first neural network.

### Claim 11

A device, comprising:
an upper lid;
a lower lid coupled to the upper lid;
a depth sensor positioned in the upper lid, the depth sensor configured to generate a depth map including a plurality of zones;
a first inertial measurement unit positioned in the upper lid, the first inertial measurement unit configured to generate a first inertial measurement of the device;
a second inertial measurement unit positioned in the lower lid, the second inertial measurement unit configured to generate a second inertial measurement of the device; and
a sensor processing unit configured to:
receive the first inertial sensor data, the second inertial sensor data, and the depth map;
to determine a heading of the device based on one or both of the first inertial sensor data and the second inertial sensor data;
to determine a rotational angle of the upper lid and the lower lid;
to identify an expected zone from the plurality of zones of the depth map in which a user of the electronic device is expected to be located based on the heading and the rotational angle, wherein identifying an expected zone includes identifying the expected zone based on rotation motion indicated by first and second inertial sensor data;
to detect a presence of the user within the expected zone by analyzing the expected zone; and
track a face of the user based on the first and second inertial sensor data indicating rotational motion.

### Claim 12 (depends on 11)

The device of claim 11, wherein the sensor processing unit is configured to detect the presence of the user based on whether or not reflectance measurement of the depth sensor is less than a threshold reflectance.

### Claim 13 (depends on 11)

The device of claim 11, wherein the sensor processing unit includes a first neural network.

### Claim 14 (depends on 13)

The device of claim 13, wherein the sensor processing unit includes a second neural network.

### Claim 15 (depends on 11)

The device of claim 11, wherein the first inertial measurement unit includes an accelerometer.

### Claim 16 (depends on 11)

The device of claim 11, wherein the first inertial measurement unit includes a gyroscope.