# US 2021/0341620 A1 · SYSTEMS, DEVICES AND METHODS FOR MICRO-VIBRATION DATA EXTRACTION USING A TIME OF FLIGHT (ToF) IMAGING DEVICE

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0341620 A1
- Document ID: US20210341620A1
- Status: Published application
- Kind: A1
- Application number: 17/298,996
- Filed: 2019-12-01
- Published: 2021-11-04
- Earliest claimed: 2018-12-02
- Assignees: Gentex Corporation
- Applicants: Gentex Corporation
- Inventors: Guy RAZ; Yuval GRONAU
- Web page: https://patentgrep.com/patent/US20210341620A1

## CPC

- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- B60Q 3/20: for lighting specific fittings of passenger or driving compartments; mounted on specific fittings of passenger or driving compartments (lighting for dashboards B60Q3/10)
- G01B 11/22: for measuring depth
- G01H 9/00: Measuring mechanical vibrations or ultrasonic, sonic or infrasonic waves by using radiation-sensitive means, e.g. optical means
- G01H 9/002: {for representing acoustic field distribution (sonar systems for imaging G01S7/56, G01S15/89; acoustic holography G03H3/00)}
- G01S 7/4814: {of transmitters alone}
- G01S 7/4816: {of receivers alone}
- G01S 7/4915: Time delay measurement, e.g. operational details for pixel components (signal extraction and conditioning G01S7/493); Phase measurement
- G01S 17/36: with phase comparison between the received signal and the contemporaneously transmitted signal
- G01S 17/88: Lidar systems specially adapted for specific applications
- B60Q 2400/20: Multi-color single source or LED matrix, e.g. yellow blinker and red brake lamp generated by single lamp
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- G01H: MEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES

## Abstract

System and methods are provided for detecting vibrations of one or more object in a scene comprising at least one illumination source configured to project light in a structured light pattern on the scene; a Time-of-Flight (ToF) imaging device comprising: an illumination source configured to project modulated light into the scene, a ToF sensor, configured to capture a plurality of images comprising reflections of the modulated light, the structured light pattern from the one or more objects in the scene and ambient light images of one or more objects in the scene; and at least one processor configured to: extract depth data of said one or more objects by analyzing the plurality of images and analyze one or more changes in one or more speckle patterns of at least one of the reflections of said structured light pattern in at least some consecutive images of the plurality of images; and identify the vibrations of the at least one object based on said speckle pattern analysis and said depth data.

## Claims

### Claim at position 1

-22. (canceled)

### Claim 23

A system for detecting vibrations, of one or more objects in a scene, the system comprising:
at least one illumination source configured to project light in a structured light pattern on the scene;
a Time-of-Flight (ToF) imaging device comprising:
an illumination source configured to project modulated light into the scene;
a ToF sensor, said ToF sensor is configured to:
capture a plurality of images, said plurality of images comprising reflections of said modulated light and said structured light pattern from the one or more objects in the scene, and ambient light images of one or more objects in the scene; and
at least one processor configured to:
extract depth data of said one or more objects by analyzing the plurality of images;
analyze one or more changes in one or more speckle patterns of at least one of the reflections of said structured light pattern in at least some consecutive images of the plurality of images; and
identify the vibrations of the one or more objects based on said speckle pattern analysis and said depth data.

### Claim 24 (depends on 23)

The system of claim 23, wherein said depth data is extracted by measuring the phase difference between the projected and reflected modulated light and further analyze the plurality of images to extract said ambient illumination images of said scene and wherein said ambient illumination images are used to analyze one or more changes in the speckle pattern of at least one of the plurality of reflected diffused light elements in at least some consecutive images of the plurality of images.

### Claim 25 (depends on 24)

The system of claim 24 wherein said depth data, said ambient light images and said detected vibrations are obtained and detected simultaneously.

### Claim 26 (depends on 23)

The system of claim 23 wherein said at least one illumination source is a single mode laser source, comprising a beam splitter for projecting a multi-beam pattern onto the scene.

### Claim 27 (depends on 26)

The system of claim 26 wherein said beam splitter comprises one or more of optical elements for splitting said beam, said optical elements are selected from the group comprising of: DOE, split mirrors, diffuser.

### Claim 28 (depends on 23)

The system of claim 23 comprising a single illumination source configured to project the modulated light in the structured light pattern.

### Claim 29 (depends on 28)

The system of claim 28 wherein the at least one processor is configured to:
measure the phase and amplitude of the reflected modulated structured light pattern to yield an illumination intensity of said one or more speckle patterns; analyze the phase to extract depth data of the scene; and
analyze, on said measured illumination intensity, the one or more changes in the speckle pattern of at least one of the plurality of reflected diffused light elements in at least some consecutive images of the plurality of images;
identify at least one micro-vibration of the one or more objects based on said speckle pattern analysis and said measured illumination intensity.

### Claim 30 (depends on 23)

The system of claim 23 wherein the processor is further configured to: classify or identify attributes of the one or more objects based on said identified at least one micro-vibration.

### Claim 31 (depends on 23)

The system of claim 23 wherein the processor is further configured to: classify the least one or more objects present in the scene by visually analyzing at least one image of the plurality of images.

### Claim 32 (depends on 31)

The system of claim 31 wherein the visual analysis comprises using at least one trained machine learning model for classifying the one or more objects.

### Claim 33 (depends on 23)

The system of claim 23, wherein the processor is being capable of identifying a human based on said vibrations.

### Claim 34 (depends on 23)

The system of claim 23, wherein the predefined light pattern is selected from a group consisting of: a dot, a line, and a combination thereof.

### Claim 35 (depends on 23)

The system of claim 23, wherein said ToF sensor includes a photodiode or photodiode array and each speckles field is mapped to 1-100 pixels of said ToF sensor.

### Claim 36 (depends on 23)

The system of claim 23, configured for mounting inside a vehicle.

### Claim 37

A system for detecting vibrations of one or more objects in a scene, the system comprising:
a single illumination source configured to project a modulated and structured light pattern on the scene;
a Time-of-Flight (ToF) imaging device comprising:
a ToF sensor, said ToF sensor is configured to:
capture a plurality of images, said plurality of images comprising reflections of said modulated and structured light pattern from one or more objects in the scene; and
at least one processor configured to:
extract depth data of said one or more objects by analyzing the plurality of images;
analyze one or more changes in one or more speckle patterns of at least one of the reflections of said structured light pattern in at least some consecutive images of the plurality of images; and
identify the vibrations of the at least one object based on said speckle pattern analysis and said depth data.

### Claim 38 (depends on 37)

The system of claim 37, wherein said depth data is extracted by measuring the phase difference between the projected and further analyze the plurality of images to extract an ambient illumination image of said scene and wherein said ambient illumination image is used to analyze one or more changes in the speckle pattern of at least one of the plurality of reflected diffused light elements in at least some consecutive images of the plurality of images.

### Claim 39 (depends on 38)

The system of claim 38 wherein the at least one processor is configured to:
measure the phase and amplitude of the reflected modulated structured light pattern to yield an illumination intensity of said one or more speckle patterns;
analyze the phase to extract depth information data of the scene; and
analyze, on said measured illumination intensity, the one or more changes in the speckle pattern of at least one of the plurality of reflected diffused light elements in at least some consecutive images of the plurality of images;
identify at least one micro-vibration of the one or more objects based on said speckle pattern analysis and said measured illumination intensity.

### Claim 40 (depends on 39)

The system of claim 39, wherein the processor is further configured to: classify or identify attributes of the one or more objects based on said identified at least one micro-vibration.

### Claim 41 (depends on 39)

The system of claim 39, wherein the processor is further configured to: classify the least one or more objects present in the scene by visually analyzing at least one image of the plurality of images.

### Claim 42 (depends on 41)

The system of claim 41, wherein the visual analysis comprises using at least one trained machine learning model for classifying the one or more at least one objects.
