# US 2021/0334569 A1 · Image depth determining method and living body identification method, circuit, device, and medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334569 A1
- Document ID: US20210334569A1
- Status: Published application
- Kind: A1
- Application number: 17/239,302
- Filed: 2021-04-23
- Published: 2021-10-28
- Earliest claimed: 2020-04-28
- Assignees: Nextvpu (Shanghai) Co., Ltd.
- Applicants: Nextvpu (Shanghai) Co., Ltd.
- Inventors: Shu Fang; Ji Zhou; Xinpeng Feng
- Web page: https://patentgrep.com/patent/US20210334569A1

## CPC

- G06T 7/593: from stereo images
- G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F 18/24: Classification techniques
- G06T 7/33: using feature-based methods
- G06T 7/55: from multiple images
- G06T 7/73: using feature-based methods
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/64: Three-dimensional [3D] objects
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/166: {using acquisition arrangements}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/45: {Detection of the body part being alive}
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30201: Face
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image depth determining method and a living body identification method, a circuit, a device, and a medium. The image depth determining method includes obtaining pixel coordinates of a feature point pair associated with an object point on a subject, determining a first straight line passing through the origin of a first camera coordinate system of a first camera based on first pixel coordinates and intrinsic parameters of the first camera, determining a second straight line passing through the origin of a second camera coordinate system of a second camera based on second pixel coordinates and intrinsic parameters of the second camera, and determining the depth of the object point based on the first straight line, the second straight line, and extrinsic parameters describing a relative position relationship between the first camera and the second camera.

## Claims

### Claim 1

An image depth determining method, comprising:
obtaining pixel coordinates of a feature point pair associated with an object point on a subject, wherein the feature point pair comprises a first pixel in a first image corresponding to the object point and a second pixel in a second image corresponding to the object point, wherein the first image is an image captured by a first camera for the subject, and the second image is an image captured by a second camera for the subject, and wherein the pixel coordinates comprise first pixel coordinates of the first pixel in the first image and second pixel coordinates of the second pixel in the second image;
determining a first straight line passing through the origin of a first camera coordinate system of the first camera based on the first pixel coordinates and intrinsic parameters of the first camera, wherein a first coordinate representation of the object point in the first camera coordinate system is located on the first straight line;
determining a second straight line passing through the origin of a second camera coordinate system of the second camera based on the second pixel coordinates and intrinsic parameters of the second camera, wherein a second coordinate representation of the object point in the second camera coordinate system is located on the second straight line; and
determining the depth of the object point based on the first straight line, the second straight line, and extrinsic parameters describing a relative position relationship between the first camera and the second camera.

### Claim 2 (depends on 1)

The method of claim 1, wherein the determining the depth of the object point based on the first straight line, the second straight line, and extrinsic parameters describing a relative position relationship between the first camera and the second camera comprises:
determining a representation of the second straight line in the first camera coordinate system based on the second straight line and the extrinsic parameters; and
determining, in the first camera coordinate system, the depth of the object point based on the first straight line and the second straight line.

### Claim 3 (depends on 2)

The method of claim 2, wherein the determining a representation of the second straight line in the first camera coordinate system based on the second straight line and the extrinsic parameters comprises:
converting, by using the extrinsic parameters, the second coordinate representation to a third coordinate representation in the first camera coordinate system, and determining a fourth coordinate representation of the origin of the second camera coordinate system in the first camera coordinate system; and
determining the representation of the second straight line in the first camera coordinate system based on the third coordinate representation and the fourth coordinate representation.

### Claim 4 (depends on 2)

The method of claim 2, wherein the determining, in the first camera coordinate system, the depth of the object point based on the first straight line and the second straight line comprises:
determining a common perpendicular of the first straight line and the second straight line in response to the first straight line and the second straight line being non-coplanar;
determining coordinates of a first intersection point of the first straight line and the common perpendicular and coordinates of a second intersection point of the second straight line and the common perpendicular; and
determining the depth of the object point based on a first depth component of the coordinates of the first intersection point and a second depth component of the coordinates of the second intersection point.

### Claim 5 (depends on 4)

The method of claim 4, wherein the determining coordinates of a first intersection point of the first straight line and the common perpendicular and coordinates of a second intersection point of the second straight line and the common perpendicular comprises:
determining the coordinates of the first intersection point and the coordinates of the second intersection point according to the following spatial relationships:
a direction vector of the first straight line is parallel to a direction vector pointing from the first intersection point to the origin of the first camera coordinate system;
a direction vector of the second straight line is parallel to a direction vector pointing from the second intersection point to the origin of the second camera coordinate system; and
a direction vector of the common perpendicular is parallel to a direction vector pointing from the second intersection point to the first intersection point.

### Claim 6 (depends on 4)

The method of claim 4, wherein the determining the depth of the object point based on a first depth component of the coordinates of the first intersection point and a second depth component of the coordinates of the second intersection point comprises:
determining a predefined function value of the first depth component and the second depth component as the depth of the object point.

### Claim 7 (depends on 6)

The method of claim 6, wherein the predefined function value comprises a weighted average.

### Claim 8 (depends on 7)

The method of claim 7, wherein the weighted average comprises an arithmetic average.

### Claim 9 (depends on 2)

The method of claim 2, wherein the determining, in the first camera coordinate system, the depth of the object point based on the first straight line and the second straight line comprises:
determining a depth component of coordinates of an intersection point of the first straight line and the second straight line as the depth of the object point, in response to the first straight line and the second straight line being coplanar.

### Claim 10 (depends on 1)

The method of claim 1, wherein the first camera and the second camera are cameras of different models.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
performing undistortion on the first pixel coordinates before the determining a first straight line passing through the origin of a first camera coordinate system of the first camera based on the first pixel coordinates and intrinsic parameters of the first camera; and
performing undistortion on the second pixel coordinates before the determining a second straight line passing through the origin of a second camera coordinate system of the second camera based on the second pixel coordinates and intrinsic parameters of the second camera.

### Claim 12

A chip circuit, comprising:
circuit units configured to perform operations comprising:
obtaining pixel coordinates of a feature point pair associated with an object point on a subject, wherein the feature point pair comprises a first pixel in a first image corresponding to the object point and a second pixel in a second image corresponding to the object point, wherein the first image is an image captured by a first camera for the subject, and the second image is an image captured by a second camera for the subject, and wherein the pixel coordinates comprise first pixel coordinates of the first pixel in the first image and second pixel coordinates of the second pixel in the second image;
determining a first straight line passing through the origin of a first camera coordinate system of the first camera based on the first pixel coordinates and intrinsic parameters of the first camera, wherein a first coordinate representation of the object point in the first camera coordinate system is located on the first straight line;
determining a second straight line passing through the origin of a second camera coordinate system of the second camera based on the second pixel coordinates and intrinsic parameters of the second camera, wherein a second coordinate representation of the object point in the second camera coordinate system is located on the second straight line; and
determining the depth of the object point based on the first straight line, the second straight line, and extrinsic parameters describing a relative position relationship between the first camera and the second camera.

### Claim 13

A computing device, comprising:
one or more processors; and
a memory, which stores a program comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
obtaining pixel coordinates of a feature point pair associated with an object point on a subject, wherein the feature point pair comprises a first pixel in a first image corresponding to the object point and a second pixel in a second image corresponding to the object point, wherein the first image is an image captured by a first camera for the subject, and the second image is an image captured by a second camera for the subject, and wherein the pixel coordinates comprise first pixel coordinates of the first pixel in the first image and second pixel coordinates of the second pixel in the second image;
determining a first straight line passing through the origin of a first camera coordinate system of the first camera based on the first pixel coordinates and intrinsic parameters of the first camera, wherein a first coordinate representation of the object point in the first camera coordinate system is located on the first straight line;
determining a second straight line passing through the origin of a second camera coordinate system of the second camera based on the second pixel coordinates and intrinsic parameters of the second camera, wherein a second coordinate representation of the object point in the second camera coordinate system is located on the second straight line; and
determining the depth of the object point based on the first straight line, the second straight line, and extrinsic parameters describing a relative position relationship between the first camera and the second camera.

### Claim 14 (depends on 1)

A living body identification method, comprising:
determining, for each of a plurality of feature point pairs associated with a plurality of object points on a subject, respective depths of the plurality of object points by performing the image depth determining method according to claim 1, wherein a first pixel of each feature point pair is a facial key-point in the first image, and a second pixel of each feature point pair is a facial key-point in the second image; and
identifying whether the subject is a living body, based on the respective depths of the plurality of object points.

### Claim 15 (depends on 14)

The living body identification method of claim 14, wherein the identifying whether the subject is a living body, based on the respective depths of the plurality of object points comprises: identifying whether the subject is a living body, according to a relationship between a function value of the respective depths of the plurality of object points and a predetermined threshold.

### Claim 16 (depends on 14)

The living body identification method of claim 14, wherein coordinate components of the first coordinate representation of each object point are a function of the depth of the object point, and the identifying whether the subject is a living body, based on the respective depths of the plurality of object points comprises:
substituting the depth of each object point into the first coordinate representation of the object point, to obtain spatial coordinates of the object point;
determining alignment parameters for aligning the respective spatial coordinates of the plurality of object points with the respective key-point spatial coordinates of the standard face;
aligning, by using the determined alignment parameters, the respective spatial coordinates with the respective key-point spatial coordinates of the standard face, to obtain corresponding aligned spatial coordinates;
constructing a depth feature vector by using depth components of the corresponding aligned spatial coordinates; and
inputting the depth feature vector to a trained support vector machine classifier, to obtain an identification result.

### Claim 17 (depends on 16)

The living body identification method of claim 16, wherein the alignment parameters comprise a scaling scale, and wherein the living body identification method further comprises:
determining, in response to the identification result indicating that the subject is a living body, whether the scaling scale falls within a predetermined range;
identifying the subject as a living body in response to determining that the scaling scale is within the predetermined range; and
identifying the subject as a non-living body in response to determining that the scaling scale is not within the predetermined range.

### Claim 18 (depends on 14)

The living body identification method of claim 14, wherein coordinate components of the first coordinate representation of each object point are a function of the depth of the object point, and the identifying whether the subject is a living body, based on the respective depths of the plurality of object points comprises:
substituting the depth of each object point into the first coordinate representation of the object point, to obtain spatial coordinates of the object point;
determining alignment parameters for aligning the respective spatial coordinates of the plurality of object points with the respective key-point spatial coordinates of the standard face, wherein the alignment parameters comprise a scaling scale;
identifying the subject as a living body in response to the scaling scale being within the predetermined range; and
identifying the subject as a non-living body in response to the scaling scale being not within the predetermined range.

### Claim 19 (depends on 14)

The living body identification method of claim 14, wherein coordinate components of the first coordinate representation of each object point are a function of the depth of the object point, and the identifying whether the subject is a living body, based on the respective depths of the plurality of object points comprises:
substituting the depth of each object point into the first coordinate representation of the object point, to obtain spatial coordinates of the object point;
determining alignment parameters for aligning the respective spatial coordinates of the plurality of object points with the respective key-point spatial coordinates of the standard face;
aligning, by using the determined alignment parameters, the respective spatial coordinates with the respective key-point spatial coordinates of the standard face, to obtain corresponding aligned spatial coordinates;
constructing an input vector by using the corresponding aligned spatial coordinates; and
inputting the input vector to a trained artificial neural network, to obtain an identification result.

### Claim 20 (depends on 19)

The living body identification method of claim 19, wherein the alignment parameters comprise a scaling scale, and wherein the living body identification method further comprises:
determining, in response to the identification result indicating that the subject is a living body, whether the scaling scale is within a predetermined range;
identifying the subject as a living body in response to determining that the scaling scale is within the predetermined range; and
identifying the subject as a non-living body in response to determining that the scaling scale is not within the predetermined range.
