# US 2021/0279448 A1 · Feature surface projection system and projection method for feature surface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279448 A1
- Document ID: US20210279448A1
- Status: Published application
- Kind: A1
- Application number: 17/191,596
- Filed: 2021-03-03
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: Coretronic Corporation
- Applicants: Coretronic Corporation
- Inventors: Chia-Tien Chen; Wen-Chieh Chung; Hung-Pin Chen
- Web page: https://patentgrep.com/patent/US20210279448A1

## CPC

- G06V 40/174: {Facial expression recognition}
- H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- G06T 7/33: using feature-based methods
- G06T 7/73: using feature-based methods
- G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- G06V 40/168: {Feature extraction; Face representation}
- H04N 9/3176: {wherein the projection device is incorporated in a camera (details of cameras adapted for combination with a projector not peculiar to the presence of an electronic spatial light modulator G03B17/54)}
- H04N 9/3194: {including sensor feedback}
- H04N 23/20: for generating image signals from infrared radiation only
- G06T 2207/30201: Face
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A feature surface projection system includes a projector, a camera, and a controller. The projector projects a structured light and an image beam to a feature surface at different times. The structured light forms a structured light pattern on the feature surface, and the image beam forms an image on the feature surface. The camera photographs the structured light pattern and the image at different times. The controller is electrically connected to the projector and the camera, calculates a position of a feature point of the feature surface, and determines a difference between a position of a feature point of the image and the position of the feature point of the feature surface to determine whether to adjust the image beam. A projection method for a feature surface is also provided. The feature surface projection system provides an automatic alignment function between a projected content and the feature surface.

## Claims

### Claim 1

A feature surface projection system comprising a projector, a camera, and a controller, wherein
the projector is configured to project a structured light and an image beam to a feature surface at different times, wherein the structured light forms a structured light pattern on the feature surface, and the image beam forms an image on the feature surface,
the camera is configured to photograph the structured light pattern and the image at different times, and
the controller is electrically connected to the projector and the camera, is configured to calculate a position of a feature point of the feature surface according to the structured light pattern, and is configured to determine a difference between a position of a feature point of the image and the position of the feature point of the feature surface to determine whether to adjust the image beam.

### Claim 2 (depends on 1)

The feature surface projection system according to claim 1, wherein when the controller determines that the difference between the position of the feature point of the image and the position of the feature point of the feature surface exceeds a predetermined range, the controller adjusts a content of the image through the projector to reduce the difference between the position of the feature point of the image and the position of the feature point of the feature surface, and
when the controller determines that the difference between the position of the feature point of the image and the position of the feature point of the feature surface falls within the predetermined range, the controller does not adjust the content of the image.

### Claim 3 (depends on 1)

The feature surface projection system according to claim 1, wherein when the controller determines that the difference between the position of the feature point of the image and the position of the feature point of the feature surface exceeds a predetermined range, the controller adjusts a relative position or orientation of the projector with respect to the feature surface through an actuator to reduce the difference between the position of the feature point of the image and the position of the feature point of the feature surface, and
when the controller determines that the difference between the position of the feature point of the image and the position of the feature point of the feature surface falls within the predetermined range, the controller does not adjust the relative position or orientation of the projector with respect to the feature surface.

### Claim 4 (depends on 1)

The feature surface projection system according to claim 1, wherein the feature surface is a surface of a human face or a face mask, and the feature points of the feature surface and the image comprise at least one of a nose tip, an eye corner, a mouth corner, and an earlobe.

### Claim 5 (depends on 1)

The feature surface projection system according to claim 1, wherein the feature surface is a surface of a human face or a face mask, and the controller first determines a position of the feature surface by using a central feature point, and then determines an orientation of the feature surface by using a symmetric feature point, wherein the central feature point comprises a nose tip, and the symmetric feature point comprises an eye corner, a mouth corner, or an earlobe.

### Claim 6 (depends on 1)

The feature surface projection system according to claim 1, wherein projection ranges of the structured light and the image beam are larger than a range of the feature surface.

### Claim 7 (depends on 1)

The feature surface projection system according to claim 1, wherein the structured light is infrared light.

### Claim 8 (depends on 1)

The feature surface projection system according to claim 1, wherein the structured light is visible light.

### Claim 9 (depends on 1)

The feature surface projection system according to claim 1, wherein the feature surface is a surface of a human face, and the projector and the camera are provided outside the human face.

### Claim 10 (depends on 9)

The feature surface projection system according to claim 9, wherein the projector repeatedly projects the structured light and the image beam alternately, the camera continuously photographs the structured light pattern and the image alternately, and the controller continuously determines whether to adjust the image beam to keep projecting a correct image on the human face as the human face dynamically moves.

### Claim 11 (depends on 1)

The feature surface projection system according to claim 1, wherein the feature surface is a surface of a face mask, and the projector and the camera are provided inside the face mask.

### Claim 12 (depends on 11)

The feature surface projection system according to claim 11, wherein when the controller determines that the difference between the position of the feature point of the image and the position of the feature point of the feature surface falls within a predetermined range, and the controller does not adjust a relative position or orientation of the projector with respect to the feature surface, the projector stops projecting the structured light but continues to project the image beam, and the camera stops photographing.

### Claim 13

A projection method for a feature surface, adapted for projecting on a feature surface, the projection method comprising:
projecting a structured light and an image beam to the feature surface at different times, wherein the structured light forms a structured light pattern on the feature surface, and the image beam forms an image on the feature surface;
photographing the structured light pattern and the image at different times;
calculating a position of a feature point of the feature surface according to the structured light pattern; and
determining a difference between a position of a feature point of the image and the position of the feature point of the feature surface to determine whether to adjust the image beam.

### Claim 14 (depends on 13)

The projection method for a feature surface according to claim 13, wherein when it is determined that the difference between the position of the feature point of the image and the position of the feature point of the feature surface exceeds a predetermined range, a content of the image is adjusted to reduce the difference between the position of the feature point of the image and the position of the feature point of the feature surface, and
when it is determined that the difference between the position of the feature point of the image and the position of the feature point of the feature surface falls within the predetermined range, the content of the image is not adjusted.

### Claim 15 (depends on 13)

The projection method for a feature surface according to claim 13, wherein when it is determined that the difference between the position of the feature point of the image and the position of the feature point of the feature surface exceeds a predetermined range, a relative position or orientation of a projector, which projects the structured light and the image beam, with respect to the feature surface is adjusted through an actuator to reduce the difference between the position of the feature point of the image and the position of the feature point of the feature surface, and
when it is determined that the difference between the position of the feature point of the image and the position of the feature point of the feature surface falls within the predetermined range, the relative position or orientation of the projector with respect to the feature surface is not adjusted.

### Claim 16 (depends on 13)

The projection method for a feature surface according to claim 13, wherein the feature surface is a surface of a human face or a face mask, and the feature points of the feature surface and the image comprise at least one of a nose tip, an eye corner, a mouth corner, and an earlobe.

### Claim 17 (depends on 13)

The projection method for a feature surface according to claim 13, wherein the feature surface is a surface of a human face or a face mask, and the step of calculating the position of the feature point of the feature surface according to the structured light pattern comprises:
first determining a position of the feature surface by using a central feature point, and then determining an orientation of the feature surface by using a symmetric feature point, wherein the central feature point comprises a nose tip, and the symmetric feature point comprises an eye corner, a mouth corner, or an earlobe.

### Claim 18 (depends on 13)

The projection method for a feature surface according to claim 13, wherein projection ranges of the structured light and the image beam are larger than a range of the feature surface.

### Claim 19 (depends on 13)

The projection method for a feature surface according to claim 13, wherein the feature surface is a surface of a human face, the structured light and the image beam are projected toward an outer surface of the human face, and the camera photographs toward the outer surface of the human face.

### Claim 20 (depends on 13)

The projection method for a feature surface according to claim 13, wherein the feature surface is a surface of a face mask, the structured light and the image beam are projected toward an inner surface of the face mask, and the camera photographs toward the inner surface of the face mask.
