# US 2019/0171866 A1 · Apparatus and method for data processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171866 A1
- Document ID: US20190171866A1
- Status: Published application
- Kind: A1
- Application number: 16/210,158
- Filed: 2018-12-05
- Published: 2019-06-06
- Earliest claimed: 2017-12-05
- Assignees: Fujitsu Limited
- Applicants: Fujitsu Limited
- Inventors: Pingping LU; Rujie Liu
- Web page: https://patentgrep.com/patent/US20190171866A1

## CPC

- G06T 15/503: {Blending, e.g. for anti-aliasing}
- G06F 18/253: {of extracted features}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06T 3/02: Affine transformations (for image registration G06T3/147; for image mosaicing G06T3/4038)
- G06T 7/60: Analysis of geometric attributes
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06N 3/045: Combinations of networks
- G06T 2207/20081: Training; Learning
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An apparatus and a method for data processing are provided. The apparatus for data processing includes a modeler configured to build an occlusion object model for an image containing an occlusion object; a renderer configured to render the occlusion object model according to a geometric relationship between the occlusion object and a face image containing no occlusion object, such that the rendered occlusion object image and the face image containing no occlusion object have same scale and attitude; and a merger configured to merge the face image containing no occlusion object and the rendered occlusion object image into an occluded face image. With the data processing apparatus and the data processing method face data enhancement, face data in the case of having an occlusion object is generated, so that the number of face training data sets can be effectively increased, thereby improving performance of a face-related module.

## Claims

### Claim 1

An apparatus for data processing, including:
a memory, and
a processor coupled to the memory to implement,
a modeler configured to build an occlusion object model for an image containing an occlusion object;
a renderer configured to render the occlusion object model according to a geometric relationship between the occlusion object and a face image containing no occlusion object, such that the rendered occlusion object image and the face image containing no occlusion object have same scale and attitude; and
a merger configured to merge the face image containing no occlusion object and the rendered occlusion object image into an occluded face image.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the occlusion object include a rigid occlusion object or a non-rigid occlusion object.

### Claim 3 (depends on 2)

The apparatus according to claim 2, the processor is to further implement a preprocessor configured to perform first preprocessing on a face image containing the non-rigid occlusion object, the first preprocessing including performing face detection and face feature point detection on the face image containing the non-rigid occlusion object.

### Claim 4 (depends on 3)

The apparatus according to claim 3, wherein the first preprocessing further includes performing the face detection and the face feature point detection automatically or manually.

### Claim 5 (depends on 3)

The apparatus according to claim 3, wherein the first preprocessing further includes optimizing face feature points in the non-rigid occlusion object by a standard face or average face image.

### Claim 6 (depends on 5)

The apparatus according to claim 5, wherein the first preprocessing further includes performing segmentation on the optimized face image, to obtain a non-rigid occlusion object image.

### Claim 7 (depends on 6)

The apparatus according to claim 6, wherein the modeler is further configured to build an occlusion object deformation model for the non-rigid occlusion object image by a Triangulated Irregular Network TIN.

### Claim 8 (depends on 2)

The apparatus according to claim 2, wherein the processer is to further implement a preprocessor configured to perform first preprocessing on a face image containing no occlusion object, the first preprocessing including performing face detection, face feature point detection and face segmentation, wherein the face feature point detection includes extracting face feature points.

### Claim 9 (depends on 8)

The apparatus according to claim 8, wherein the renderer is further configured to:
construct a Triangulated Irregular Network TIN for the face image containing no occlusion object based on the extracted face feature points and the occlusion object model, the Triangulated Irregular Network TIN including multiple triangulated faces, and
render, for each of the multiple triangular faces, the occlusion object model by using affine transformation.

### Claim 10 (depends on 1)

The apparatus according to claim 1, wherein the merger is further configured to control a boundary of a fusion region of the occluded face image to be steadily transited by image mixing.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein the merger is further configured to control the fusion region of the occluded face image and the face image containing no occlusion object to have same illumination condition by histogram specification.

### Claim 12 (depends on 2)

The apparatus according to claim 2, wherein the processor is to further implement a preprocessor configured to perform second preprocessing on an image containing the rigid occlusion object, the second preprocessing including determining scale information including a centroid, a height and a width, of the rigid occlusion object.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein the modeler is further configured to build an occlusion object planar model according to the position and the scale information of the rigid occlusion object.

### Claim 14 (depends on 8)

The apparatus according to claim 8, wherein the first preprocessing further includes performing attitude estimation by a geometric method.

### Claim 15 (depends on 2)

The apparatus according to claim 2, wherein the renderer is further configured to calculate an affine transformation matrix based on information of the rigid occlusion object and information of the face image containing no occlusion object.

### Claim 16 (depends on 2)

The apparatus according to claim 2, the processor is to further implement a collector configured to collect a face image containing the non-rigid occlusion object in a front view.

### Claim 17

A method for data processing by an apparatus including a memory and a processor coupled to the memory, the method including:
building an occlusion object model for an image containing an occlusion object;
rendering the occlusion object model according to a geometric relationship between the occlusion object and a face image containing no occlusion object, such that the rendered occlusion object image and the face image containing no occlusion object have same scale and attitude; and
merging the face image containing no occlusion object and the rendered occlusion object image into an occluded face image.

### Claim 18 (depends on 17)

The method according to claim 17, wherein the occlusion object include a rigid occlusion object and anon-rigid occlusion object.

### Claim 19 (depends on 18)

The method according to claim 18, further including performing first preprocessing on a face image containing the non-rigid occlusion object, the first preprocessing including performing face detection and face feature point detection on the face image containing the non-rigid occlusion object.

### Claim 20

A non-transitory machine-readable storage medium to store instruction codes, which when read and executed by a computer, enables the computer to implement:
building an occlusion object model for an image containing an occlusion object;
rendering the occlusion object model according to a geometric relationship between the occlusion object and a face image containing no occlusion object, such that the rendered occlusion object image and the face image containing no occlusion object have same scale and attitude; and
merging the face image containing no occlusion object and the rendered occlusion object image into an occluded face image.
