# US 10,769,499 B2 · Method and apparatus for training face recognition model > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,499 B2 - Document ID: US10769499B2 - Status: Granted - Kind: B2 - Application number: 16/179,292 - Filed: 2018-11-02 - Published: 2019-05-09 - Granted: 2020-09-08 - Earliest claimed: 2017-11-03 - Assignees: Fujitsu Limited - Applicants: Fujitsu Limited - Inventors: Meng Zhang; Rujie Liu; Jun Sun - Web page: https://patentgrep.com/patent/US10769499B2 ## CPC - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 40/172: {Classification, e.g. identification} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and apparatus for removing black eyepits and sunglasses in first actual scenario data having an image containing a face acquired from an actual scenario, to obtain second actual scenario data; counting a proportion of wearing glasses in the second actual scenario data; dividing original training data composed of an image containing a face into wearing-glasses and not-wearing-glasses first and second training data, where a proportion of wearing glasses in the original training data is lower than a proportion in the second actual scenario data; generating wearing-glasses third training data based on glasses data and the second training data; generating fourth training data in which a proportion of wearing glasses is equal to the proportion of wearing glasses in the second actual scenario data, based on the third training data and the original training data; and training a face recognition model based on the fourth training data. ## Claims ### Claim 1 A method for training a face recognition model, comprising: removing black eyepits and sunglasses in first actual scenario data composed of an image containing a face which is acquired from an actual scenario, to obtain second actual scenario data; counting a proportion wearing glasses in the second actual scenario data; dividing original training data composed of an image containing the face into wearing-glasses first training data and not-wearing-glasses second training data, where a proportion of wearing glasses in the original training data is lower than a proportion of wearing glasses in the second actual scenario data; generating wearing-glasses third training data based on glasses data and the second training data; generating fourth training data in which a proportion wearing glasses is equal to the proportion of wearing glasses in the second actual scenario data, based on the third training data and the original training data; and training the face recognition model based on the fourth training data. ### Claim 2 (depends on 1) The method according to claim 1, wherein the dividing comprises: detecting a position of the face in the original training data; aligning the face in the original training data with a standard face; judging, using one of a deep convolutional neural network and a classifier, whether an aligned face in the original training data is wearing glasses, to divide the original training data into the first training data and the second training data. ### Claim 3 (depends on 1) The method according to claim 1, wherein the glasses data comprises keypoint information, glasses images, and lens information of various types of glasses, the keypoint information representing a shape and a structure of the glasses, the lens information identifying a boundary between lenses and a frame. ### Claim 4 (depends on 3) The method according to claim 3, wherein the generating wearing-glasses third training data based on the glasses data and the second training data comprises: randomly selecting a glasses type from existing glasses types; randomly changing a glasses image corresponding to the selected glasses type; recognizing keypoint information near eyes in the second training data; merging the image containing the face in the second training data with a randomly changed glasses image using an affine transformation, based on keypoint information corresponding to the randomly changed glasses image and recognized keypoint information near the eyes, to obtain the third training data. ### Claim 5 (depends on 4) The method according to claim 4, wherein the randomly changing a glasses image corresponding to the selected glasses type comprises at least one of: randomly selecting a glasses frame of an existing type to substitute a glasses frame in the glasses image; randomly changing a thickness of the glasses frame; randomly changing a shape of the glasses frame; and randomly changing a color of glasses lenses. ### Claim 6 (depends on 4) The method according to claim 4, wherein when the selected glasses type is near-sighted glasses, the merging comprises: determining the recognized merged area and lens area around the eyes using an affine transformation, based on keypoint information and lens information corresponding to the randomly changed glasses image as well as the recognized keypoint information near the eyes; extracting, from the image containing the face in the second training data, an area in the image which is around the eyes and larger than the lens area, and zooming the area to have a same size as the lens area, and superimposing a zooming result with the randomly changed glasses image, to obtain map data; substituting original data in the merged area with the map data. ### Claim 7 (depends on 1) The method according to claim 1, wherein the original training data is acquired from one of an Internet and collected manually. ### Claim 8 An apparatus for training a face recognition model, comprising: a memory, and a processor coupled to the memory and the processor is to: remove black eyepits and sunglasses in first actual scenario data composed of an image containing a face which is acquired from an actual scenario, to obtain second actual scenario data; count a proportion wearing glasses in the second actual scenario data; divide original training data composed of an image containing a face into wearing-glasses first training data and not-wearing-glasses second training data, where the proportion of wearing glasses in the original training data is lower than the proportion of wearing glasses in the second actual scenario data; generate wearing-glasses third training data based on glasses data and the second training data; generate fourth training data in which the proportion of wearing glasses is equal to the proportion of wearing glasses in the second actual scenario data, based on the third training data and the original training data; and train the face recognition model based on the fourth training data. ### Claim 9 (depends on 8) The apparatus according to claim 8, wherein the processor is further to: detect a position of the face in the original training data; align the face in the original training data with a standard face; judge whether the aligned face in the original training data is wearing glasses, to divide the original training data into the first training data and the second training data. ### Claim 10 (depends on 8) The apparatus according to claim 8, wherein the glasses data comprises keypoint information, glasses images, and lens information of various types of glasses, the keypoint information representing a shape and a structure of the glasses, the lens information identifying a boundary between lenses and a frame. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein the processor is further to: randomly select a glasses type from existing glasses types; randomly change a glasses image corresponding to a selected glasses type; recognize keypoint information near eyes in the second training data; merge the image containing the face in the second training data with randomly changed glasses image using an affine transformation, based on the keypoint information corresponding to the randomly changed glasses image and the recognized keypoint information near the eyes, to obtain the third training data. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein the processor is further to: randomly select a glasses frame of an existing type to substitute a glasses frame in the glasses image; randomly change a thickness of the glasses frame; randomly change a shape of the glasses frame; randomly change a color of glasses lenses. ### Claim 13 (depends on 11) The apparatus according to claim 11, wherein when the selected glasses type is near-sighted glasses, the processor is further to: determine a recognized merged area and lens area around the eyes using an affine transformation, based on keypoint information and lens information corresponding to the randomly changed glasses image as well as recognized keypoint information near the eyes; extract, from the image containing the face in the second training data, an area in the image which is around the eyes and larger than the lens area, and zoom the area to have a same size as the lens area, and superimpose a zooming result with the randomly changed glasses image, to obtain map data; substitute original data in the merged area with the map data. ### Claim 14 (depends on 8) The apparatus according to claim 8, wherein the original training data is acquired from one of an Internet and collected manually. ### Claim 15 A non-transitory computer-readable storage medium storing thereon a program that, when executed by an information processing apparatus, causes the information processing apparatus to execute the operations of: removing black eyepits and sunglasses in first actual scenario data composed of an image containing a face which is acquired from an actual scenario, to obtain second actual scenario data; counting a proportion of wearing glasses in the second actual scenario data; dividing original training data composed of an image containing a face into wearing-glasses first training data and not-wearing-glasses second training data, where the proportion of wearing glasses in the original training data is lower than the proportion of wearing glasses in the second actual scenario data; generating wearing-glasses third training data based on glasses data and the second training data; generating fourth training data in which a proportion of wearing glasses is equal to the proportion of wearing glasses in the second actual scenario data, based on the third training data and the original training data; and training the face recognition model based on the fourth training data.