# US 12,183,118 B2 · Facial recognition adversarial patch adjustment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,118 B2 - Document ID: US12183118B2 - Status: Granted - Kind: B2 - Application number: 17/535,598 - Filed: 2021-11-25 - Published: 2023-05-25 - Granted: 2024-12-31 - Earliest claimed: 2021-11-25 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Inderjeet Singh; Toshinori Araki - Web page: https://patentgrep.com/patent/US12183118B2 ## CPC - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/172: {Classification, e.g. identification} - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 18/22: Matching criteria, e.g. proximity measures - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - G06V 10/82: using neural networks - G06V 40/162: {using pixel segmentation or colour matching} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Facial recognition adversarial patch adjustment is performed by applying a patch to a digital image including image data representing a face to obtain a patched image, applying a face detector to the patched image to obtain a detection value representing a likelihood that the patched image includes image data representing facial features, applying a feature extractor to the patched image to obtain a feature vector, applying a similarity determiner to the feature vector and a target feature vector to obtain a similarity value representing a likelihood that the patched image includes image data representing a target face, calculating a loss based on the detection value and the similarity value, and adjusting the patch based on the loss. ## Claims ### Claim 1 A non-transitory computer-readable medium including instructions executable by a computer to cause the computer to perform operations comprising: applying a patch to a digital image including image data representing a face to obtain a patched image; performing a face detection on the patched image to obtain a detection value representing a likelihood that the patched image includes image data representing facial features; performing a feature extraction on the patched image to obtain a feature vector; performing a similarity determination on the feature vector and a target feature vector to obtain a similarity value representing a likelihood that the patched image includes image data representing a target face; calculating a loss based on the detection value and the similarity value, wherein the calculating the loss includes calculating a detection loss based on a difference between the obtained detection value and a target detection value; and adjusting the patch based on the loss. ### Claim 2 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the loss is based on an additive inverse of the similarity value. ### Claim 3 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein calculating the loss includes multiplying one of the detection loss and the similarity value by a weighting factor. ### Claim 4 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the image data representing the face is located within a window of the digital image; wherein the performing the face detection outputs a plurality of detection values, each detection value representing a likelihood with respect to a corresponding window among a plurality of windows of the digital image; and wherein the detection value represents a likelihood that the patched image includes image data representing facial features within the window. ### Claim 5 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the adjusting the patch includes adjusting a color of each pixel of the patch according to a gradient based on the loss. ### Claim 6 (depends on 5) The non-transitory computer-readable medium of claim 5, wherein the applying the patch includes replacing image data of a partial area of the sample image with image data of the patch. ### Claim 7 (depends on 5) The non-transitory computer-readable medium of claim 5, wherein the operations are repeated for a number of iterations; wherein the applied patch of each iteration is the adjusted patch of an immediately preceding iteration. ### Claim 8 (depends on 7) The non-transitory computer-readable medium of claim 7, further comprising: performing a facial recognition on the patched image to obtain a recognition result; and estimating a risk value based on the recognition result. ### Claim 9 (depends on 7) The non-transitory computer-readable medium of claim 7, further comprising: applying a facial recognition system to a plurality of patched images to obtain a plurality of recognition results; and estimating a risk value based on a number of successful recognition results and a number of unsuccessful recognition results. ### Claim 10 A method comprising: applying a patch to a digital image including image data representing a face to obtain a patched image; performing a face detection on the patched image to obtain a detection value representing a likelihood that the patched image includes image data representing facial features; performing a feature extraction on the patched image to obtain a feature vector; performing a similarity determination on the feature vector and a target feature vector to obtain a similarity value representing a likelihood that the patched image includes image data representing a target face; calculating a loss based on the detection value and the similarity value, wherein the calculating the loss includes calculating a detection loss based on a difference between the obtained detection value and a target detection value; and adjusting the patch based on the loss. ### Claim 11 (depends on 10) The method of claim 10, wherein calculating the loss includes multiplying one of the detection loss and the similarity value by a weighting factor. ### Claim 12 (depends on 10) The method of claim 10, wherein the adjusting the patch includes adjusting a color of each pixel of the patch according to a gradient based on the loss. ### Claim 13 (depends on 12) The method of claim 12, wherein the operations are repeated for a number of iterations; wherein the applied patch of each iteration is the adjusted patch of an immediately preceding iteration. ### Claim 14 (depends on 13) The method of claim 13, further comprising: performing a facial recognition on the patched image to obtain a recognition result; and estimating a risk value based on the recognition result. ### Claim 15 An apparatus comprising: a controller including circuitry configured to: apply a patch to a digital image including image data representing a face to obtain a patched image; perform a face detection on the patched image to obtain a detection value representing a likelihood that the patched image includes image data representing facial features; perform feature extraction on the patched image to obtain a feature vector; perform similarity determination on the feature vector and a target feature vector to obtain a similarity value representing a likelihood that the patched image includes image data representing a target face; calculate a loss based on the detection value and the similarity value, wherein the calculating the loss includes calculating a detection loss based on a difference between the obtained detection value and a target detection value; and adjust the patch based on the loss. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein calculating the loss includes multiplying one of the detection loss and the similarity value by a weighting factor. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the adjusting the patch includes adjusting a color of each pixel of the patch according to a gradient based on the loss.