Facial anonymization with consistent facial attribute preservation in video

A method for consistently anonymizing a source video comprising:

• using an identity-preserving encoder, trained with a decoder to preserve identity information unique to an individual in an encoded space, encoding one or more source images in a source video of a person with a source identity to generate identity-preserving encoded data representing the source identity information in the encoded space;
• using a de-identification engine, generating de-identifying encoded data representing information for a target identity different than the source identity in the encoded space,
• wherein the de-identification engine includes a de-identification encoder that is trained to preserve non-identity information not unique to an individual, but is not trained to, or is trained not to, preserve the identity information unique to an individual,
• wherein the identity-preserving encoder is a neural network trained to optimize weights of the neural network as:
• w
• ′
• =
• argmin
• w
• ⁢
• ∑
• x
• d
• ⁡
• (
• D
• ⁡
• (
• E
• w
• (
• x
• )
• )
• -
• x
• )
• where x is an input image, Ew is an encoder with a set of candidate weights w, and d is a difference function.

A system for consistently anonymizing a source video, comprising:

• one or more memories to store a source video comprising a plurality of source images of a person with a source identity;
• and
• one or more processors configured to:
• execute an identity-preserving encoder and decoder trained to preserve identity information unique to an individual in an encoded space, the identity-preserving encoder executed to encode one or more of the source images in the source video of the person with a source identity to generate identity-preserving encoded data representing the source identity information in the encoded space,
• wherein the de-identification engine includes a de-identification encoder that is trained to preserve non-identity information not unique to an individual, but is not trained to, or is trained not to, preserve the identity information unique to an individual,
• wherein the de-identification encoder is a neural network trained to optimize weights of the neural network as:
• w
• ′
• =
• argmin
• w
• ⁢
• ∑
• x
• d
• ⁡
• (
• D
• ⁡
• (
• E
• w
• (
• x
• )
• )
• -
• x
• )
• where x is an input image, Ew is an encoder with a set of candidate weights w, A is a non-identifying attribute function, and d is a difference function,
• execute a de-identification engine to generate de-identifying encoded data representing information for a target identity different than the source identity in the encoded space,
• compute a transformation based on a difference between the identity-preserving encoded data and the de-identifying encoded data, the difference representing the removal of the source identity information and the addition of the target identity information,
• swap the identity from the source to the target in the encoded space by applying the same transformation to the identity-preserving encoded data for a plurality of source images causing the removal of the source identity information and the addition of the same target identity information for all of the plurality of source images, and
• execute the decoder to decode the swapped identity encoded data to generate a plurality of output images representing the non-identity information from the plurality of source images and the same target identity, but not the source identity, information, for all the plurality of output images in an output video.