# US 8,752,197 B2 · Application independent system, method, and architecture for privacy protection, enhancement, control, and accountability in imaging service systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,752,197 B2 - Document ID: US8752197B2 - Status: Granted - Kind: B2 - Application number: 10/175,236 - Filed: 2002-06-18 - Published: 2003-12-18 - Granted: 2014-06-10 - Earliest claimed: 2002-06-18 - Assignees: International Business Machines Corporation - Applicants: Rudolf M. Bolle; Lisa M. Brown; Jonathan H. Connell; Arun Hampapur; Sharatchandra Pankanti; Andrew W. Senior; Ying-Li Tian - Inventors: Rudolf M. Bolle; Lisa M. Brown; Jonathan H. Connell; Arun Hampapur; Sharatchandra Pankanti; Andrew W. Senior; Ying-Li Tian - Web page: https://patentgrep.com/patent/US8752197B2 ## CPC - H04L 9/0847: {involving identity based encryption [IBE] schemes} - G06T 1/0021: {Image watermarking} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The system and method obscures descriptive image information about one or more images. The system comprises a selector for selecting the descriptive image information from one or more of the images, a transformer that transforms the descriptive information into a transformed state, and an authorizer that provides authorization criteria with the image. In a preferred embodiment, the transformed state is the respective image encoded with the descriptive information. The descriptive information can be obscured so that the descriptive information in the transformed state can be decoded only if one or more authorization inputs satisfy the authorization criteria. ## Claims ### Claim 1 A system for protecting privacy of information conveyed by one or more images in a common image stream, the system comprising: a selector for selecting descriptive data from the one or more images in a video frame in the common image stream, wherein the descriptive data includes a plurality of different objects displayed within the video frame; an encryption element configured to selectively encrypt each of the plurality of different objects displayed within the video frame in the common image stream with one or more different, respective encryption keys, wherein each of the plurality of different objects is encrypted with a different set of one or more encryption keys, wherein the one or more different, respective encryption keys are identity-specific based on an identifier of an individual user and generated using a key generation process based on the identifier, wherein an authentication module provides the identifier to the key generation process and a transformation parameter generation process in accordance with system policies in order to generate one or more encryption keys, wherein the plurality of different objects are labeled, and consistently labeled objects are encrypted and decrypted with consistent encryption keys; and an authorizer that provides authorization criteria with the one or more images, the descriptive data in an encrypted state capable of being decrypted only if one or more authorization inputs satisfy the authorization criteria, wherein each of the plurality of different objects is decrypted with one or more keys generated by the key generation process using the identifier upon authorization of the user. ### Claim 2 (depends on 1) A system, as in claim 1, wherein the system further comprises: a transformer that transforms the descriptive data in the common image stream into a transformed state, the transformed state being the respective image encoded with descriptive information, where the descriptive data is obscured in the transformed state, and wherein the transformer is configured to transform the descriptive data by performing any one or more of the following: a removal of one or more frames of the image, and an insertion of one or more additional frames in the image. ### Claim 3 (depends on 1) A system, as in claim 1, where the images are frames in a video stream, and the video stream is processed any one or more of the following ways: (a) a frame analysis; (b) a transforming of extracted information (c) a selective encoding of information about one or more foreground objects; (d) a selective encoding of information about one or more backgrounds for each of the frames of the video; and (e) a creation of certificates for the encoded descriptions vouching for their authenticity. ### Claim 4 (depends on 1) The system, as in claim 1, wherein the obscuring of the descriptive information includes a change in sequence of two or more frames of the one or more of the images. ### Claim 5 (depends on 1) The system of claim 1, wherein at least two of the components respectively convey, in a common image of said one or more images in the common image stream, the identity of different users, wherein the encryption key with which a first component of the at least two of the components is encrypted is a first key, wherein the encryption key with which a second component of the at least two of the components is encrypted is a second key, wherein the authorization criteria indicates that a first user of said users is authorized to access the first component with the first key and is not authorized to access the second component with the second key and wherein the authorization criteria indicates that a second user of said users that is different from the first user is authorized to access the second component with the second key and is not authorized to access the first component with the first key. ### Claim 6 A system for encrypting one or more images in a common image stream, the system comprising: an image input device for receiving the images in a video frame in a common image stream, wherein at least one of the images includes a plurality of different objects displayed within the video frame; an extractor that extracts one or more objects from the images; a processor that processes the images; a selector that selects one or more pieces of descriptive image information about the images according to a profile; and an encryption element configured to selectively encrypt each of the plurality of different objects displayed within the video frame in the common image stream with one or more different, respective encryption keys, wherein each of the plurality of different objects is encrypted with a different set of one or more encryption keys, wherein the one or more different, respective encryption keys are identity-specific based on an identifier of an individual user and generated using a key generation process based on the identifier, wherein an authentication module provides the identifier to the key generation process and a transformation parameter generation process in accordance with system policies in order to generate one or more encryption keys, wherein the plurality of different objects are labeled, and consistently labeled objects are encrypted and decrypted with consistent encryption keys, and an authorizer that provides authorization criteria with the one or more images, the descriptive data in an encrypted state capable of being decrypted only if one or more authorization inputs satisfy the authorization criteria, wherein each of the plurality of different objects is decrypted with one or more keys generated by the key generation process using the identifier upon authorization of the user. ### Claim 7 (depends on 6) A system, as in claim 6, where the processor at least classifies one or more of the objects into an object class and the descriptive information is about the object class. ### Claim 8 (depends on 6) A system, as in claim 6, where the processor at least identifies a relative identity of one or more of the objects in an object class and the descriptive information is about the relative identity, further comprising: a video analysis and extraction system that identifies and tracks foreground regions, categorizes one or more objects into categories, segments one or more foreground regions into component parts, and tracks the component parts of the one or more objects to infer subsequent actions of the one or more objects, wherein the information obtained from tracking the one or more objects is employed to infer present and future interactions with other objects within the same image stream. ### Claim 9 (depends on 6) A system, as in claim 6, where the processor at least identifies an absolute identity of one or more of the objects in an object class and the descriptive information is about the object class. ### Claim 10 (depends on 6) A system, as in claim 6, where the one or more objects comprise two or more persons, and the processor analyzes the two or more persons to detect one or more interrelationships among body parts of the two or more persons. ### Claim 11 (depends on 6) A system, as in claim 6, where the processor relates identity of one or more of the objects to the objects in one or more previous images in one or more previous frames to form tracks of objects. ### Claim 12 (depends on 6) A system, as in claim 6, where the descriptive information includes one or more of the following: an identity, a location, a track, a time, a region of interest, a foreground region, and a model. ### Claim 13 (depends on 6) A system, as in claim 6, further comprising: a transformer that obscures one or more of the pieces of descriptive information using an obscuring method selected according to the profile to create a transformed state of the images, where the transformer performs a reordering. ### Claim 14 (depends on 6) A system as in claim 6, further comprising: a transformer that obscures one or more of the pieces of descriptive information using an obscuring method selected according to the profile to create a transformed state of the images, where the transformer requires an authorization. ### Claim 15 (depends on 6) A system, as in claim 6, further comprising: a transformer that obscures one or more of the pieces of descriptive information using an obscuring method selected according to the profile to create a transformed state of the images, where the transformer obscures by a de-identification measure that includes any one or more of the following: a pseudo label. ### Claim 16 (depends on 6) A system, as in claim 6, further comprising: a transformer that obscures one or more of the pieces of descriptive information using an obscuring method selected according to the profile to create a transformed state of the image; and an authenticator that incorporates an access code with the obscured descriptive information in the images. ### Claim 17 (depends on 6) A system, as in claim 6, further comprising: a transformer that obscures one or more of the pieces of descriptive information using an obscuring method selected according to the profile to create a transformed state of the images; and an authenticator that incorporates an access code with the obscured descriptive information. ### Claim 18 (depends on 17) A system, as in claim 17, where the access code includes a level of detail to a reaction of a foreground object. ### Claim 19 (depends on 17) A system, as in claim 17, where the access code includes any one or more of the following non-image access codes: a token, a key, a password, a biometric, and information that is pre-registered and exclusively known to authorized users. ### Claim 20 (depends on 17) A system, as in claim 17, where the access code is a biometric within the images. ### Claim 21 (depends on 6) A system, as in claim 6, where the extractor extracts an object emotional state. ### Claim 22 (depends on 6) A system, as in claim 6, further comprising an authorizer that provides an authorization criteria with the images. ### Claim 23 (depends on 22) A system, as in claim 22, where the authorization criteria exists in levels of authorization. ### Claim 24 (depends on 23) A system, as in claim 23, where the authorization levels permit decoding of a foreground of the images. ### Claim 25 (depends on 24) A system, as in claim 24, where the decoding of the foreground is any one or more of the following: a time, an image resolution, a foreground object, a foreground object identity, a foreground object location, a track access authorization to a foreground object, a track access authorization to a level of detail of a foreground object, a reaction of a foreground object, a level of detail to a reaction of a foreground object, an interaction between two or more foreground objects, a detail of an interaction between two or more foreground objects, a foreground object position, a foreground object component, a foreground object pose, and a foreground object action. ### Claim 26 (depends on 23) A system, as in claim 23, where the authorization levels permit decoding of a background of the images. ### Claim 27 (depends on 26) A system, as in claim 26, where the decoding of the background is any one or more of the following: a time, an image resolution, a background object, an interaction between a foreground object and the background. ### Claim 28 (depends on 23) A system, as in claim 23, where the authorization levels are determined by an identity of a user or by an authority of the user. ### Claim 29 (depends on 23) A system, as in claim 23, where the authorization levels permit only statistical information about the sensitive information in the images and not to any aspect of any individual identity. ### Claim 30 (depends on 6) The system of claim 6, wherein the one or more objects comprise one or more persons, and the extractor extracts a relative identity of at least one of the one or more persons. ### Claim 31 (depends on 6) A system, as in claim 6, wherein the descriptive image information comprises an estimated speed of a person in a foreground region. ### Claim 32 (depends on 6) The system of claim 6, wherein at least two of the components respectively convey, in a common image of said one or more images in the common image stream, the identity of different users, wherein the encryption key with which a first component of the at least two of the components is encrypted is a first key, wherein the encryption key with which a second component of the at least two of the components is encrypted is a second key, wherein the system further comprises an authorizer configured to provide authorization criteria authorizing decryption of the at least two components that indicates that a first user of said users is authorized to access the first component with the first key and is not authorized to access the second component with the second key and wherein the authorization criteria indicates that a second user of said users that is different from the first user is authorized to access the second component with the second key and is not authorized to access the first component with the first key. ### Claim 33 A method for selecting descriptive information from one or more images in a common stream, the method comprising the steps of: selecting descriptive information about sensitive image information from one or more of the images in a video frame in the common image stream, wherein at least one of the images includes a plurality of different objects displayed within the video frame; transforming the descriptive information into a transformed state defined by one or more pieces of the descriptive information using an obscuring method; selectively encrypting each of the plurality of different objects displayed within the video frame in the common image stream with one or more different, respective encryption keys, wherein each of the plurality of different objects is encrypted with a different set of one or more encryption keys, wherein the one or more different, respective encryption keys are identity-specific based on an identifier of an individual user and generated using a key generation process based on the identifier, wherein an authentication module provides the identifier to the key generation process and a transformation parameter generation process in accordance with system policies in order to generate one or more encryption keys, wherein the plurality of different objects are labeled, and consistently labeled objects are encrypted and decrypted with consistent encryption keys; and providing one or more authorization criteria, the authorization criteria determined from one or more second pieces of the descriptive information, wherein each of the plurality of different objects is decrypted with one or more keys generated by the key generation process using the identifier upon authorization of the user. ### Claim 34 (depends on 33) A method, as in claim 33, further comprising the step of: decoding the transformed information to an untransformed state to be provided to a user inputting the authorization criteria only if the second piece of descriptive information provided by a user matches criteria about the transformed information. ### Claim 35 (depends on 33) The method of claim 33, wherein at least two of the components respectively convey, in a common image of said one or more images in the common image stream, the identity of different users, wherein the encryption key with which a first component of the at least two of the components is encrypted is a first key, wherein the encryption key with which a second component of the at least two of the components is encrypted is a second key, wherein the authorization criteria indicates that a first user of said users is authorized to access the first component with the first key and is not authorized to access the second component with the second key and wherein the authorization criteria specifies that a second user of said users that is different from the first user is authorized to access the second component with the second key and is not authorized to access the first component with the first key.