# US 10,769,915 B2 · Privacy preserving camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,915 B2 - Document ID: US10769915B2 - Status: Granted - Kind: B2 - Application number: 16/236,987 - Filed: 2018-12-31 - Published: 2020-07-02 - Granted: 2020-09-08 - Earliest claimed: 2018-12-31 - Assignees: Tcl Research America Inc. - Applicants: Tcl Research America Inc. - Inventors: Haohong Wang - Web page: https://patentgrep.com/patent/US10769915B2 ## CPC - G08B 13/19686: {Interfaces masking personal details for privacy, e.g. blurring faces, vehicle license plates} - G11B 27/036: Insert-editing - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/172: {Classification, e.g. identification} - G08B 13/19608: {Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position} - H04N 23/61: based on recognised objects - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A privacy reserving camera including a frame buffer, an object detection and tracking module, an object identification module, an instance segmentation module and an object removal and inpainting module. The frame buffer stores frames of a video stream captured by the privacy preserving camera. The object detection and tracking module detects a moving object in the video stream. The object identification module determines whether the moving object is a privacy protecting object. The instance segmentation module parses the moving object in the video stream. The object removal and inpainting module removes at least a portion of the moving object from the video stream, fills an area of the removed moving object in the video stream with a compensated patch from a neighbor frame of the video stream, and outputs a modified video stream having at least a portion of the moving object replaced with the compensated patch. ## Claims ### Claim 1 A privacy preserving camera, comprising: a frame buffer storing frames of a video stream captured by the privacy preserving camera; a memory storing computer-executable instructions; and a processor, when executing the computer-executable instructions stored in the memory, configured to perform: detecting a moving object in the video stream; determining whether the moving object is a privacy protecting object, wherein when the moving object is determined not to be the privacy protecting object, outputting the video stream; parsing the moving object in the video stream when the moving object is determined as the privacy protecting object; removing at least a portion of the moving object from the video stream, filling an area of the removed moving object in the video stream with a compensated patch from a neighbor frame or a background of a current frame of the video stream, and outputting a modified video stream having at least a portion of the moving object replaced with the compensated patch; performing a skin area detection to the moving object to determine a nakedness level of the moving object; determining an identity of the moving object; and determining whether to allow a first frame at a first time instance of the moving object to appear according to the object identity of the moving object in a second frame at a second time instance and the nakedness level of the moving object in the second frame at the second time instance, the second time instance being a later time instance of the first time instance. ### Claim 2 (depends on 1) The privacy preserving camera according to claim 1, wherein the processor is further configured to perform: detecting the moving object and tracking the moving object through a video clip; and outputting a number of bounding boxes of the moving object and a class of the moving object. ### Claim 3 (depends on 1) The privacy preserving camera according to claim 1, wherein the processor is further configured to: perform a face recognition process to determine whether the moving object is a family member, a privacy requesting member, or an intruder. ### Claim 4 (depends on 3) The privacy preserving camera according to claim 3, wherein the processor is further configured to perform: inputting photos of the family member and the privacy requesting member in an initial stage; comparing the detected moving object with a database of the input photos of the family member and the privacy requesting member to confirm an identity of the moving object. ### Claim 5 (depends on 1) The privacy preserving camera according to claim 1, wherein the processor is further configured to perform: generating a segmentation mask for the moving object; and painting an area of the generated segmentation mask with a color mask. ### Claim 6 A method for privacy preserving comprising: obtaining a video stream captured by a camera and storing frames of the video stream in a frame buffer; detecting a moving object in the video stream; determining whether the moving object is a privacy protecting object; when the moving object is determined not to be the privacy protecting object, outputting the video stream; when the moving object is determined as the privacy protecting object: parsing the moving object in the video stream; removing at least a portion of the moving object from the video stream; filling an area of the removed moving object in the video stream with a compensated patch from a neighbor frame or a background of a current frame of the video stream; outputting a modified video stream having at least a portion of the moving object replaced with the compensated patch; performing a skin area detection to the moving object to determine a nakedness level of the moving object; determining an identity of the moving object; and determining whether to allow a first frame at a first time instance of the moving object to appear according to the object identity of the moving object in a second frame at a second time instance and the nakedness level of the moving object in the second frame at the second time instance, the second time instance being a later time instance of the first time instance. ### Claim 7 (depends on 6) The method according to claim 6, wherein the detecting a moving object in the video stream, further comprises: detecting the moving object and tracking the moving object through a video clip; and outputting a number of bounding boxes of the moving object and a class of the moving object. ### Claim 8 (depends on 6) The method according to claim 6, wherein the determining whether the moving object is the privacy protecting object, further comprises: performing a face recognition process to determine whether the moving object is a family member, a privacy requesting member, or an intruder. ### Claim 9 (depends on 8) The method according to claim 8, wherein the determining whether the moving object is the privacy protecting object, further comprises: inputting photos of the family member and the privacy requesting member in an initial stage; comparing the detected moving object with a database of the input photos of the family member and the privacy requesting member to confirm an identity of the moving object. ### Claim 10 (depends on 6) The method according to claim 6, wherein performing the skin area detection includes determining a skin area of an image of the moving object using a skin tone detection process. ### Claim 11 (depends on 6) The method according to claim 6, wherein the parsing the moving object in the video stream, further comprising: generating a segmentation mask for the moving object; and painting an area of the generated segmentation mask with a color mask. ### Claim 12 A non-transitory computer readable medium storing executable computer program instructions for privacy preserving, the computer program instructions comprising instructions for: obtaining a video stream captured by a camera and storing frames of the video stream in a frame buffer; detecting a moving object in the video stream; determining whether the moving object is a privacy protecting object; when the moving object is determined not to be the privacy protecting object, outputting the video stream; when the moving object is determined as the privacy protecting object: parsing the moving object in the video stream; removing at least a portion of the moving object from the video stream; filling an area of the removed moving object in the video stream with a compensated patch from a neighbor frame or a background of a current frame of the video stream; outputting a modified video stream having at least a portion of the moving object replaced with the compensated patch; performing a skin area detection to the moving object to determine a nakedness level of the moving object; determining an identity of the moving object; and determining whether to allow a first frame at a first time instance of the moving object to appear according to the object identity of the moving object in a second frame at a second time instance and the nakedness level of the moving object in the second frame at the second time instance, the second time instance being a later time instance of the first time instance. ### Claim 13 (depends on 12) The non-transitory computer readable medium according to claim 12, wherein the detecting a moving object in the video stream, further comprises: detecting the moving object and tracking the moving object through a video clip; and outputting a number of bounding boxes of the moving object and a class of the moving object. ### Claim 14 (depends on 12) The non-transitory computer readable medium according to claim 12, wherein the determining whether the moving object is the privacy protecting object, further comprises: performing a face recognition process to determine whether the moving object is a family member, a privacy requesting member, or an intruder. ### Claim 15 (depends on 14) The non-transitory computer readable medium according to claim 14, wherein the determining whether the moving object is the privacy protecting object, further comprises: inputting photos of the family member and the privacy requesting member in an initial stage; comparing the detected moving object with a database of the input photos of the family member and the privacy requesting member to confirm an identity of the moving object. ### Claim 16 (depends on 12) The non-transitory computer readable medium according to claim 12, wherein performing the skin area detection includes determining a skin area of an image of the moving object using a skin tone detection process. ### Claim 17 (depends on 12) The non-transitory computer readable medium according to claim 12, wherein the parsing the moving object in the video stream, further comprising: generating a segmentation mask for the moving object; and painting an area of the generated segmentation mask with a color mask.