# US 12,223,014 B2 · Data collection and user feedback in edge video devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,014 B2 - Document ID: US12223014B2 - Status: Granted - Kind: B2 - Application number: 17/516,188 - Filed: 2021-11-01 - Published: 2023-05-04 - Granted: 2025-02-11 - Earliest claimed: 2021-11-01 - Assignees: Western Digital Technologies, Inc. - Applicants: Western Digital Technologies, Inc. - Inventors: Shaomin Xiong; Toshiki Hirano; Damien Kah; Rajeev Nagabhirava - Web page: https://patentgrep.com/patent/US12223014B2 ## CPC - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06F 18/2163: {Partitioning the feature space} - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 10/776: Validation; Performance evaluation - G06V 10/82: using neural networks - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06T 2207/10016: Video; Image sequence - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06V 2201/10: Recognition assisted with metadata - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A digital video camera architecture for updating an object identification and tracking model deployed with the camera is disclosed. The invention comprises optics, a processor, a memory, and an artificial intelligence logic which may further comprise artificial neural networks. The architecture identifies objects according to a first confidence threshold of the model and identifies candidate objects according to the first confidence threshold and a second confidence threshold. The model may track the motion of the candidate objects within a visual field, separate the candidate objects into false positive candidate objects and false negative candidate objects according to their tracked motions, and present at least a portion of the false positive candidate objects and false negative candidate objects for further annotation. ## Claims ### Claim 1 A device, comprising: an artificial intelligence logic configured to: identify candidate objects according to a first confidence threshold and a second confidence threshold, wherein a true positive candidate object and a false positive candidate object are differentiated based on a confidence score; track a motion of the candidate objects within a visual field, wherein each candidate object comprises a bounding box defining its location in the visual field; separate the candidate objects into false positive candidate objects and false negative candidate objects according to their tracked motions, wherein the separating comprises: separating a candidate object as a false positive candidate object in response to the bounding box of a candidate object occurring within an area of non-motion designated by the presence of a single logic value associated with non-motion within the bounding box; and separating a candidate object as a false negative candidate object in response to the bounding box of the candidate object occurring within an area of motion designated by the presence of a single logic value associated with motion within the bounding box, wherein a relative physical location of the area of motion is illustrated relative to the bounding box; and present at least a portion of the false positive candidate objects and false negative candidate objects for further annotation, wherein the presenting comprises sending a request to annotate a false negative candidate object that has been separated in response to determining that the object's bounding box is present in one or more previous frames in an area of motion. ### Claim 2 (depends on 1) The device of claim 1, wherein: the visual field is partitioned into areas of motion and areas of non-motion. ### Claim 3 (depends on 2) The device of claim 2, wherein the areas of motion and the areas of non-motion are partitioned according to a Hadamard matrix. ### Claim 4 (depends on 1) The device of claim 1, wherein if a defined time interval has passed since either a false positive candidate object or a false negative candidate object has been presented for further annotation, then another false positive candidate object or false negative candidate object is presented for further annotation. ### Claim 5 (depends on 4) The device of claim 4, wherein the device is a video camera. ### Claim 6 (depends on 1) The device of claim 1, wherein the device is a video camera. ### Claim 7 (depends on 1) The device of claim 1, wherein the artificial intelligence logic is configured with artificial neural networks to identify objects and track the motion of objects. ### Claim 8 A system, comprising: a plurality of installations, wherein: each installation comprises a plurality of video cameras; and each video camera comprises: an artificial intelligence logic configured to: identify candidate objects according to a first confidence threshold and a second confidence threshold, wherein a true positive candidate object and a false positive candidate object are differentiated based on a confidence score; track a motion of the candidate objects within a visual field, wherein each candidate object comprises a bounding box defining its location in the visual field; separate the candidate objects into false positive candidate objects and false negative candidate objects according to their tracked motions, wherein the separating comprises: separating a candidate object as a false positive candidate object in response to the bounding box of a candidate object occurring within an area of non-motion designated by the presence of a single logic value associated with non-motion within the bounding box; and separating a candidate object as a false negative candidate object in response to the bounding box of the candidate object occurring within an area of motion designated by the presence of a single logic value associated with motion within the bounding box, wherein a relative physical location of the area of motion is illustrated relative to the bounding box; and present at least a portion of the false positive candidate objects and false negative candidate objects for further annotation, wherein the presenting comprises sending a request to annotate a false negative candidate object that has been separated in response to determining that the object's bounding box is present in one or more previous frames in an area of motion. ### Claim 9 (depends on 8) The system of claim 8, wherein the visual field is partitioned into areas of motion and areas of non-motion. ### Claim 10 (depends on 9) The system of claim 9, wherein: the false positive candidate objects and the false negative candidate objects are analyzed according to a physical distribution of the video cameras; and the false positive candidate objects and the false negative candidate objects are selected for further annotation according to the physical distribution of the video cameras. ### Claim 11 (depends on 10) The system of claim 10, wherein if a defined time interval has passed since either a false positive candidate object or a false negative candidate object has been selected for presentation for further annotation from the same video camera, then another selected false positive candidate object or false negative candidate object from that camera is presented for further annotation. ### Claim 12 (depends on 11) The system of claim 11, wherein a separated false positive candidate object or a separated false negative candidate object is presented for human annotation on a computer. ### Claim 13 (depends on 11) The system of claim 11, further comprising: a first computer; and a second computer coupled between at least one of the video cameras and the first computer; wherein a separated false positive candidate object or a separated false negative candidate object is presented for human annotation on the second computer. ### Claim 14 A method of collecting training examples for updating an object identification and motion tracking model, the method comprising: identifying candidate objects according to a first confidence threshold and a second confidence threshold, wherein a true positive candidate object and a false positive candidate object are differentiated based on a confidence score; tracking a motion of the candidate objects within a visual field, wherein each candidate object comprises a bounding box defining its location in the visual field; separating the candidate objects into false positive candidate objects and false negative candidate objects according to their tracked motions, wherein the separating comprises: separating a candidate object as a false positive candidate object in response to the bounding box of a candidate object occurring within an area of non-motion designated by the presence of a single logic value associated with non-motion within the bounding box; and separating a candidate object as a false negative candidate object in response to the bounding box of the candidate object occurring within an area of motion designated by the presence of a single logic value associated with motion within the bounding box, wherein a relative physical location of the area of motion is illustrated relative to the bounding box; and presenting at least a portion of the false positive candidate objects and false negative candidate objects for further annotation, wherein the presenting comprises sending a request to annotate a false negative candidate object that has been separated in response to determining that the object's bounding box is present in one or more previous frames in an area of motion. ### Claim 15 (depends on 14) The method of claim 14, wherein the visual field is partitioned into areas of motion and areas of non-motion. ### Claim 16 (depends on 15) The method of claim 15, wherein the areas of motion and the areas of non-motion are partitioned according to a Hadamard matrix. ### Claim 17 (depends on 14) The method of claim 14, wherein: the false positive candidate objects and the false negative candidate objects are analyzed according to a distribution of a plurality of video cameras; and the false positive candidate objects and the false negative candidate objects are selected for further annotation according to the distribution of the plurality of video cameras. ### Claim 18 (depends on 17) The method of claim 17, wherein if a defined time interval has passed since either a false positive candidate object or a false negative candidate object has been presented for further annotation, then another false positive candidate object or false negative candidate object is presented for further annotation. ### Claim 19 (depends on 14) The method of claim 14, wherein the presenting at least a portion of the false positive candidate objects and false negative candidate objects for further annotation further comprises performing a distribution formation analysis. ### Claim 20 (depends on 19) The method of claim 19, further comprising, in response to the at least a portion of the false positive candidate objects and false negative candidate objects following a distribution, requesting feedback.