# US 2022/0036090 A1 · Image/video analysis with activity signatures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0036090 A1 - Document ID: US20220036090A1 - Status: Published application - Kind: A1 - Application number: 16/943,140 - Filed: 2020-07-30 - Published: 2022-02-03 - Earliest claimed: 2020-07-30 - Assignees: NCR Corporation - Inventors: Joshua Migdal; Vikram Srinivasan - Web page: https://patentgrep.com/patent/US20220036090A1 ## CPC - G06V 20/47: {Detecting features for summarising video content} - G06F 16/71: Indexing; Data structures therefor; Storage structures - G06F 16/739: {in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames} - G06F 18/24: Classification techniques - G06N 20/00: Machine learning - G06T 7/269: using gradient-based methods - G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition - G06V 10/72: Data preparation, e.g. statistical preprocessing of image or video features - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - G06T 2207/10016: Video; Image sequence - G06T 2207/20081: Training; Learning - G06T 2207/30196: Human being; Person - G06T 2207/30232: Surveillance - G06V 20/44: {Event detection} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Video frames from a video are compressed into a single image or a single data structure that represents a unique visual flowprint or visual signature for a given activity being modeled from the video frames. The flowprint comprises a computed summary of the original pixel values associated with the video frames within the single image and the flowprint is specific to movements occurring within the video frames that are associated with the given activity. In an embodiment, the flowprint is provided as input to a machine-learning algorithm to allow the algorithm to perform object tracking and monitoring from the flowprint rather than from the video frames of the video, which substantially improves processor load and memory utilization on a device that executes the algorithm, and substantially improved responsiveness of the algorithm. ## Claims ### Claim 1 A method, comprising: obtaining video frames from a video; generating a single image from the video frames; and providing the single image as a visual flowprint for an activity that was captured in the video frames. ### Claim 2 (depends on 1) The method of claim 1, wherein generating further includes tracking a region within each video frame associated with a modeled activity. ### Claim 3 (depends on 2) The method of claim 2, wherein identifying further includes obtaining pixel values for each video frame associated with points along an expected path of movement for the modeled activity within the region. ### Claim 4 (depends on 3) The method of claim 3, wherein obtaining the expected path further comprises determining an aggregated pixel value for each point along the expected path of movement across the video frames from the corresponding pixel values captured across the video frames for the corresponding point. ### Claim 5 (depends on 4) The method of claim 4, wherein determining further includes calculating the aggregated pixel value as an average of the corresponding pixel values across the video frames. ### Claim 6 (depends on 4) The method of claim 4, wherein determining further includes selecting the aggregated pixel value as a minimum pixel value or a maximum pixel value associated with the corresponding pixel values across the video frames. ### Claim 7 (depends on 4) The method of claim 4, wherein determining further includes calculating the aggregated pixel value as a standard deviation associated with the corresponding pixel values across the video frames. ### Claim 8 (depends on 4) The method of claim 4, wherein determining further includes calculating the aggregated pixel value as an optical flow summary value representing a magnitude of movement along the expected path of movement for the corresponding pixel values across the video frames. ### Claim 9 (depends on 1) The method of claim 1, wherein generating further includes removing known background pixel values from the video frames before generating the single image. ### Claim 10 (depends on 1) The method of claim 1, wherein generating further includes removing pixel values from the video frames that are not associated with a region of interest for a modeled activity before generating the single image. ### Claim 11 (depends on 1) The method of claim 1, wherein generating further includes extracting features from the video frames and generating the single image as a summary of each feature across the video frames. ### Claim 12 (depends on 1) The method of claim 1, wherein providing further includes determining that a size of the single image is incompatible with a machine-learning algorithm that expects a second size for an input image provided as input to the machine-learning algorithm, normalizing the single image into the second size and providing the single image in the second size as the input image to the machine-learning algorithm. ### Claim 13 A method, comprising: obtaining at least one image from a video feed; identifying a path of activity laid out within the at least one image for a modeled activity; calculating a measure over the path; aggregating the measure along the path at each point on the path as an aggregated measure; accumulating the aggregated measure into a data structure across a plurality of additional images from the video feed as a time series; and normalizing the data structure for a monitored activity appearing in the video feed and associated with the modeled activity. ### Claim 14 (depends on 13) The method of claim 13, wherein calculating further includes calculating the measure as an optical flow for the monitored activity appearing within the video feed. ### Claim 15 (depends on 13) The method of claim 13, wherein calculating further includes calculating the measure based on pixel values appearing on the path within the at least one image. ### Claim 16 (depends on 13) The method of claim 13, wherein calculating further includes calculating the measure from silhouette information provided from visual trackers. ### Claim 17 (depends on 13) The method of claim 13, wherein aggregating further includes aggregating the measure at each point along the path and along a line orthogonal to the path at the corresponding point that extends out on either side of the path by a fixed amount. ### Claim 18 (depends on 13) The method of claim 13 further comprising augmenting the data structure with data captured from at least one data source, and wherein the at least one data source comprising source data obtained from one or more of a weight sensor, a barcode scanner, a card reader, and a transaction terminal. ### Claim 19 A system, comprising: a camera; a server comprising a processor and a non-transitory computer-readable storage medium comprising executable instructions; and the executable instructions when executed by the processor from the non-transitory computer-readable storage medium causing the processor to perform processing comprising: maintaining a single aggregated data structure from video frames of a video feed based on changes occurring along a path within each video frame associated with movement for a modeled activity along the path; and providing the single aggregated data structure as a summary of an activity appearing in the video frames of the video to an application that monitors, tracks, or performs an action based on the activity. ### Claim 20 (depends on 19) The system of claim 19, wherein the application is a machine-learning algorithm that takes the single aggregated data structure as input in place of the video frames of the video.