# US 12,112,540 B2 · Queue monitoring in occlusion conditions through computer vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,540 B2 - Document ID: US12112540B2 - Status: Granted - Kind: B2 - Application number: 17/497,732 - Filed: 2021-10-08 - Published: 2022-04-14 - Granted: 2024-10-08 - Earliest claimed: 2020-10-09 - Assignees: Sensormatic Electronics Corporation - Applicants: Sensormatic Electronics, LLC - Inventors: Michael C. Stewart - Web page: https://patentgrep.com/patent/US12112540B2 ## CPC - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/225: {based on a marking or identifier characterising the area} - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06T 2207/10016: Video; Image sequence - G06T 2207/30196: Human being; Person - G06T 2207/30232: Surveillance - G06T 2207/30242: Counting objects in image - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A system may be configured to perform enhanced detection of occluded objects in a multiple object detection system. In some aspects, the system may determine a first size of a queue captured in a video frame based on an object detection system and a video capture device, and determine a second size of the queue based on the first size of the queue and a correction model. Further, the system may detect a person at a front of the queue based on an object tracking system, determine a first period of time spent by the person at the front of the queue based on an object tracking system, and estimate a second period of time spent in the queue based at least in part on the first period of time and the second size of the queue. ## Claims ### Claim 1 A method comprising: determining, based on an object detection system and a video capture device, a first size of a queue captured in a video frame; determining a second size of the queue based on applying a correction model to the first size of the queue, the correction model being configured to correct a statistical bias associated with the object detection system or the video capture device; detecting, based on an object tracking system, a person at a front of the queue; determining, based on the object tracking system, a first period of time spent by the person at the front of the queue; and estimating a second period of time spent by the person at a non-front position in the queue based at least in part on the first period of time and the second size of the queue. ### Claim 2 (depends on 1) The method of claim 1, further comprising: estimating a third period of time spent in the queue, the third period of time corresponding to another person that has exited the queue; and determining, based on the second period of time and the third period of time, an average wait time associated with the queue. ### Claim 3 (depends on 1) The method of claim 1, wherein the video frame is a first video frame, and further comprising: presenting a graphical user interface displaying a second video frame associated with the queue; and receiving, via the graphical user interface, selection of a location of the front of the queue or a direction of the queue. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining a first headcount associated with the object detection system and the video capture device; comparing the first headcount to a second headcount; and generating the correction model based on the comparing. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the first period of time comprises: identifying a bounding representation closest to the front of the queue at a first time instance, the bounding representation corresponding to the person; and determining that the bounding representation is no longer closest to the front of the queue at a second time instance, the first period of time equal to a difference between the second time instance and the first time instance. ### Claim 6 (depends on 5) The method of claim 5, wherein the bounding representation is a bounding box. ### Claim 7 (depends on 1) The method of claim 1, further comprising: determining, based on a plurality of video frames, a plurality of path representations corresponding to at least one bounding representation corresponding to another person, an individual path representation identifying movement of the another person between a first video frame of the plurality of video frames and a second video frame of the plurality of video frames; and determining, based on the plurality of path representations, a location of the front of the queue or a direction of the queue. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the second size of the queue based on applying the correction model to the first size of the queue comprises: determine a statistical bias value of the statistical bias based on comparing an estimation of the length of the queue to the actual length of the queue over a period of time; and adjust the first size of the queue by the statistical bias value to obtain the second size of the queue. ### Claim 9 A system comprising: a memory storing computer-executable instructions; and at least one processor coupled with the memory and configured to execute the computer-executable instructions to: determine, based on an object detection system and a video capture device, a first size of a queue captured in a video frame; determine a second size of the queue based on applying a correction model to the first size of the queue, the correction model being configured to correct a statistical bias associated with the object detection system or the video capture device; detect, based on an object tracking system, a person at a front of the queue; determine, based on the object tracking system, a first period of time spent by the person at the front of the queue; and estimate a second period of time spent by the person at a non-front position in the queue based at least in part on the first period of time and the second size of the queue. ### Claim 10 (depends on 9) The system of claim 9, wherein the at least one processor is further configured to: estimate a third period of time spent in the queue, the third period of time corresponding to another person that has exited the queue; and determine, based on the second period of time and the third period of time, an average wait time associated with the queue. ### Claim 11 (depends on 9) The system of claim 9, wherein the video frame is a first video frame, and the at least one processor is further configured to: present a graphical user interface displaying a second video frame associated with the queue; and receive, via the graphical user interface, selection of a location of the front of the queue or a direction of the queue. ### Claim 12 (depends on 9) The system of claim 9, wherein the at least one processor is further configured to: determine a first headcount associated with the object detection system and the video capture device; compare the first headcount to a second headcount; and generate the correction model based on the comparing. ### Claim 13 (depends on 9) The system of claim 9, wherein to determine the first period of time, the at least one processor is further configured to: identify a bounding representation closest to the front of the queue at a first time instance, the bounding representation corresponding to the person; and determine that the bounding representation is no longer closest to the front of the queue at a second time instance, the first period of time equal to a difference between the second time instance and the first time instance. ### Claim 14 (depends on 13) The system of claim 13, wherein the bounding representation is a bounding box. ### Claim 15 (depends on 9) The system of claim 9, wherein the at least one processor is further configured to: determine, based on a plurality of video frames, a plurality of path representations corresponding to at least one bounding representation corresponding to another person, an individual path representation identifying movement of the another person between a first video frame of the plurality of video frames and a second video frame of the plurality of video frames; and determine, based on the plurality of path representations, a location of the front of the queue or a direction of the queue. ### Claim 16 A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising: determining, based on an object detection system and a video capture device, a first size of a queue captured in a video frame; determining a second size of the queue based on the first size of the queue and a correction model, the correction model being configured to correct a statistical bias associated with the object detection system or the video capture device; detecting, based on an object tracking system, a person at a front of the queue; determining, based on the object tracking system, a first period of time spent by the person at the front of the queue; and estimating a second period of time spent by the person at a non-front position in the queue based at least in part on the first period of time and the second size of the queue. ### Claim 17 (depends on 16) The non-transitory computer-readable device of claim 16, the operations further comprising: estimating a third period of time spent in the queue, the third period of time corresponding to another person that has exited the queue; and determining, based on the second period of time and the third period of time, an average wait time associated with the queue. ### Claim 18 (depends on 16) The non-transitory computer-readable device of claim 16, the operations further comprising: presenting a graphical user interface displaying a second video frame associated with the queue; and receiving, via the graphical user interface, selection of a location of the front of the queue or a direction of the queue. ### Claim 19 (depends on 16) The non-transitory computer-readable device of claim 16, wherein determining the first period of time comprises: determining a first headcount associated with the object detection system and the video capture device; comparing the first headcount to a second headcount; and generating the correction model based on the comparing. ### Claim 20 (depends on 16) The non-transitory computer-readable device of claim 16, wherein determining the first period of time comprises: identifying a bounding representation closest to the front of the queue at a first time instance, the bounding representation corresponding to the person; and determining that the bounding representation is no longer closest to the front of the queue at a second time instance, the first period of time equal to a difference between the second time instance and the first time instance. ### Claim 21 (depends on 16) The non-transitory computer-readable device of claim 16, the operations further comprising: determining, based on a plurality of video frames, a plurality of path representations corresponding to at least one bounding representation corresponding to another person, an individual path representation identifying movement of the another person between a first video frame of the plurality of video frames and a second video frame of the plurality of video frames; and determining, based on the plurality of path representations, a location of the front of the queue or a direction of the queue.