# US 12,112,224 B2 · Integrated camera and ultra-wideband location devices and related systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,224 B2 - Document ID: US12112224B2 - Status: Granted - Kind: B2 - Application number: 17/215,888 - Filed: 2021-03-29 - Published: 2021-09-30 - Granted: 2024-10-08 - Earliest claimed: 2020-03-30 - Assignees: Wiser Systems, Inc. - Applicants: Wiser Systems, Inc. - Inventors: Seth Edward-Austin Hollar; Ryan Michael Tedrick - Web page: https://patentgrep.com/patent/US12112224B2 ## CPC - G06K 7/10306: {ultra wide band} - G01S 5/0264: {at least one of the systems being a non-radio wave positioning system} - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G06F 18/253: {of extracted features} - G06K 7/0004: {Hybrid readers} - G06K 7/10356: {using a plurality of antennas, e.g. configurations including means to resolve interference between the plurality of antennas} - G06K 7/10415: {the interrogation device being fixed in its position, such as an access control device for reading wireless access cards, or a wireless ATM (banking machines in general G07F19/00)} - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/292: Multi-camera tracking - G06V 10/806: {of extracted features} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - G08B 13/19645: {Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over} - G08B 13/2462: {Asset location systems combined with EAS (inventory, tracking, logistic G06Q10/00; entrance control systems G07C9/00)} - G08B 13/248: {EAS system combined with another detection technology, e.g. dual EAS and video or other presence detection system} - G01S 5/16: using electromagnetic waves other than radio waves - G01S 2205/01: specially adapted for specific applications - G06T 2207/30196: Human being; Person - G06T 2207/30232: Surveillance - G08B 13/19608: {Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position} - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS ## Abstract Real time location systems are provided including one or more ultra-wideband (UWB) sensors positioned in an environment; one or more image capture sensors positioned in the environment; and at least one UWB tag associated with an object in the environment to provide a tagged item in the environment. The one or more UWB sensors and the one or more image capture sensors are integrated into at least one location device. The at one location device includes a UWB location device, a combination UWB/camera location device and/or a camera location device. A location of the tagged item is tracked using the at least one location device. and wherein a location of the tagged item is tracked using the at least one location device. ## Claims ### Claim 1 A real time location system, the system comprising: one or more ultra-wideband (UWB) sensors positioned in an environment; one or more image capture sensors positioned in the environment; wherein the one or more UWB sensors and the one or more image capture sensors are integrated into at least one location device; wherein the at one location device comprises a UWB location device, a combination UWB/camera location device and/or a camera location device; and at least one UWB tag associated with an object in the environment to provide a tagged item in the environment and wherein a location of the tagged item is tracked using the at least one location device, wherein the system is configured to pair the UWB tag with the object using visual attributes and/or characteristics of the object; wherein the at least one UWB tag contains properties of the object that include visual attributes and/or characteristics of the object; wherein when a camera recognizes the object, the camera pulls visual attributes from the object and locates a best fit of the pulled visual attributes among UWB tags; and wherein the located best fit is used to identify an object ID and determine the location of the UWB tag. ### Claim 2 (depends on 1) The system of claim 1, wherein the UWB tag is integrated into a separate device. ### Claim 3 (depends on 2) The system of claim 2, wherein the separate device comprises one of a portable electronic device, a smartphone, a computer and a key fob. ### Claim 4 (depends on 1) The system of claim 1, wherein the UWB tag is affixed to the object and wherein the object comprises one of a stationary object and a moving object. ### Claim 5 (depends on 1) The system of claim 1, wherein if multiple objects are identified within a field of view of a camera image, an object that is closest to a location of the UWB tag within the camera image is selected and paired with the object that is closest. ### Claim 6 (depends on 1) The system of claim 1, wherein the system locates a paired UWB tag in the environment using a UWB network associated with the UWB location device, a camera network associated with the camera location device and/or a combination of a UWB measurements from the UWB network and camera sensor data from the camera network. ### Claim 7 (depends on 1) The system of claim 1: wherein the object comprises static attributes that do not change over time and a dynamic attribute that changes over time; and wherein the system continually updates the changing visual attributes associated with the object to facilitate camera-based tracking and object recognition of the object. ### Claim 8 (depends on 6) The system of claim 6, wherein the system updates the changing visual attributes of the object associated with the UWB tag by one of: determining the location of the object associated with the UWB tag within a field of view of a captured image or video stream; identifying the location of the object associated with the UWB tag within the captured image stream using vision processing and a proximity to the UWB tag from the captured image; and extracting dynamic visual attributes of the object from the captured image to associate with the object using vision processing. ### Claim 9 (depends on 7) The system of claim 7, wherein the changing visual attributes comprise one or more of clothing type, clothing color, hairstyles, presence or absence of a head covering, type of shoes, eye color, shirt color, height, body shape, presence or absence of a beard and/or presence or absence of eyeglasses. ### Claim 10 (depends on 1) The system of claim 1: wherein one or more UWB location devices are provided in a first portion of the environment and one or more camera location devices are provided in a second portion of the environment, different and separate from the first portion of the environment; and wherein the one or more UWB location devices are used to track the tagged item in the first portion of the environment and the one or more camera location devices are used to track the tagged item in second portion of the environment. ### Claim 11 (depends on 1) The system of claim 1, wherein one or more UWB location devices and one or more camera location devices are distributed in the environment such that the whole environment is tracked by the one or more UWB location devices and the one or more camera location devices. ### Claim 12 (depends on 1) The system of claim 1, wherein the environment includes both an indoor environment and an outdoor environment. ### Claim 13 (depends on 1) The system of claim 1, wherein a type associated with the object is unknown to the system and wherein the system uses vision processing to determine the type associated with the object. ### Claim 14 (depends on 1) The system of claim 1, wherein the system is configured to: overlaying a box around the object paired with the UWB tag on a captured image using vision processing; and projecting the UWB tag is projected onto the captured image and overlaying a circle on the capture image, the circle having its center at a location of the UWB tag, wherein the UWB tag associated with the box closest to the circle is paired with the object. ### Claim 15 (depends on 1) The system of claim 1: wherein the system stores visual attributes associated objects tagged in the system; and wherein a new object is introduced into the environment and stored visual attributes of a pre-existing object match visual attributes of the new object the system determines that then new object and the pre-existing object are a same object. ### Claim 16 (depends on 1) The system of claim 1, wherein the system locates objects in a choke point between a first tracking area of the one or more UWB sensors and a second tracking area of the one or more image capture sensors, the first and second tracking areas being separate and distinct. ### Claim 17 (depends on 1) The system of claim 1, wherein the system accesses external databases to identify the object in the environment. ### Claim 18 (depends on 1) The system of claim 1, wherein the image capture device comprises one of a charge coupled device (CDD), a LiDAR device and a CMOS chip and senses light in one or more of the visible light range; the infrared light range and the ultraviolet light range. ### Claim 19 (depends on 1) The system of claim 1, wherein the UWB tag comprises an enclosure including a processing unit and one or more sensors.