# US 12,114,084 B2 · Image based localization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,114,084 B2 - Document ID: US12114084B2 - Status: Granted - Kind: B2 - Application number: 17/872,086 - Filed: 2022-07-25 - Published: 2023-09-14 - Granted: 2024-10-08 - Earliest claimed: 2022-03-08 - Assignees: NEC Corporation Of America - Applicants: NEC Corporation Of America - Inventors: Yaacov Hoch - Web page: https://patentgrep.com/patent/US12114084B2 ## CPC - G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera} - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - H04N 23/61: based on recognised objects - H04N 23/74: by influencing the scene brightness using illuminating means - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06V 20/54: of traffic, e.g. cars on the road, trains or boats - G08G 1/096725: {where the received information generates an automatic action on the vehicle control} - G08G 1/096775: {where the origin of the information is a central station} - G08G 9/00: Traffic control systems for craft where the kind of craft is irrelevant or unspecified - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/56: provided with illuminating means - H04N 23/662: {by using camera arrangements where one camera controls another camera to affect the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image} - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - H04N 23/75: by influencing optical camera components - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract There is provided a computer implemented method of computing a location of an object, comprising: accessing a wide field of view (wFOV) image captured by a wFOV image sensor located relative to an object, analyzing the wFOV image to identify a predefined feature, wherein the predefined feature indicates a low accuracy location of the object, capturing a high resolution image by a high resolution image sensor located relative to the object, the high resolution image depicting the predefined feature, and computing a high accuracy of location of the object according to an analysis of the predefined feature and according to a correlation between a location and orientation of the wFOV image sensor and the high resolution image sensor. ## Claims ### Claim 1 A computer implemented method of computing a location of an object, comprising: accessing a wide field of view (wFOV) image captured by a wFOV image sensor installed on an object; analyzing the wFOV image to identify a predefined feature, wherein the predefined feature indicates a low accuracy location of the object; capturing a high resolution image by a high resolution image sensor installed on the object, the high resolution image depicting the predefined feature; and computing a high accuracy of location of the object according to an analysis of the predefined feature and according to a correlation between a location and orientation of the wFOV image sensor and the high resolution image sensor. ### Claim 2 (depends on 1) The computer implemented method of claim 1, further comprising feeding the high accuracy of the location into a controller for controlling movement of the object. ### Claim 3 (depends on 1) The computer implemented method of claim 1, wherein the object is selected from a group comprising: a vehicle, a robot, an autonomous car, and a drone. ### Claim 4 (depends on 1) The computer implemented method of claim 1, further comprising: generating instructions for adjustment of at least one of an orientation and a location of the high resolution image sensor for capturing the high resolution image, wherein the high accuracy location of the object is computed according to a correlation between the location and orientation of the wFOV image sensor and the orientation and location of the high resolution image sensor in response to implementing the generated instructions. ### Claim 5 (depends on 4) The computer implemented method of claim 4, wherein the location and orientation of the high resolution image sensor is adaptable relative to the object and adaptable relative to the wFOV image sensor, wherein generating instructions comprises generating instructions for execution by a controller that adjusts the at least one of the orientation and the location of the high resolution image sensor relative to the object. ### Claim 6 (depends on 5) The computer implemented method of claim 5, further comprising dynamically computing the correlation between the location and orientation of the wFOV image sensor used to capture the wFOV image and the location and orientation of the high resolution image senor used to capture the high resolution image, wherein the dynamically computed correlation is used to dynamically compute the high accuracy of the location. ### Claim 7 (depends on 4) The computer implemented method of claim 4, wherein the location and orientation of the high resolution image sensor is fixed with respect to the object and fixed with respect to the wFOV image sensor, and wherein generating instructions comprises generating instructions for execution by a controller that adjusts at least one of an orientation and location of the object thereby adjusting the at least one of orientation and location of the high resolution image sensor. ### Claim 8 (depends on 7) The computer implemented method of claim 7, further comprising computing the correlation between the location and orientation of the wFOV image sensor and the location and orientation of the high resolution image sensor prior to the capture of the wFOV image and the high resolution image, and accessing the previously computed correlation for computing the high accuracy of the location. ### Claim 9 (depends on 1) The computer implemented method of claim 1, wherein the correlation comprises a mapping from pixels of the wFOV image to pixels of the high resolution image, the mapping computed according to a relationship between a location, orientation, and pixel resolution of the wFOV image sensor and location, orientation, and pixel resolution of the high resolution image sensor. ### Claim 10 (depends on 1) The computer implemented method of claim 1, further comprising: computing an initial low accuracy location of the object based on the wFOV image; and computing a higher accuracy location of the object as a correction of the initial low accuracy location based on the high resolution image. ### Claim 11 (depends on 1) The computer implemented method of claim 1, wherein the wFOV image comprises a single image, wherein the single image is analyzed to identify a plurality of predefined features, further comprising each one of a plurality of predefined featured: accessing a high resolution image depicting the one predefined feature, computing a candidate location of the object based on the high resolution image, and computing the high accuracy of the location of the object as an aggregation of a plurality of the candidate locations. ### Claim 12 (depends on 11) The computer implemented method of claim 11, wherein the candidate location is computed as a vector pointing between the one predefined feature and the object, and wherein the high accuracy of the location of the object is computed as a intersection of a plurality of the vectors. ### Claim 13 (depends on 1) The computer implemented method of claim 1, wherein the wFOV sensor comprises a wFOV camera, and the high resolution sensor comprises a pan-tilt-zoom (PTZ) camera. ### Claim 14 (depends on 1) The computer implemented method of claim 1, wherein the wFOV sensor captures images at a resolution below a threshold above a zoom level, and the high resolution sensor captures images at a resolution above the threshold when above the zoom level. ### Claim 15 (depends on 1) The computer implemented method of claim 1, wherein the wFOV image sensor and the high resolution image sensor are a same image sensor, wherein the high resolution image is created by a physical optical zoom-in operation. ### Claim 16 (depends on 1) The computer implemented method of claim 1, further comprising activating an electromagnetic illumination source at a selected illumination pattern for generating electromagnetic illumination for providing visibility of the predefined feature, wherein at least one of the wFOV image and the high resolution images are captured at the electromagnetic spectrum of the electromagnetic illumination. ### Claim 17 (depends on 16) The computer implemented method of claim 16, wherein the electromagnetic illumination source generates electromagnetic illumination at the short wave infrared (SWIR) range, and at least one of the wFOV image sensor and the high resolution image sensor comprises a SWIR sensor. ### Claim 18 (depends on 17) The computer implemented method of claim 17, wherein the SWIR range comprises a solar blind range. ### Claim 19 A system for computing a location of an object, comprising: accessing a wide field of view (wFOV) image captured by a wFOV image sensor installed on an object; analyzing the wFOV image to identify a predefined feature, wherein the predefined feature indicates a low accuracy location of the object; capturing a high resolution image by a high resolution image sensor installed on the object, the high resolution image depicting the predefined feature; and computing a high accuracy of location of the object according to an analysis of the predefined feature and according to a correlation between a location and orientation of the wFOV image sensor and the high resolution image sensor. ### Claim 20 A non-transitory medium storing program instructions for computing a location of an object, which, when executed by a processor, cause the processor to: access a wide field of view (wFOV) image captured by a wFOV image sensor installed on an object; analyze the wFOV image to identify a predefined feature, wherein the predefined feature indicates a low accuracy location of the object; capture a high resolution image by a high resolution image sensor installed on the object, the high resolution image depicting the predefined feature; and compute a high accuracy of location of the object according to an analysis of the predefined feature and according to a correlation between a location and orientation of the wFOV image sensor and the high resolution image sensor.