# US 12,007,782 B2 · Automatic robotically steered sensor for targeted high performance perception and vehicle control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,007,782 B2 - Document ID: US12007782B2 - Status: Granted - Kind: B2 - Application number: 18/159,961 - Filed: 2023-01-26 - Published: 2023-06-01 - Granted: 2024-06-11 - Earliest claimed: 2018-08-20 - Assignees: Uatc, LLC - Applicants: Uatc, LLC - Inventors: Mark Calleija - Web page: https://patentgrep.com/patent/US12007782B2 ## CPC - G05D 1/0246: {using a video camera in combination with image processing means} - B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles} - G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras - G06F 18/25: Fusion techniques - G06F 18/251: {of input or preprocessed data} - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10) - G06V 10/98: Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/582: {of traffic signs} - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]} - G08G 1/015: with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles - G08G 1/04: using optical or ultrasonic detectors - G08G 1/09623: {Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence)} - G08G 1/163: {involving continuous checking} - G08G 1/165: {for passive traffic, e.g. including static obstacles, trees} - G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes} - G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection} - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - B60W 2554/402: Type - B60W 2555/60: Traffic rules, e.g. speed limits or right of way - B60W 2556/20: Data confidence level - B60W 2556/25: Data precision - G06T 7/73: using feature-based methods - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20076: Probabilistic image processing - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30241: Trajectory - G06T 2207/30256: Lane; Road marking - H04N 23/56: provided with illuminating means - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Disclosed are methods, systems, and non-transitory computer readable media that control an autonomous vehicle via at least two sensors. One aspect includes capturing an image of a scene ahead of the vehicle with a first sensor, identifying an object in the scene at a confidence level based on the image, determining the confidence level of the identifying is below a threshold, in response to the confidence level being below the threshold, directing a second sensor having a field of view smaller than the first sensor to generate a second image including a location of the identified object, further identifying the object in the scene based on the second image, controlling the vehicle based on the further identification of the object. ## Claims ### Claim 1 A method of controlling a vehicle, the method comprising: capturing a first image with a first sensor, wherein the first image includes an object; forecasting motion of the object in the first image; determining a capture location of the object for obtaining a second image of the object by a second sensor, wherein the capture location of the object is based on the forecasting; directing the second sensor to capture the second image of the object at the capture location, wherein the second image has a different field of view than the first image; identifying the object based on the second image; and controlling the vehicle based on the identifying the object. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining a movement vector of the object, wherein the movement vector is indicative of the motion of the object across a plurality of images captured by the first sensor. ### Claim 3 (depends on 1) The method of claim 1, further comprising: determining a delay in positioning the second sensor based on a current position of the second sensor and the motion of the object; and determining the capture location of the object based on the motion of the object and the delay. ### Claim 4 (depends on 1) The method of claim 1, wherein the capture location is based on a position of the vehicle and a time when the first image is captured. ### Claim 5 (depends on 1) The method of claim 1, wherein directing the second sensor to capture the second image of the object at the capture location further comprises: determining a distance from a current position of the second sensor to a capture position of the second sensor, wherein the capture position is indicative of the capture location; and adjusting a zoom of the second sensor based on the distance from the current position of the second sensor to the capture position of the second sensor. ### Claim 6 (depends on 5) The method of claim 5, further comprising: determining a vertical difference between the current position of the second sensor and the capture position of the second sensor; and adjusting a pitch of the second sensor, wherein adjusting the pitch compensates for the vertical difference between the current position of the second sensor and the capture position of the second sensor. ### Claim 7 (depends on 5) The method of claim 5, further comprising: determining a horizontal difference between the current position of the second sensor and the capture position of the second sensor; and adjusting a yaw of the second sensor, wherein adjusting the yaw compensates for the horizonal difference between the current position of the second sensor and the capture position of the second sensor. ### Claim 8 (depends on 1) The method of claim 1, wherein forecasting the motion of the object comprises determining a direction and a speed of the object. ### Claim 9 (depends on 1) The method of claim 1, wherein the second image has a higher resolution than the first image. ### Claim 10 A computing system for controlling a vehicle, comprising: one or more hardware processors; one or more tangible, non-transitory, computer readable media that store instructions executable by the one or more hardware processors to cause the one or more hardware processors to perform operations, the operations comprising: capturing a first image with a first sensor, wherein the first image includes an object; forecasting motion of the object in the first image; determining a capture location of the object for obtaining a second image of the object by a second sensor, wherein the capture location of the object is based on the forecasting; directing the second sensor to capture the second image of the object at the capture location, wherein the second image has a different field of view than the first image; identifying the object based on the second image; and controlling the vehicle based on the identifying the object. ### Claim 11 (depends on 10) The computing system of claim 10, the operations further comprising: determining a movement vector of the object, wherein the movement vector is indicative of the motion of the object between a plurality of images captured by the first sensor. ### Claim 12 (depends on 10) The computing system of claim 10, the operations further comprising: determining a delay in positioning the second sensor based on a current position of the second sensor and the motion of the object; and determining the capture location of the object based on the motion of the object and the delay. ### Claim 13 (depends on 10) The computing system of claim 10, wherein the second sensor includes one or more positioning motors used to traverse the second sensor at a traverse rate in a horizonal dimension and a traverse rate in a vertical dimension. ### Claim 14 (depends on 10) The computing system of claim 10, wherein directing the second sensor to capture the second image of the object at the capture location further comprise: determining a distance from a current position of the second sensor to a capture position of the second sensor, wherein the capture position is indicative of the capture location; and adjusting a zoom of the second sensor based on the distance from the current position of the second sensor to the capture position of the second sensor. ### Claim 15 (depends on 10) The computing system of claim 10, wherein forecasting the motion of the object comprises determining a direction and a speed of the object. ### Claim 16 (depends on 10) The computing system of claim 10, wherein the second image has a higher resolution than the first image. ### Claim 17 A vehicle comprising: vehicle controls for controlling the vehicle; a first sensor and a second sensor; one or more processors; a memory storing instructions that are executable by the one or more processors to cause the one or more processors to: capture a first image with the first sensor, wherein the first image includes an object; forecasting motion of the object in the first image; determining a capture location of the object for obtaining a second image of the object by a second sensor, wherein the capture location of the object is based on the forecasting; directing the second sensor to capture the second image of the object at the capture location, wherein the second image has a different field of view than the first image; identifying the object based on the second image; and controlling the vehicle based on the identifying the object. ### Claim 18 (depends on 17) The vehicle of claim 17, wherein the instructions further cause the one or more processors to: determine a movement vector of the Object, wherein the movement vector is indicative of the motion of the object between a plurality of images captured by the first sensor. ### Claim 19 (depends on 17) The vehicle of claim 17, wherein the instructions further cause the one or more processors to: determine a delay in positioning the second sensor based on a current position of the second sensor and the motion of the object; and determine the capture location of the object based on the motion of the object and the delay. ### Claim 20 (depends on 17) The vehicle of claim 17, wherein the second sensor includes one or more positioning motors used to traverse the second sensor at a traverse rate in a horizonal dimension and a traverse rate in a vertical dimension.