# US 11,908,171 B2 · Enhanced object detection for autonomous vehicles based on field view > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,171 B2 - Document ID: US11908171B2 - Status: Granted - Kind: B2 - Application number: 18/145,632 - Filed: 2022-12-22 - Published: 2023-08-03 - Granted: 2024-02-20 - Earliest claimed: 2018-12-04 - Assignees: Tesla, Inc. - Applicants: Tesla, Inc. - Inventors: Anting Shen; Romi Phadte; Gayatri Joshi - Web page: https://patentgrep.com/patent/US11908171B2 ## CPC - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)} - G05D 1/0253: {extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow (determining position or orientation from images G06T7/70)} - G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements - G05D 1/228: Command input arrangements located on-board unmanned vehicles - G05D 1/2435: {Extracting 3D information} - G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM] - G05D 1/2465: {using a 3D model of the environment} - G06F 18/211: Selection of the most significant subset of features - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data} - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods for enhanced object detection for autonomous vehicles based on field of view. An example method includes obtaining an image from an image sensor of one or more image sensors positioned about a vehicle. A field of view for the image is determined, with the field of view being associated with a vanishing line. A crop portion corresponding to the field of view is generated from the image, with a remaining portion of the image being downsampled. Information associated with detected objects depicted in the image is outputted based on a convolutional neural network, with detecting objects being based on performing a forward pass through the convolutional neural network of the crop portion and the remaining portion. ## Claims ### Claim 1 A method implemented by a system of one or more processors, the method comprising: obtaining a set of images from of a plurality of image sensors positioned about a vehicle, wherein image information from a subset of image sensors is associated with a forward direction of the vehicle; determining a field of view associated with the image information, the field of view being associated with a vanishing line, wherein the field of view is determined, at least in part, based on an attention-based network and the image information; generating, based on the attention-based network, information comprising a crop portion corresponding to the field of view, and a remaining portion, wherein the remaining portion is downsampled; and outputting, via a neural network based, at least in part, on the generated information, information associated with detected objects, wherein detecting objects comprises performing a forward pass through the neural network of the generated information. ### Claim 2 (depends on 1) The method of claim 1, wherein the neural network receives additional data as input, and wherein the additional data includes data from an inertial measurement unit. ### Claim 3 (depends on 1) The method of claim 1, wherein the neural network receives additional data as input, and wherein the additional data includes map data. ### Claim 4 (depends on 1) The method of claim 1, wherein the output information associated with detected objects combines detected objects from the crop portion and the remaining portion. ### Claim 5 (depends on 1) The method of claim 1, wherein output from neural network is associated with a combination of output associated with the set of images. ### Claim 6 (depends on 1) The method of claim 1, wherein the crop portion is aligned with the remaining portion. ### Claim 7 (depends on 1) The method of claim 1, wherein the vanishing line is indicative of a horizon line. ### Claim 8 A system comprising one or more processors and non-transitory computer storage media storing instructions that when executed by the one or more processors, cause the processors to perform operations comprising: obtaining a set of images from of a plurality of image sensors positioned about a vehicle, wherein image information from a subset of image sensors is associated with a forward direction of the vehicle; determining a field of view associated with the image information, the field of view being associated with a vanishing line, wherein the field of view is determined, at least in part, based on an attention-based network and the image information; generating, based on the attention-based network, information comprising a crop portion corresponding to the field of view, and a remaining portion, wherein the remaining portion is downsampled; and outputting, via a neural network based, at least in part, on the generated information, information associated with detected objects, wherein detecting objects comprises performing a forward pass through the neural network of the generated information. ### Claim 9 (depends on 8) The system of claim 8, wherein the neural network receives additional data as input, and wherein the additional data includes data from an inertial measurement unit. ### Claim 10 (depends on 8) The system of claim 8, wherein the neural network receives additional data as input, and wherein the additional data includes map data. ### Claim 11 (depends on 8) The system of claim 8, wherein the output information associated with detected objects combines detected objects from the crop portion and the remaining portion. ### Claim 12 (depends on 8) The system of claim 8, wherein output from neural network is associated with a combination of output associated with the set of images. ### Claim 13 (depends on 8) The system of claim 8, wherein the crop portion is aligned with the remaining portion. ### Claim 14 (depends on 8) The system of claim 8, wherein the vanishing line is indicative of a horizon line. ### Claim 15 Non-transitory computer storage media storing instructions that when executed by a system of one or more processors, cause the one or more processors to perform operations comprising: obtaining a set of images from of a plurality of image sensors positioned about a vehicle, wherein image information from a subset of image sensors is associated with a forward direction of the vehicle; determining a field of view associated with the image information, the field of view being associated with a vanishing line, wherein the field of view is determined, at least in part, based on an attention-based network and the image information; generating, based on the attention-based network, information comprising a crop portion corresponding to the field of view, and a remaining portion, wherein the remaining portion is downsampled; and outputting, via a neural network based, at least in part, on the generated information, information associated with detected objects, wherein detecting objects comprises performing a forward pass through the neural network of the generated information. ### Claim 16 (depends on 15) The computer storage media of claim 15, wherein the neural network receives additional data as input, and wherein the additional data includes data from an inertial measurement unit and/or map data. ### Claim 17 (depends on 15) The computer storage media of claim 15, wherein the output information associated with detected objects combines detected objects from the crop portion and the remaining portion. ### Claim 18 (depends on 15) The computer storage media of claim 15, wherein the vanishing line is indicative of a horizon line. ### Claim 19 (depends on 15) The computer storage media of claim 15, wherein output from neural network is associated with a combination of output associated with the set of images. ### Claim 20 (depends on 15) The computer storage media of claim 15, wherein the crop portion is aligned with the remaining portion.