# US 10,872,435 B2 · Method and apparatus for iteratively establishing object position > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,435 B2 - Document ID: US10872435B2 - Status: Granted - Kind: B2 - Application number: 16/165,087 - Filed: 2018-10-19 - Published: 2020-04-23 - Granted: 2020-12-22 - Earliest claimed: 2018-10-19 - Assignees: HERE Global B.V. - Applicants: Here Global B.V. - Inventors: Anirudh Viswanathan - Web page: https://patentgrep.com/patent/US10872435B2 ## CPC - G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera} - G01C 11/06: by comparison of two or more pictures of the same area - G01C 21/3815: {Road data} - G01C 21/3837: {Data obtained from a single source} - G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)} - G06T 7/74: {involving reference images or patches} - G06V 20/582: {of traffic signs} - G06T 2207/20212: Image combination - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method, apparatus and computer program product are provided for iteratively establishing an optimized position of a feature from a plurality of images of the feature. Methods may include: receiving sensor data from at least one image sensor; processing the plurality of images to detect features within each of the plurality of images; for every combination of two images including the first feature, determining an estimated first feature position based on a triangulated position; establishing a spread of estimated first feature positions for the plurality of combinations of two images; determining a best-fit line passing through the spread of estimated first feature positions; iteratively minimizing a perpendicular error projection of each estimated first feature position relative to the best-fit line to obtain a reduced spread of optimized estimated first feature positions; and establishing from the reduced spread of optimized estimated feature positions an optimized estimated first feature position. ## Claims ### Claim 1 An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least: receive sensor data from at least one image sensor, wherein the sensor data comprises a plurality of images, and each image comprises sensor position data; process the plurality of images using a detection module to detect features within each of the plurality of images; parse the plurality of images for images including a first feature; for each of a plurality of combinations of two images including the first feature, determine an estimated first feature position based on a triangulated position between the first feature in each of the two images and the sensor position data associated with each of the two images; establish a spread of estimated first feature positions for the plurality of combinations of two images; determine a best-fit line passing through the spread of estimated first feature positions; iteratively reduce a perpendicular error projection of each estimated first feature position relative to the best-fit line to obtain a reduced spread of estimated first feature positions; establish from the reduced spread of estimated first feature positions a resulting estimated first feature position; and position an image of the first feature within a map based on the established resulting estimated first feature position. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the apparatus is further caused to provide for autonomous control of a vehicle in response to the position of the image of the first feature within the map. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the position data comprises image sensor orientation data with respect to the first feature. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein causing the apparatus to determine an estimated feature position comprises causing the apparatus to determine an estimated position of an anchor point of the first feature, wherein the anchor point of the first feature is consistent across the images including the first feature. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein causing the apparatus to determine an estimated first feature position based on a triangulated position between the first feature in each of the two images and the sensor position data associated with each of the two images comprises causing the apparatus to determine an estimated anchor point position of the first feature based on a position of the anchor point in each of the two images and an orientation of the respective image sensor that captured each of the two images. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the at least one image sensor is associated with a vehicle traveling along a road segment, wherein the first feature comprises a road sign along the road segment. ### Claim 7 (depends on 6) The apparatus of claim 6, wherein the road sign comprises information regarding travel restrictions along the road segment, wherein the apparatus is further caused to provide for autonomous control of the vehicle based, at least in part, on the information of the road sign. ### Claim 8 A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to: receive sensor data from at least one image sensor, wherein the sensor data comprises a plurality of images, and each image comprises sensor position data; process the plurality of images using a detection module to detect features within each of the plurality of images; parse the plurality of images for images including a first feature; for each of a plurality of combinations of two images including the first feature, determine an estimated first feature position based on a triangulated position between the first feature in each of the two images and the sensor position data associated with each of the two images; establish a spread of estimated first feature positions for the plurality of combinations of two images; determine a best-fit line passing through the spread of estimated first feature positions; iteratively reduce a perpendicular error projection of each estimated first feature position relative to the best-fit line to obtain a reduced spread of estimated first feature positions; establish from the reduced spread of estimated first feature positions a resulting estimated first feature position; and position an image of the first feature within a map based on the established resulting estimated first feature position. ### Claim 9 (depends on 8) The computer program product of claim 8, further comprising program code instructions to provide for autonomous control of a vehicle in response to the position of the image of the first feature within the map. ### Claim 10 (depends on 8) The computer program product of claim 8, wherein the position data comprises image sensor orientation data with respect to the first feature. ### Claim 11 (depends on 8) The computer program product of claim 8, wherein the program code instructions to determine an estimated feature position comprise program code instructions to determine an estimated position of an anchor point of the first feature, wherein the anchor point of the first feature is consistent across the images including the first feature. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the program code instructions to determine an estimated first feature position based on a triangulated position between the first feature in each of the two images and the sensor position data associated with each of the two images comprise program code instructions to determine an estimated anchor point position of the first feature based on a position of the anchor point in each of the two images and an orientation of the respective image sensor that captured each of the two images. ### Claim 13 (depends on 8) The computer program product of claim 8, wherein the at least one image sensor is associated with a vehicle traveling along a road segment, wherein the first feature comprises a road sign along the road segment. ### Claim 14 (depends on 13) The computer program product of claim 13, wherein the road sign comprises information regarding travel restrictions along the road segment, wherein the computer program product further includes program code instructions to provide for autonomous control of the vehicle based, at least in part, on the information of the road sign. ### Claim 15 A method comprising: receiving sensor data from at least one image sensor, wherein the sensor data comprises a plurality of images, and each image comprises sensor position data; processing the plurality of images using a detection module to detect features within each of the plurality of images; parsing the plurality of images for images including a first feature; for each of a plurality of combinations of two images including the first feature, determining an estimated first feature position based on a triangulated position between the first feature in each of the two images and the sensor position data associated with each of the two images; establishing a spread of estimated first feature positions for the plurality of combinations of two images; determine a best-fit line passing through the spread of estimated first feature positions; iteratively reducing a perpendicular error projection of each estimated first feature position relative to the best-fit line to obtain a reduced spread of estimated first feature positions; establishing from the reduced spread of optimized estimated first feature positions a resulting estimated first feature position; and positioning an image of the first feature within a map based on the established resulting estimated first feature position. ### Claim 16 (depends on 15) The method of claim 15, further comprising providing for autonomous control of a vehicle in response to the position of the image of the first feature within the map. ### Claim 17 (depends on 15) The method of claim 15, wherein the position data comprises image sensor orientation data with respect to the first feature. ### Claim 18 (depends on 15) The method of claim 15, wherein determining an estimated feature position comprises determining an estimated position of an anchor point of the first feature, wherein the anchor point of the first feature is consistent across the images including the first feature. ### Claim 19 (depends on 18) The method of claim 18, wherein determining an estimated first feature position based on a triangulated position between the first feature in each of the two images and the sensor position data associated with each of the two images comprises determining an estimated anchor point position of the first feature based on a position of the anchor point in each of the two images and an orientation of the respective image sensor that captured each of the two images. ### Claim 20 (depends on 15) The method of claim 15, wherein the at least one image sensor is associated with a vehicle traveling along a road segment, wherein the first feature comprises a road sign along the road segment comprising information regarding travel restrictions along the road segment, wherein the method further comprises providing for autonomous control of the vehicle based, at least in part, on the information of the road sign.