# US 10,872,244 B2 · Road marking extraction from in-vehicle video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,244 B2 - Document ID: US10872244B2 - Status: Granted - Kind: B2 - Application number: 15/248,823 - Filed: 2016-08-26 - Published: 2017-03-02 - Granted: 2020-12-22 - Earliest claimed: 2015-08-31 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Alexander Bovyrin; Nikita Manovich - Web page: https://patentgrep.com/patent/US10872244B2 ## CPC - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/22: Display screens - B60K 2360/146: Instrument input by gesture - B60W 2420/403: Image sensing, e.g. optical camera - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract A two-stage pipeline includes a first stage that is a static filter applied to an inverse perspective mapping (IPM) image. A SLAT (Symmetrical Local Adaptive Threshold) method, according to some embodiments, is more adaptive to environment changes. Additional geometry restrictions in the static filter remove clutter. The second stage of the pipeline is a dynamic filter. The motion of blobs is analyzed to find regions that look and behave like road marking. ## Claims ### Claim 1 A method comprising: applying a static filter to a current input image by: extracting connected components of bright pixels in the current input image, wherein extracting the connected components of bright pixels includes dynamically determining a threshold to distinguish bright pixels; applying geometric restrictions to remove connected components that do not look like road marking; and after applying the geometric restrictions, outputting static filtered connected components in the current input image; and applying a dynamic filter by: matching the static filtered connected components in the current input image to connected components of bright pixels from a previous input image; building a trajectory for each of the static filtered connected components in the current input image based on the matching; and removing any static filtered connected components with trajectories that are not substantially along a direction in which a vehicle is moving, wherein applying the static filter comprises: applying a symmetrical local filter with zero threshold; analyzing all found bright pixels to find an improved threshold; and removing all bright pixels that are below the improved threshold. ### Claim 2 (depends on 1) The method of claim 1 wherein applying geometric restrictions includes applying restrictions that define known geometric characteristics of lane markings. ### Claim 3 (depends on 2) The method of claim 2 wherein applying restrictions includes applying one or more restrictions based on a number of pixels and shape of lane markings. ### Claim 4 (depends on 3) The method of claim 3 wherein applying restrictions includes analyzing at least one of a length or thickness of a candidate lane marking. ### Claim 5 (depends on 1) The method of claim 1 including grouping pixels in a histogram based on brightness to distinguish bright pixels. ### Claim 6 (depends on 1) The method of claim 1 including removing trajectories based on a known speed of travel of a vehicle. ### Claim 7 (depends on 6) The method of claim 6 including removing trajectories based on shape. ### Claim 8 (depends on 1) The method of claim 1 including transforming an image taken by a camera facing in a direction of vehicle motion to a bird's eye depiction. ### Claim 9 One or more non-transitory computer readable media storing instructions to perform a sequence comprising: applying a static filter to a current input image by: extracting connected components of bright pixels in the current input image wherein extracting the connected components of bright pixels includes dynamically determining a threshold to distinguish bright pixels; applying geometric restrictions to remove connected components that do not look like road marking; and after applying the geometric restrictions, outputting static filtered connected components in the current input image; and applying a dynamic filter by: matching the static filtered connected components in the current input image to connected components of pixels from a previous input image; building a trajectory for each of the static filtered connected components in the current input image based on the matching; and removing any static filtered connected components with trajectories that are not substantially along a direction in which a vehicle is moving, wherein applying the static filter comprises: applying a symmetrical local filter with zero threshold; analyzing all found bright pixels to find an improved threshold; and removing all bright pixels that are below the improved threshold. ### Claim 10 (depends on 9) The media of claim 9, further storing instructions to perform a sequence wherein applying geometric restrictions includes applying restrictions that define known geometric characteristics of lane markings. ### Claim 11 (depends on 10) The media of claim 10, further storing instructions to perform a sequence wherein applying restrictions includes applying one or more restrictions based on a number of pixels and shape of lane markings. ### Claim 12 (depends on 11) The media of claim 11, further storing instructions to perform a sequence wherein applying restrictions includes analyzing at least one of a length or thickness of a candidate lane marking. ### Claim 13 (depends on 9) The media of claim 9, further storing instructions to perform a sequence including grouping pixels in a histogram based on brightness to distinguish bright pixels. ### Claim 14 (depends on 9) The media of claim 9, further storing instructions to perform a sequence including removing trajectories based on a known speed of travel of a vehicle. ### Claim 15 (depends on 14) The media of claim 14, further storing instructions to perform a sequence including removing trajectories based on shape. ### Claim 16 (depends on 9) The media of claim 9, further storing instructions to perform a sequence including transforming an image taken by a camera facing in a direction of vehicle motion to a bird's eye depiction. ### Claim 17 An apparatus comprising: a processor to: apply a static filter to a current input image by: extracting connected components of bright pixels, wherein extracting the connected components of bright pixels includes dynamically determining a threshold to distinguish bright pixels, applying geometric restrictions to remove connected components that do not look like road marking, and after applying the geometric restrictions, outputting static filtered connected components in the current input image; and apply a dynamic filter by: matching the static filtered connected components in the current input image to connected components of bright pixels from a previous input image, building a trajectory for each of the static filtered connected components in the current input image based on the matching, and removing any static filtered connected components with trajectories that are not substantially along a direction in which a vehicle is moving, wherein applying the static filter comprises applying a symmetrical local filter with zero threshold, analyzing all found bright pixels to find an improved threshold, and removing all bright pixels that are below the improved threshold; and a memory coupled to said processor. ### Claim 18 (depends on 17) The apparatus of claim 17 wherein said processor to apply restrictions that define known geometric characteristics of lane markings. ### Claim 19 (depends on 18) The apparatus of claim 18 wherein said processor to apply one or more restrictions based on a number of pixels and shape of lane markings. ### Claim 20 (depends on 19) The apparatus of claim 19 wherein said processor to analyze at least one of a length or thickness of a candidate lane marking. ### Claim 21 (depends on 17) The apparatus of claim 17, said processor to group pixels in a histogram based on brightness to distinguish bright pixels. ### Claim 22 (depends on 17) The apparatus of claim 17, said processor to remove trajectories based on a known speed of travel of a vehicle. ### Claim 23 (depends on 22) The apparatus of claim 22, said processor to remove trajectories based on shape. ### Claim 24 (depends on 17) The apparatus of claim 17, said processor to transform an image taken by a camera facing in a direction of vehicle motion to a bird's eye depiction.