# US 12,762,868 B2 · Method for predicting turn points > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,868 B2 - Document ID: US12762868B2 - Status: Granted - Kind: B2 - Application number: 18/292,862 - Filed: 2021-07-28 - Published: 2024-10-31 - Granted: 2026-10-13 - Earliest claimed: 2021-07-28 - Assignees: Harman International Industries, Incorporated - Applicants: Harman International Industries, Incorporated - Inventors: Andrey Viktorovich Filimonov; Dmitry Vladimirovich Gorbunov; Dmitry Aleksandrovich Yashunin; Tamir Igorevich Baydasov; Yuliya Gennadevna Kukushkina - Web page: https://patentgrep.com/patent/US12762868B2 ## CPC - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G06V 10/82: using neural networks - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Computer-implemented method for predicting one or more turn points related to a road a vehicle is travelling on, the one or more turn points indicating locations where the vehicle can change direction, the method comprising: obtaining training images of roads and their environment; receiving labels associated with the roads in the training images, each label comprising a training turn marker; training an artificial neural network on a training dataset to predict one or more turn points, wherein the training dataset comprises the received labels and the obtained training images; recording at least one road image of a road and its environment; and processing the road image by the artificial neural network to predict one or more turn points on the road image. ## Claims ### Claim 1 A computer-implemented method for predicting one or more turn points related to a road a vehicle is travelling on, the one or more turn points indicating locations where the vehicle can change direction, the method comprising: obtaining training images of roads and their environment; receiving labels associated with the roads on the training images, each label comprising a training turn marker, wherein each training turn marker comprises a turn line indicative of a road border section where a vehicle can change travelling direction and physically exit a carriageway of the road, wherein a turn line indicates the road border section only if the beginning and the end of the road border section are visible on the training image; training an artificial neural network on a training dataset to predict one or more turn points, wherein the training dataset comprises the received labels and the obtained training images; recording at least one road image of a road and its environment; and processing the road image by the artificial neural network to predict one or more turn points on the road image. ### Claim 2 (depends on 1) The method of claim 1, wherein each training turn marker further comprises a turn point. ### Claim 3 (depends on 1) The method of claim 1, further comprising determining, for each turn line, a turn point at the centre of the turn line. ### Claim 4 (depends on 1) The method of claim 1, wherein a turn line indicates a road border section comprising a section of a road border of a main road where: the main road forms a junction, an intersection, or a crossroads to a crossed road; and/or an exit of a main road is blocked by a temporary barrier, a physical separating strip, and/or traffic signage blocking traffic. ### Claim 5 (depends on 1) The method of claim 1, wherein a turn line does not indicate a road border section of a main road where one or more of the following apply: the main road is crossed by a crosswalk, the main road is curved without comprising any of a junction, intersection or crossroads, an edge of a road border section is invisible on the training image, and/or an edge of a road that is not an edge of a carriageway of the road. ### Claim 6 (depends on 1) The method of claim 1, wherein the training images include training images with randomly added shadows, colour transformations, horizontal flipping, blurring, random resizing and/or random cropping. ### Claim 7 (depends on 1) The method of claim 1, wherein the artificial neural network comprises an output layer comprising outputs indicative of heat maps indicative of one or more of turn lines, turn points, and road segments. ### Claim 8 (depends on 7) The method of claim 7, further comprising, for each heat map: applying a step function to set values below a predefined threshold to zero and all other values to one, determining one or more contiguous zones of non-zero values, and for each zone, determining a centre of mass position of the zone as a turn point. ### Claim 9 (depends on 1) The method of claim 1, further comprising applying a Gaussian filter to the labels of the training dataset. ### Claim 10 (depends on 1) The method of claim 1, wherein training the artificial neural network comprises minimizing a mean squared error of the predicted turn points with respect to the training turn markers. ### Claim 11 (depends on 10) The method of claim 10, further comprising: determining a confidence value for the predicted turn points on one or more road images; comparing the confidence value to a predetermined threshold; and including the one or more road images into the training dataset as training images if the confidence value is below the threshold. ### Claim 12 (depends on 10) The method of claim 10, further comprising: displaying the road image and/or other environmental data, superimposed with graphical and/or text output based on the predicted turn points. ### Claim 13 (depends on 1) The method of claim 1, wherein the steps of recording the road image and processing the road image are executed by a computer attached to or comprised in a mobile device. ### Claim 14 (depends on 1) The method of claim 1, wherein the training dataset further comprises: at least one label indicating a boundary of at least one image segment; and/or for each image segment, a label for a segment type indicating an object represented by the segment; and wherein training and/or processing includes predicting boundaries and/or types of image segments. ### Claim 15 (depends on 14) The method of claim 14, wherein predicting boundaries and types of image segments comprises application of online hard example mining, and/or wherein training comprises applying a binary cross-entropy loss function. ### Claim 16 (depends on 1) The method of claim 1, further comprising recording the training and/or inference images by a vehicle-mounted camera. ### Claim 17 (depends on 1) The method of claim 1, further comprising: recording the training images at a fixed frame rate, and removing each training image if it depicts the same junction as a second training image. ### Claim 18 (depends on 13) A system for predicting turn points indicating locations where the vehicle can change direction, the system comprising means for executing the steps of claim 13.