# US 12,602,931 B2 · Identification of unknown traffic objects > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,931 B2 - Document ID: US12602931B2 - Status: Granted - Kind: B2 - Application number: 18/489,040 - Filed: 2023-10-18 - Published: 2024-04-25 - Granted: 2026-04-14 - Earliest claimed: 2022-10-20 - Assignees: Zenseact AB - Applicants: Zenseact AB - Inventors: Willem Verbeke; Olle Månsson - Web page: https://patentgrep.com/patent/US12602931B2 ## CPC - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - B60W 60/0027: {using trajectory prediction for other traffic participants} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade} - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2520/10: Longitudinal speed - B60W 2554/4042: Longitudinal speed - B60W 2556/45: External transmission of data to or from the vehicle - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for generating training data for a machine learning (ML) algorithm configured for identification of an unknown traffic object present on a road is disclosed. The method includes obtaining sensor data from a sensor system of an ego vehicle travelling on the road, the sensor data including one or more images of a surrounding environment of the vehicle and speed information of the ego vehicle and/or speed of at least one external vehicle. The method further includes determining a presence of the unknown traffic object in the surrounding environment of the ego vehicle and determining a change of speed of the ego vehicle and/or of the at least one external vehicle. In an instance of a co-occurrence of the determined change of speed and the determined presence of the unknown traffic object, the method includes selecting one or more images of the at least one unknown traffic object. ## Claims ### Claim 1 A computer-implemented method for generating training data for a machine learning (ML) algorithm configured for identification of at least one unknown traffic object on a road, the method comprising: obtaining, by a processing unit of a vehicle, sensor data from a sensor system of an ego vehicle travelling on the road, the sensor data comprising one or more images, captured by a vehicle-mounted camera, of a surrounding environment of the vehicle and information indicative of a speed of the ego vehicle and/or speed of at least one external vehicle; determining, by the processing unit, a presence of at least one unknown traffic object in the surrounding environment of the ego vehicle based on the obtained sensor data; determining, by the processing unit, a change of speed of the ego vehicle, and/or of the at least one external vehicle travelling on the road and being present in the surrounding environment of the ego vehicle, based on the obtained sensor data; wherein, said change of speed corresponds to a substantially consistent new vehicle speed which is maintained by the ego vehicle and/or by the at least one external vehicle for at least a predetermined period of time; and in an instance of a co-occurrence of the determined change of speed and the determined presence of the at least one unknown traffic object in the surrounding environment of the ego vehicle: automatically selecting and storing, by the processing unit, one or more images of the at least one unknown traffic object for forming a training data set for the ML algorithm configured for identification of the at least one unknown traffic object. ### Claim 2 (depends on 1) The computer-implemented method according to claim 1, wherein determining the new vehicle speed of the ego vehicle and/or of the at least one external vehicle is based on a real-time change of vehicle speed based on the obtained sensor data. ### Claim 3 (depends on 1) The computer-implemented method according to claim 1, wherein the method further comprises: transmitting, by the processing unit, the selected one or more images of the at least one unknown traffic object to a remote server and/or storing the selected one or more images in a memory unit of the ego vehicle for a subsequent transmission to the remote server, for generating a corresponding traffic object identification annotation for the one or more images of the at least one unknown traffic object for forming the training data set for the ML algorithm. ### Claim 4 (depends on 1) The computer-implemented method according to claim 1, wherein the method further comprises: obtaining, by the processing unit, a geographical positioning data of the ego vehicle corresponding to the co-occurrence of the determined change of speed and the determined presence of the at least one unknown traffic object in the surrounding environment of the ego vehicle; associating, by the processing unit, the co-occurrence to the obtained geographical positioning data; determining, by the processing unit, a confidence value based on confidence criteria comprising a calculated frequency of the association of the co-occurrence to the obtained geographical positioning data; and if the determined confidence value exceeds a predetermined confidence threshold value: automatically selecting and storing, by the processing unit, the one or more images of the at least one unknown traffic object for forming the training data set for the ML algorithm configured for identification of the at least one unknown traffic object. ### Claim 5 (depends on 1) The computer-implemented method according to claim 1, wherein the ego vehicle comprises an Automated Driving System; (ADS). ### Claim 6 (depends on 1) The computer-implemented method according to claim 1, wherein the method further comprises: storing the one or more selected images in a memory unit of the ego vehicle for generating the corresponding traffic object identification annotation for the one or more images of the at least one unknown traffic object for forming the training data set for the ML algorithm. ### Claim 7 (depends on 6) The computer-implemented method according to claim 6, wherein the method further comprises: determining, by the processing unit, based on the selected one or more images of the at least one unknown traffic object, that the at least one unknown traffic object is indicative of a built-up area; and generating, by the processing unit, the corresponding traffic object identification annotation for the selected one or more images of the at least one unknown traffic object indicative of the built-up area. ### Claim 8 (depends on 7) The computer-implemented method according to claim 7, wherein the method further comprises: indicating, by the processing unit, the at least one built-up area traffic object within map data associated with an HD map based on the one or more annotated images. ### Claim 9 (depends on 1) A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an in-vehicle processing system, the one or more programs comprising instructions for performing the method according to claim 1. ### Claim 10 A system for generating training data for a machine learning (ML) algorithm configured for identification of at least one unknown traffic object on a road, the system comprising a processing circuitry configured to: obtain sensor data from a sensor system of an ego vehicle travelling on the road, the sensor data comprising one or more images, captured by a vehicle-mounted camera, of a surrounding environment of the vehicle and information indicative of a speed of the ego vehicle and/or speed of at least one external vehicle; determine a presence of at least one unknown traffic object in the surrounding environment of the ego vehicle based on the obtained sensor data; determine a change of speed of the ego vehicle and/or of the at least one external vehicle travelling on the road and being present in the surrounding environment of the ego vehicle, based on the obtained sensor data; wherein, said change of speed corresponds to a substantially consistent new vehicle speed which is maintained by the ego vehicle and/or by the at least one external vehicle for at least a predetermined period of time; and in an instance of a co-occurrence of the determined change of speed and the determined presence of the at least one unknown traffic object in the surrounding environment of the ego vehicle: select one or more images of the at least one unknown traffic object for forming a training data set for the ML algorithm configured for identification of the at least one unknown traffic object. ### Claim 11 (depends on 10) The system according to claim 10, wherein the processing circuitry is further configured to: transmit the selected one or more images of the at least one unknown traffic object to a remote server and/or store the selected one or more images in a memory unit of the ego vehicle for a subsequent transmission to the remote server, for generating a corresponding traffic object identification annotation for the one or more images of the at least one unknown traffic object for forming the training data set for the ML algorithm. ### Claim 12 (depends on 10) The system according to claim 10, wherein the processing circuitry is further configured to: obtain a geographical positioning data of the ego vehicle corresponding to the co-occurrence of the determined change of speed and the determined presence of the at least one unknown traffic object in the surrounding environment of the ego vehicle; associate the co-occurrence to the obtained geographical positioning data; determine a confidence value based on confidence criteria comprising a calculated frequency of the association of the co-occurrence to the obtained geographical positioning data; and if the determined confidence value exceeds a predetermined confidence threshold value: select the one or more images of the at least one unknown traffic object for forming the training data set for the ML algorithm configured for identification of the at least one unknown traffic object. ### Claim 13 (depends on 10) The system according to claim 10, wherein the processing circuitry is further configured to: store the one or more selected images in the memory unit of the ego vehicle, and wherein the processing circuitry is further configured to generate the corresponding traffic object identification annotation for the one or more images of the at least one unknown traffic object for forming the training data set for the ML algorithm. ### Claim 14 (depends on 10) A vehicle comprising: one or more vehicle-mounted sensors configured to monitor a surrounding environment of the vehicle; a localization system configured to monitor a geographical position and heading of the vehicle; and a system according to claim 10.