Method for determining a cleaning information, method for training of a neural network algorithm, control unit, camera sensor system, vehicle, computer program and storage medium

A method for determining a cleaning information for an at least partially blocked camera sensor, which comprises a blockage on a transparent camera sensor component in an optical path of the camera sensor, wherein the method uses a neural network algorithm that comprises:

• controlling the camera sensor to capture at least one camera image,
• processing, by a computing device, the at least one camera image with a neural network algorithm, wherein the neural network algorithm is configured to determine as an output, from the at least one camera image, a degree of camera sensor blockage by segmentation of at least a part of the at least one camera image and a blockage class of the camera sensor blockage from a plurality of blockage classes by classification of at least a part of the at least one camera image,
• wherein the degree of camera sensor blockage relates to a comparison of a first area of segments in the at least one camera image depicting the blockage to a second area of unblocked segments in the at least one camera image depicting surroundings;
• determining cleaning information in dependency of the degree of camera sensor blockage and the blockage class of the camera sensor blockage, wherein the cleaning information describes that a cleaning of the camera sensor is required, if a cleaning criterion is assigned to the blockage class of the camera sensor blockage and if at least one degree threshold is exceeded by the determined degree of camera sensor blockage, and
• transmitting the cleaning information to a cleaning device associated with the camera sensor in order to clean the camera sensor according to the cleaning information,
• wherein the neural network algorithm is trained by a training method comprising the following steps:
• providing a plurality of training camera images captured by at least one unblocked camera sensor,
• augmenting at least some of the training camera images by superimposing a blockage mask on each training camera image, wherein the blockage mask is assigned to a blockage class of the plurality of blockage classes, wherein the blockage mask blocks a portion of the training camera image according to a blocking degree of the blockage mask, wherein the blocking degree is determined stochastically for each training camera image,
• associating a label to each training camera image, wherein the label describes the blockage class and the blocking degree of the blockage mask augmented to the training camera image,
• generating an output of the neural network algorithm for each augmented training camera image by processing the augmented camera image through one or more network layers of the neural network algorithm in accordance with parameters associated with the one or more network layers,
• comparing the generated output for each augmented camera image with the label associated with the augmented camera image using an objective function, and
• updating the parameters associated with the one or more network layers based on the comparison.