Indoor mobile robot glass detection and map updating method based on depth image inpainting

An indoor mobile robot glass detection and map updating method based on depth image inpainting, comprising:

• S1, processing laser radar information to obtain laser radar distance data, and screening, based on the distance data, a region in which glass is suspected to be present;
• S2, selecting an image of an RGBD camera based on information of the region in which glass is suspected to be present, using a convolutional neural network to identify the image of the RGBD camera, and determining whether there is glass in the region, defining an absence of glass as a first situation, and defining a presence of glass as a second situation;
• S3, normally performing, when a result is the first situation, map updating without inpainting;
• S4, determining, when the result is the second situation, a defect point type in depth data obtained by the RGBD camera, wherein using the defect point as a center, if a quantity of defect points of a same type in a neighboring region is less than or equal to a first threshold, it is determined that the defect point is a first-type defect point, or if a quantity of defect points of the same type in the neighboring region is greater than a first threshold, it is determined that the defect point is a second-type defect point;
• S5, performing supplementing, when the defect point is the first-type defect point, by using a median filtering, or first detecting, when the defect point is the second-type defect point, a defect edge and then calculating distance values between the defect point and around the defect point to perform supplementing;
• and
• S6, performing plane sampling on an inpainted depth image to obtain distance data, and outputting the distance data to a map updating step to obtain an inpainted new map for planning, wherein
• S1 comprises:
• S1.1, defining a distance change threshold and a variance threshold;
• S1.2, continuously calculating a difference Δ=|D(0)−Dt(0)−1| between two pieces of data before and after in the returned distance data, searching for a timestamp Ti in which a distance difference is greater than the distance change threshold and denoting a set of the timestamp Ti as T, recording laser radar data Gi of these points and denoting a set of the laser radar data Gi as G, and recording laser radar distance information Si of N timestamps after the points in T and denoting a set of the laser radar distance information Si as S;
• S1.3, calculating an average value Ei of data in the set S according to the following formula, and denoting the set of the average value Ei as E;
• E
• i
• =
• 1
• N
• ⁢
• ∑
• n
• =
• 1
• N
• ⁢
• S
• i
• n
• (
• 1
• )
• calculating a variance Di of data in the set S according to the following calculation formula, and denoting a variance set as D;
• and
• D
• i
• =
• 1
• N
• ⁢
• ∑
• n
• =
• 1
• N
• ⁢
• (
• S
• i
• n
• -
• E
• i
• )
• 2
• (
• 2
• )
• recording data corresponding to a timestamp exceeding the variance threshold and screening out a value of Di>=Dmin in D, and recording its index number i and denoting the set of the index number i as I;
• and
• S1.4, selecting, based on the set I, laser radar data corresponding to the index number in G, setting a maximum length of a segment and dividing, based on time continuity, data corresponding to the timestamp exceeding the variance threshold into a plurality of segments, and recording the segment as Gsuspect, that is, a segment in which glass is suspected to be present.