Hazard detection ensemble architecture system and method

One or more non-transitory computer-readable media having computer-executable instructions stored thereon that when executed cause the following operations to be performed:

• receiving a stereo image having a red channel, a blue channel, and a green channel, wherein the red channel, the blue channel, and the green channel produce a 3-channel RGB image;
• receiving a depth image;
• concatenating the depth image to the red, green, and blue channels of the 3-channel RGB image to produce a 4-channel RGBD image, and providing the 4-channel RGBD image as a single input into a same layer of a neural network, wherein the neural network includes:
• a first plurality of layers, wherein each layer in the first plurality of layers includes:
• a convolutional layer;
• a batch normal layer;
• a leaky ReLU activation function;
• and
• a max pooling layer;
• a second plurality of layers, wherein each layer in the second plurality of layers includes:
• a convolutional layer;
• a batch normal layer;
• and
• a leaky ReLU activation function, and wherein the second plurality of layers does not include a max pooling layer;
• a third plurality of layers, wherein each layer in the third plurality of layers includes:
• a convolutional layer;
• a batch normal layer;
• a leaky ReLU activation function;
• and
• a you-only-look-once (“YOLO”) layer;
• and
• one or more skip architectures;
• determining one or more hazards within the stereo image using the neural network;
• concatenating an output of the neural network with an output of an auxiliary semantic segmentation decoder to force the neural network to learn one or more features for a drivable space to determine the one or more hazards within the stereo image;
• and
• driving a vehicle through the drivable space while avoiding the one or more hazards.

A system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media having computer-executable instructions stored thereon that when executed cause the system to perform the following operations:
• receiving a stereo image having a red channel, a blue channel, and a green channel, wherein the red channel, the blue channel, and the green channel produce a 3-channel RGB image;
• receiving a depth image;
• concatenating the depth image to the red, green, and blue channels of the 3-channel RGB image to produce a 4-channel RGBD image, and providing the 4-channel RGBD image as a single input into a same layer of a neural network, wherein the neural network includes:
• a first plurality of layers, wherein each layer in the first plurality of layers includes:
• a convolutional layer;
• a batch normal layer;
• a leaky ReLU activation function;
• and
• a max pooling layer;
• a second plurality of layers, wherein each layer in the second plurality of layers includes:
• a convolutional layer;
• a batch normal layer;
• and
• a leaky ReLU activation function, and wherein the second plurality of layers does not include a max pooling layer;
• a third plurality of layers, wherein each layer in the third plurality of layers includes:
• a convolutional layer;
• a batch normal layer;
• a leaky ReLU activation function;
• and
• a you-only-look-once (“YOLO”) layer;
• and
• one or more skip architectures;
• determining one or more hazards within the stereo image using the neural network;
• concatenating an output of the neural network with an output of an auxiliary semantic segmentation decoder to force the neural network to learn one or more features for a drivable space to determine the one or more hazards within the stereo image;
• and
• driving a vehicle through the drivable space while avoiding the one or more hazards.