Image processing to detect edges, walls, and surfaces for a virtual painting application

A method for enabling a user to paint an image uploaded to a device, the method comprising:

• uploading an image to the device, wherein the image is of a scene having at least one wall;
• performing a depth estimation on the image to generate a depth map;
• performing a first edge detection on the image to generate a first edge map, wherein the first edge map includes a first set of edge lines of the scene outlining the at least one wall;
• performing the first edge detection on the depth map to generate a second edge map, wherein the second edge map includes a second set of edge lines of the scene;
• performing a skeletonization function on the first edge map to generate a third edge map, wherein the skeletonization function thins a width of each edge line in the first set of edge lines;
• performing the skeletonization function on the depth map to generate a fourth edge map;
• performing a second edge detection on the image to generate a fifth edge map;
• performing the second edge detection on a saturation channel of the image to generate a sixth edge map;
• generating a final edge map using the third edge map, the fourth edge map, the fifth edge map, and the sixth edge maps;
• generating a colorized image by applying color to the final edge map to paint the at least one wall;
• and
• displaying the colorized image on a display of the device.

A method for enabling a user to paint an image uploaded to a device, the method comprising:

• uploading an image to the device, wherein the image is of a scene having at least one wall, and the scene includes at least one shadow and at least one highlight;
• performing a depth estimation on the image to generate a depth map;
• performing an edge detection on the image to generate an edge map, wherein the edge map includes an outline of the at least one wall;
• generating a final edge map using the depth map and the edge map;
• performing a grayscale function on the image to generate a grayscale image;
• performing a transparency function on the image to generate an alpha gray image, wherein the alpha gray image represents the at least one shadow as an opaque region and represents the at least one highlight as a transparent region;
• performing a normalization function on the image to generate a re-gray image, wherein the normalization function includes measuring one or more values of at least one pixel of the image and normalizing the one or more values;
• generating a colorized image by applying a color to the final edge map and using the grayscale image, alpha gray image and re-gray image;
• and
• displaying the colorized image on a display of the device.

A system comprising a computing device having a processor and a memory configured to:

• receive an image uploaded to the computing device, wherein the image is of a scene having at least one wall,
• perform a depth estimation on the image to generate a depth map,
• perform an edge detection on the image to generate a first edge map, wherein the first edge map includes a first set of edge lines of the scene outlining the at least one wall,
• perform the edge detection on the depth map to generate a second edge map, wherein the first edge map includes a second set of edge lines of the scene,
• perform a skeletonization function on the first edge map to generate a third edge map, wherein the skeletonization function thins a width of each edge line in the first set of edge lines,
• perform the skeletonization function on the depth map to generate a fourth edge map,
• generate a final edge map using the third edge map and the fourth edge map,
• generate a colorized image by applying color to the final edge map to paint the at least one wall, and
• display the colorized image on a display of the computing device.

A system comprising a computing device having a processor and a memory configured to:

• receive an image uploaded to the computing device, wherein the image is of a scene having at least one wall,
• perform a depth estimation on the image to generate a depth map,
• perform a first edge detection on the image to generate a first edge map, wherein the first edge map includes a first set of edge lines of the scene outlining the at least one wall,
• perform the first edge detection on the depth map to generate a second edge map, wherein the second edge map includes a second set of edge lines of the scene,
• perform a skeletonization function on the first edge map to generate a third edge map, wherein the skeletonization function thins a width of each edge line in the first set of edge lines,
• perform the skeletonization function on the depth map to generate a fourth edge map,
• perform a second edge detection on the image to generate a fifth edge map,
• perform the second edge detection on a saturation channel of the image to generate a sixth edge map,
• generate a final edge map using the third edge map, the fourth edge map, the fifth edge map, and the sixth edge maps,
• generate a colorized image by applying color to the final edge map to paint the at least one wall, and
• display the colorized image on a display of the computing device.

A system comprising a computing device having a processor and a memory configured to:

• receive an image uploaded to the computing device, wherein the image is of a scene having at least one wall, and the scene includes at least one shadow and at least one highlight,
• perform a depth estimation on the image to generate a depth map,
• perform an edge detection on the image to generate an edge map, wherein the edge map includes an outline of the at least one wall,
• generate a final edge map using the depth map and the edge map,
• perform a grayscale function on the image to generate a grayscale image,
• perform a transparency function on the image to generate an alpha gray image, wherein the alpha gray image represents the at least one shadow as an opaque region and represents the at least one highlight as a transparent region,
• perform a normalization function on the image to generate a re-gray image, wherein the normalization function includes measuring one or more values of at least one pixel of the image and normalizing the one or more values,
• generate a colorized image by applying a color to the final edge map and using the grayscale image, alpha gray image and re-gray image, and
• display the colorized image on a display of the computing device.