Techniques for enhanced image capture using a computer-vision network

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:

• capturing a set of pixels representing a scene visible to an image capturing device including a display, the set of pixels including a plurality of border pixels, and each border pixel of the plurality of border pixels being located at or within a defined range of a boundary of the set of pixels;
• detecting a physical structure depicted within the set of pixels, the physical structure being represented by a subset of the set of pixels;
• generating a segmentation mask associated with the physical structure depicted within the set of pixels, the segmentation mask including one or more segmentation pixels;
• determining a pixel value for each border pixel of the plurality of border pixels;
• generating an indicator based on the pixel value of one or more border pixels of the plurality of border pixels, and the indicator representing an instruction for framing the physical structure within the display;
• and
• presenting the indicator.

A computer-implemented method, comprising:

• receiving a first set of pixels of a first image frame representing a scene visible to an image capturing device;
• detecting a physical structure depicted within the first set of pixels, the physical structure being represented by a subset of the first set of pixels;
• generating a first segmentation mask associated with the physical structure depicted within the first set of pixels of the first image frame, the first segmentation mask including one or more first segmentation pixels;
• receiving a second set of pixels of a second image frame representing the scene visible to the image capturing device;
• detecting the physical structure depicted within the second set of pixels, the physical structure being represented by a subset of the second set of pixels;
• generating a second segmentation mask associated with the physical structure depicted within the second set of pixels of the second image frame, the second segmentation mask including one or more second segmentation pixels, and at least one first segmentation pixel being different from at least one second segmentation pixel;
• generating an aggregated segmentation mask based on the first segmentation mask and the second segmentation mask, wherein the aggregated segmentation mask is generated to encompass the physical structure depicted in a third image frame captured by the image capturing device;
• and
• generating a feedback signal using the aggregated segmentation mask, the feedback signal corresponding to an instruction to change a position or orientation of the image capturing device to include the physical structure within the third image frame.

A computer-implemented method, comprising:

• capturing a set of two-dimensional (2D) images of a physical structure, each 2D image of the set of 2D images depicting the physical structure from an angle;
• generating a three-dimensional (3D) coverage metric for the set of 2D images by:
• detecting, for each pair of images within the set of 2D images, one or more feature matches between a first 2D image and a second 2D image of the pair of images, wherein each feature match of the one or more feature matches indicates that a first 3D position associated with a first pixel of the first 2D image matches a second 3D position associated with a second pixel of the second 2D image;
• transforming the set of 2D images into a graph based on a result of the detection, the graph including a plurality of nodes and a plurality of edges, each node of the plurality of nodes representing a 2D image of the set of 2D images, and each edge of the plurality of edges connecting two nodes together and representing an existence of at least one feature match between two images associated with the two nodes;
• performing a clustering operation on the graph, the clustering operation forming one or more clusters of nodes of the plurality of nodes;
• and
• generating the 3D coverage metric based on a result of performing the clustering operation;
• generating, based on the 3D coverage metric, a feedback signal in response to capturing the set of 2D images, the feedback signal indicating an instruction to capture one or more additional 2D images to add to the set of 2D images;
• capturing the one or more additional 2D images;
• and
• forming an updated set of 2D images including the set of 2D images and the one or more additional 2D images.

A computer-implemented method, comprising:

• initiating an image capture session using an image capturing device including a display;
• and
• during the image capture session:
• capturing a first 2D image of a physical structure from a first pose, the first 2D image including a first pixel associated with a first three-dimensional (3D) position on the physical structure;
• capturing a second 2D image depicting the physical structure from a second pose, the second 2D image including a second pixel associated with a second 3D position on the physical structure, and the first pose being different from the second pose;
• detecting one or more feature matches between the first 2D image and the second 2D image, wherein a feature match of the one or more feature matches indicates that the first 3D position associated with the first pixel of the first 2D image matches the second 3D position associated with the second pixel of the second 2D image;
• and
• determining a 3D reconstruction condition based on the one or more feature matches between the first 2D image and the second 2D image.