Automated flow chart generation and visualization system

A method for generating digital flowcharts, the method performed at a system comprising a display, one or more processors, and memory storing computer program code executable by the one or more processors configured to cause the system to carry out the method, the method comprising:

• receiving sketch image data comprising a plurality of shapes and text;
• processing the sketch image data to generate flowchart data, wherein processing the sketch image data comprises:
• applying a first model configured to generate shape data based on the sketch image data, wherein applying the first model to generate the shape data comprises:
• detecting the presence of a shape in the sketch image data;
• generating classification data indicating a predicted class of the detected shape in the sketch image data;
• and
• generating first location data indicating a determined location of the shape;
• and
• processing the shape data to generate the flowchart data, wherein generating the flowchart data comprises generating linking data that associates the shape data with one or more other objects in the flowchart data, wherein the one or more other objects comprise an arrow shape object and a text object;
• wherein generating linking data that associates the shape data with the arrow shape object comprises:
• calculating a minimum distance between the determined location of the shape and a location of a portion of the arrow shape object;
• determining that the minimum distance is less than or equal to a predetermined threshold distance;
• and
• in accordance with determining that the minimum distance is less than or equal to the predetermined threshold distance, generating the linking data that associates the shape data with the arrow shape object;
• and
• wherein generating linking data that associates the shape data with the text object comprises:
• determining that a bounding box of the shape overlaps a bounding box of the text object;
• in accordance with determining that the bounding box of the shape overlaps the bounding box of the text object, generating the linking data that associates the shape data with the text object;
• wherein generating the flowchart data comprises, in accordance with generating the linking data that associates the shape data with the text object, modifying center coordinates of the text bounding box to match center coordinates of the shape bounding box;
• and
• generating and displaying, on the display, a visualization of the flowchart data.

A non-transitory computer-readable storage medium for generating digital flowcharts, the non-transitory computer-readable storage medium storing computer program code configured to be executed by a system comprising a display and one or more processors to cause the system to:

• receive sketch image data comprising a plurality of shapes and text;
• process the sketch image data to generate flowchart data, wherein processing the sketch image data comprises:
• applying a first model configured to generate shape data based on the sketch image data, wherein applying the first model to generate the shape data comprises:
• detecting the presence of a shape in the sketch image data;
• generating classification data indicating a predicted class of the detected shape in the sketch image data;
• and
• generating first location data indicating a determined location of the shape;
• processing the shape data to generate the flowchart data, wherein generating the flowchart data comprises generating linking data that associates the shape data with one or more other objects in the flowchart data, wherein the one or more other objects comprise an arrow shape object and a text object;
• wherein generating linking data that associates the shape data with the arrow shape object comprises:
• calculating a minimum distance between the determined location of the shape and a location of a portion of the arrow shape object;
• determining that the minimum distance is less than or equal to a predetermined threshold distance;
• and
• in accordance with determining that the minimum distance is less than or equal to the predetermined threshold distance, generating the linking data that associates the shape data with the arrow shape object;
• wherein generating linking data that associates the shape data with the text object comprises:
• determining that a bounding box of the shape overlaps a bounding box of the text object;
• in accordance with determining that the bounding box of the shape overlaps the bounding box of the text object, generating the linking data that associates the shape data with the text object;
• wherein generating the flowchart data comprises, in accordance with generating the linking data that associates the shape data with the text object, modifying center coordinates of the text bounding box to match center coordinates of the shape bounding box;
• and
• generate and display, on the display, a visualization of the flowchart data.