Template-based redesign of a document based on document content

A system for redesigning documents, the system comprising:

• a memory component;
• and
• a processing device coupled to the memory component, the processing device to perform operations comprising:
• generating an input layout image representing an input document, wherein the input layout image comprises a representation of an input element of the input document, and wherein the input element is associated with a first tag describing the input element;
• selecting a template for modifying the input document;
• accessing a template layout image representing the template, wherein the template layout image comprises a representation of a template element of the template, and wherein the template element is associated with a second tag describing the template element;
• use a prediction model trained in a generative adversarial network (GAN) to predict an interpolated layout image from the input layout image and the template layout image, wherein the interpolated layout image comprises a representation of an interpolated element associated with a third tag describing the interpolated element;
• matching the input element to the interpolated element in the interpolated layout image predicted by the prediction model, based on a similarity between a shape of the input element as represented in the input layout image and a shape of the interpolated element as represented in the interpolated layout image and further based on a similarity between the third tag describing the interpolated element and the first tag describing the input element;
• and
• generating an output document by modifying a layout of the input document to match the interpolated layout image by fitting the input element into the shape of the interpolated element of the interpolated layout image, based on matching the input element to the interpolated element.

A non-transitory computer-readable medium embodying program code for redesigning documents, the program code comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:

• selecting a template for modifying an input document, wherein selecting the template comprises:
• generating a similarity graph comprising (a) input nodes representing input elements of the input document, (b) template nodes representing template elements of a template, and (b) edges connecting the input nodes to the templates nodes;
• computing a similarity weight for a particular edge in the similarity graph, the particular edge connecting an input node representing an input element of the input document to a template node representing a template element in the template, based on comparing a shape and tag of the input element to a shape and tag of the template element;
• computing a score for the template based on the similarity graph comprising the similarity weight associated with the particular edge of the similarity graph;
• and
• selecting the template based on the score;
• using a prediction model trained in a generative adversarial network (GAN) to determine an interpolated layout from the input document and the template, wherein the interpolated layout represents an interpolation between the input document and the template;
• and
• generating an output document by modifying an input element of the input document to match an interpolated element represented in the interpolated layout.