# US 2021/0334666 A1 · Machine learning-based inference of granular font properties

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334666 A1
- Document ID: US20210334666A1
- Status: Published application
- Kind: A1
- Application number: 16/854,913
- Filed: 2020-04-22
- Published: 2021-10-28
- Earliest claimed: 2020-04-22
- Assignees: salesforce.com, inc.
- Applicants: salesforce.com, inc.
- Inventors: Jessica Lundin; Owen Winne Schoppe; Alan Martin Ross; Brian J. Lonsdorf; David James Woodward; Sönke Rohde; Michael Reynolds Sollami; Chetan Ramaiah
- Web page: https://patentgrep.com/patent/US20210334666A1

## CPC

- G06N 5/02: Knowledge representation; Symbolic representation
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 40/109: Font handling; Temporal or kinetic typography
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06N 5/04: Inference or reasoning models
- G06N 20/00: Machine learning
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 30/245: {Font recognition}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A textual properties model is used to infer values for certain font properties of interest given certain text-related data, such as rendered text images. The model may be used for numerous purposes, such as aiding with document layout, identifying font families that are similar to a given font families, and generating new font families with specific desired properties. In some embodiments, the model is trained from a combination of synthetic data that is labeled with values for the font properties of interest, and partially-labeled data from existing “real-world” documents.

## Claims

### Claim 1

A computer-implemented method for inference of font properties based on rendered text, the computer-implemented method comprising:
obtaining a rendered text image of text in a document;
obtaining a machine learning model trained to infer values for a plurality of font properties based on rendered text images; and
using the machine learning model to infer values for font properties of a font family of the rendered text.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising obtaining the rendered text image from a web browser textual rendering of text on a web site.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the font properties for which values are inferred comprise one or more of: horizontal character metrics, vertical character metrics, character pixel density, italic angle, serif type, or character bounding box.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising training the machine learning model as a convolutional neural network having custom twistor convolutional layers substantially providing textual rotational invariance.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising training the machine learning model, where the training set comprises a plurality of partially-labeled rendered text images and a plurality of synthetic rendered text images.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising generating the synthetic rendered text images, the generating comprising:
generating support vector graphic (SVG) data for text; and
deriving values for the plurality of font properties through image analysis of renderings of SVG;
wherein training the machine learning model uses the derived values for the plurality of font properties as training labels for the synthetic rendered text images.

### Claim 7 (depends on 5)

The computer-implemented method of claim 5, wherein the partially-labeled rendered text images specify values for less than all of the plurality of font properties.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
performing unsupervised pre-training on unlabeled rendered text images;
priming the machine learning model using results of the pre-training; and
training the primed machine learning model.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising:
using the inferred values for the font properties to identify a different font family that is visually similar to the font family of the rendered text.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising:
using the machine learning model to infer values for font properties of font families a plurality of additional rendered text images; and
using the inferred values for the font properties of the font family of the rendered text image, and of the inferred values of the font properties of the font families of the plurality of additional rendered text images, to generate a new font family.

### Claim 11

A non-transitory computer-readable storage medium containing instructions that when executed by a computer processor perform actions comprising::
obtaining a rendered text image of text in a document;
obtaining a machine learning model trained to infer values for a plurality of font properties based on rendered text images; and
using the machine learning model to infer values for font properties of a font family of the rendered text.

### Claim 12 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the actions further comprising obtaining the rendered text image from a web browser textual rendering of text on a web site.

### Claim 13 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, wherein the font properties for which values are inferred comprise one or more of: horizontal character metrics, vertical character metrics, character pixel density, italic angle, serif type, or character bounding box.

### Claim 14 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the actions further comprising training the machine learning model as a convolutional neural network having custom twistor convolutional layers substantially providing textual rotational invariance.

### Claim 15 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the actions further comprising training the machine learning model, where the training set comprises a plurality of partially-labeled rendered text images and a plurality of synthetic rendered text images.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the actions further comprising generating the synthetic rendered text images, the generating comprising:
generating support vector graphic (SVG) data for text; and
deriving values for the plurality of font properties through image analysis of renderings of SVG;
wherein training the machine learning model uses the derived values for the plurality of font properties as training labels for the synthetic rendered text images.

### Claim 17 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the actions further comprising:
performing unsupervised pre-training on unlabeled rendered text images;
priming the machine learning model using results of the pre-training; and
training the primed machine learning model.

### Claim 18 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the actions further comprising:
using the inferred values for the font properties to identify a different font family that is visually similar to the font family of the rendered text.

### Claim 19 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the actions further comprising:
using the machine learning model to infer values for font properties of font families a plurality of additional rendered text images; and
using the inferred values for the font properties of the font family of the rendered text image, and of the inferred values of the font properties of the font families of the plurality of additional rendered text images, to generate a new font family.

### Claim 20

A computer-implemented method for inference of font properties, the computer-implemented method comprising:
obtaining design metadata font properties of text in a document;
obtaining a machine learning model trained to infer values for a plurality of font properties based on design metadata font properties; and
using the machine learning model to infer values for font properties of a font family specified by the design metadata font properties, the inferred font properties not included in the obtained design metadata font properties.