# US 2025/0148025 A1 · Web page transformer for structure information extraction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148025 A1
- Document ID: US20250148025A1
- Status: Published application
- Kind: A1
- Application number: 19/012,342
- Filed: 2025-01-07
- Published: 2025-05-08
- Earliest claimed: 2021-12-02
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Qifan Wang; Dongfang Liu
- Web page: https://patentgrep.com/patent/US20250148025A1

## CPC

- G06F 16/322: {Trees}
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses}
- G06F 16/80: of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML (content-based retrieval of web data G06F16/95)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The technology provides a rich attention mechanism for structured information extraction of web pages and other electronic documents. An input layer of a model obtains system, information associated with the document, including field tokens representing respective fields to be extracted from the document, structured document type tokens associated, and text tokens from a text sequence in the document. An encoder connects the field tokens, the S type tokens and the text tokens according to a set of different attention patterns. The encoder generates an overall token representation based on the set of different attention patterns. An output layer of the model extracts a final text span for the each of the respective fields from the set of text tokens. The extracted final text span for each of the respective fields is stored in memory, and can be produced in response to a search query, analytics evaluation or other request.

## Claims

### Claim 1

A method for structured information extraction from electronic documents, the method comprising:
encoding, by at least one processor, a plurality of different sets of tokens from an input sequence with corresponding sets of attention patterns, the input sequence being associated with a structured electronic document;
generating, by the at least one processor, an overall token representation based on the encoding;
extracting, by the at least one processor, a text span for respective fields in the structured electronic document; and
storing, by the at least one processor, the extracted text span for each of the respective fields, the extracted text span for each of the respective fields being configured for production in response to a request.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of different sets of tokens includes:
a set of field tokens each representing the respective fields in the structured electronic document;
a set of structured document type (“S type”) tokens representing structural characteristics of the structured electronic document; and
a set of text tokens from a text sequence in the structured electronic document.

### Claim 3 (depends on 2)

The method of claim 2, wherein the corresponding sets of attention patterns includes at least two of:
an S type-to-S type attention that models relations among the S type tokens via graph attention;
an S type-to-text attention that bridges each respective S type token in the set with a corresponding text token from the set of text tokens;
a text-to-S type attention that propagates information from the set of S type tokens to the set of text tokens; or
a text-to-text attention.

### Claim 4 (depends on 3)

The method of claim 3, wherein the structured electronic document has a Hypertext Markup Language (“HTML”) layout, and the set of S type tokens is a set of HTML tokens.

### Claim 5 (depends on 4)

The method of claim 4, wherein the S type-to-S type attention is HTML-to-HTML attention in which the set of HTML tokens are connected via a document object model (“DOM”) tree graph.

### Claim 6 (depends on 2)

The method of claim 2, wherein the extracted text span for each of the respective fields is stored in memory as a contextual representation of the structured electronic document.

### Claim 7 (depends on 2)

The method of claim 2, wherein the overall token representation includes output embeddings for the set of field tokens, the set of S type tokens, and the set of text tokens, and the text span for each of the respective fields is computed based on an encoded field-dependent text embedding for that respective field.

### Claim 8 (depends on 2)

The method of claim 2, further comprising:
converting each token of the set of field tokens, the set of S type tokens, and the set of text tokens into a corresponding d-dimensional embedding vector.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
the set of S type tokens are formulated by concatenating a tag embedding and a segment embedding, and wherein the segment embedding indicates a set to which a given token belongs, and
embeddings according to the converting are trainable based on one or more hyperparameters selected during training of a model for the structured information extraction.

### Claim 10 (depends on 9)

The method of claim 9, wherein the one or more hyperparameters include at least one of batch size, training epoch, optimizer type, learning rate schedule, initial learning rate, learning rate warmup step amount, or vocabulary size.

### Claim 11 (depends on 2)

The method of claim 2, wherein a set of contextual layers connect the set of field tokens, the set of S type tokens, and the set of text tokens with the corresponding sets of attention patterns.

### Claim 12 (depends on 11)

The method of claim 11, wherein each contextual layer in the set of contextual layers is identical to each other contextual layer in the set of contextual layers.

### Claim 13 (depends on 2)

The method of claim 2, wherein:
the structured electronic document comprises a plurality of fields and the set of field tokens is a set of field tokens having each token correspond to one of the plurality of fields; and
field information for each field token is jointly encoded so that the plurality of fields shares a unique encoder.

### Claim 14

A processing system configured for structured information extraction from electronic documents, the processing system comprising:
memory configured to store data associated with the structured information extraction; and
one or more processors operatively coupled to the memory, the one or more processors being configured to:
encode a plurality of different sets of tokens from an input sequence with corresponding sets of attention patterns, the input sequence being associated with a structured electronic document;
generate an overall token representation based on the encoding;
extract a text span for respective fields in the structured electronic document; and
store, in the memory, the extracted text span for each of the respective fields, the extracted text span for each of the respective fields being configured for production in response to a request.

### Claim 15 (depends on 14)

The processing system of claim 14, wherein the extracted text span for each of the respective fields is stored in the memory as a contextual representation of the structured electronic document.

### Claim 16 (depends on 14)

The processing system of claim 14, wherein:
the plurality of different sets of tokens includes a set of field tokens each representing the respective fields in the structured electronic document, a set of structured document type (“S type”) tokens representing structural characteristics of the structured electronic document, and a set of text tokens from a text sequence in the structured electronic document; and
the corresponding sets of attention patterns includes at least two of: an S type-to-S type attention that models relations among the S type tokens via graph attention, an S type-to-text attention that bridges each respective S type token in the set with a corresponding text token from the set of text tokens, a text-to-S type attention that propagates information from the set of S type tokens to the set of text tokens, or a text-to-text attention.

### Claim 17 (depends on 16)

The processing system of claim 16, wherein the one or more processors are further configured to convert each token of the set of field tokens, the set of S type tokens, and the set of text tokens into a corresponding d-dimensional embedding vector.

### Claim 18 (depends on 17)

The processing system of claim 17, wherein the S type tokens are formulated by concatenation of a tag embedding and a segment embedding, and the segment embedding indicates a set to which a given token belongs.

### Claim 19 (depends on 17)

The processing system of claim 17, wherein embeddings according to the conversion are trainable based on one or more hyperparameters selected during training of a model for the structured information extraction, wherein the one or more hyperparameters include at least one of batch size, training epoch, optimizer type, learning rate schedule, initial learning rate, learning rate warmup step amount, or vocabulary size.

### Claim 20 (depends on 16)

The processing system of claim 16, wherein a set of contextual layers connect the set of field tokens, the set of S type tokens, and the set of text tokens with the corresponding sets of attention patterns.
