# US 10,120,844 B2 · Determining the likelihood that an input descriptor and associated text content match a target field using natural language processing techniques in preparation for an extract, transform and load process

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,120,844 B2
- Document ID: US10120844B2
- Status: Granted
- Kind: B2
- Application number: 14/522,397
- Filed: 2014-10-23
- Published: 2016-04-28
- Granted: 2018-11-06
- Earliest claimed: 2014-10-23
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Elizabeth T. Dettman; Joel C. Dubbels; Andrew R. Freed; Michael T. Payne; Michael W. Schroeder
- Web page: https://patentgrep.com/patent/US10120844B2

## CPC

- G06F 40/143: Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
- G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses}
- G06F 16/258: {Data format conversion from or to a database}
- G06F 40/103: Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191)
- G06F 40/151: Transformation
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments presented herein disclose techniques for transforming input documents having disparate formats into a normalized format (e.g., Atom, RSS, HTML, customized XML, etc.). According to one embodiment, a plurality of fields is identified in an input document that has a given format. Each field includes a descriptor and text content associated with the descriptor. For each field, semantic properties are evaluated for the descriptor and text content against a plurality of mapping rules to determine whether the field is consistent with one of a plurality of fields of a target format. Each mapping rule specifies characteristics associated with one of the fields in the target format. Once so determined, a mapping from the first field to the second field is defined.

## Claims

### Claim 1

A system, comprising:
a processor; and
a memory storing a program, which, when executed on the processor, performs an operation for mapping fields of an input document structured according to a first format, the operation comprising:
identifying a plurality of first fields in the input document, wherein each first field includes an input descriptor and a text content value included with the input descriptor;
identifying a plurality of mapping rules wherein each mapping rule specifies characteristics associated with a target field in a target format, wherein the characteristics comprise a target descriptor and a lexical answer type identifying lexical traits to locate in the plurality of first fields of the input document;
for each first field:
evaluating, via one or more natural language processing techniques, semantic properties of the input descriptor against the plurality of mapping rules to determine whether the input descriptor is consistent with one of the target fields;
evaluating, via one or more natural language processing techniques, semantic properties of the text content value against the plurality of mapping rules to determine whether the text content is consistent with one of the target fields based on the lexical answer type associated with the target field, and wherein evaluating further comprises:
determining, for each mapping rule, a descriptor score associated with the input descriptor and a content score associated with the text content value, the descriptor score and the content score indicating a likelihood that the respective input descriptor and text content value match the characteristics specified in the mapping rule; and
converging the descriptor score and the content score into a consolidated score based on a weighting between the descriptor score and the content score specified by the associated mapping rule; and
determining, based on evaluating the semantic properties of the input descriptor and the text content against the plurality of mapping rules, that the first field corresponds to a target field; and
upon determining that the first field is corresponds to the target field, defining a mapping from the first field to the corresponding target field.

### Claim 2 (depends on 1)

The system of claim 1, wherein the operation further comprises:
generating a normalized document by mapping the text content of each first field to the respective corresponding target field; and
sending the generated normalized document to a target system.

### Claim 3 (depends on 1)

The system of claim 1, wherein the operation further comprises, converting the first format of the input document into the target format based on the defined mappings.

### Claim 4 (depends on 1)

The system of claim 1, wherein the operation further comprises, sending the input document having the target format to a target system.

### Claim 5 (depends on 1)

The system of claim 1, wherein the target format is an Atom syndication format.

### Claim 6 (depends on 1)

The system of claim 1, wherein determining that the first field corresponds to the target field comprises:
determining whether the consolidated score for the mapping rule associated with one of the second fields exceeds a specified threshold.

### Claim 7

A computer program product, comprising:
a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured to perform an operation for mapping fields of an input document structured according to a first format, the operation comprising:
identifying a plurality of first fields in the input document, wherein each first field includes an input descriptor and a text content value included with the input descriptor;
identifying a plurality of mapping rules wherein each mapping rule specifies characteristics associated with a target field in a target format, wherein the characteristics comprise a target descriptor and a lexical answer type identifying lexical traits to locate in the plurality of first fields of the input document; and
for each first field:
evaluating, via one or more natural language processing techniques, semantic properties of the input descriptor against the plurality of mapping rules to determine whether the input descriptor is consistent with one of the target fields;
evaluating, via one or more natural language processing techniques, semantic properties of the text content value against the plurality of mapping rules to determine whether the text content is consistent with one of the target fields based on the lexical answer type associated with the target field, the lexical answer type identifying lexical traits to locate in the first field, and wherein evaluating further comprises:
determining, for each mapping rule, a descriptor score associated with the input descriptor and a content score associated with the text content value, the descriptor score and the content score indicating a likelihood that the respective input descriptor and text content value match the characteristics specified in the mapping rule; and
converging the descriptor score and the content score into a consolidated score based on a weighting between the descriptor score and the content score specified by the associated mapping rule; and
determining, based on evaluating the semantic properties of the input descriptor and the text content against the plurality of mapping rules, that the first field corresponds to a target field; and
upon determining that the first field corresponds to the target field, defining a mapping from the first field to the corresponding target field.

### Claim 8 (depends on 7)

The computer program product of claim 7, wherein the operation further comprises:
generating a normalized document by mapping the text content of each first field to the respective corresponding target field; and
sending the generated normalized document to a target system.

### Claim 9 (depends on 7)

The computer program product of claim 7, wherein the operation further comprises, converting the first format of the input document into the target format based on the defined mappings.

### Claim 10 (depends on 7)

The computer program product of claim 7, wherein the operation further comprises, sending the input document having the target format to a target system.

### Claim 11 (depends on 7)

The computer program product of claim 7, wherein determining that the first field corresponds to the target field comprises:
determining whether the consolidated score for the mapping rule associated with one of the second fields exceeds a specified threshold.