# US 2022/0035792 A1 · Determining metadata of a dataset

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0035792 A1
- Document ID: US20220035792A1
- Status: Published application
- Kind: A1
- Application number: 16/941,565
- Filed: 2020-07-29
- Published: 2022-02-03
- Earliest claimed: 2020-07-29
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Thomas Gschwind; Christoph Adrian Miksovic Czasch; Paolo Scotton
- Web page: https://patentgrep.com/patent/US20220035792A1

## CPC

- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 16/86: {Mapping to a database}
- G06F 16/24558: {Binary matching operations}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure relates to a method for enabling a processing of a dataset of records having a set of attributes. The method comprises: selecting a first attribute of the set of attributes and a subset of one or more second attributes of the set of attributes. Distinct values of the subset of second attributes may be determined from the dataset. For each distinct value of the determined distinct values records of the dataset that have said each distinct value may be identified, and a group of words may be formed from values of the first attribute of the identified records. Distinct word sequences may be identified in the formed groups and a level of presence of each word sequence of the word sequences in each of the formed groups may be determined. At least part of the levels of presence may be provided as metadata.

## Claims

### Claim 1

A computer implemented method for enabling a processing of a dataset of records having a set of attributes, the method comprising:
selecting a first attribute of the set of attributes and a subset of one or more second attributes of the set of attributes;
determining from the dataset distinct values of the subset of second attributes;
for each distinct value of the determined distinct values: identifying records of the dataset that have said each distinct value, and forming a group of words from values of the first attribute of the identified records;
identifying distinct word sequences in the formed groups, each word sequence comprising one or more words, and determining a level of presence of each word sequence of the word sequences in each of the formed groups; and
storing at least part of the levels of presence.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a matching request, the matching request comprising a record having a certain value of the first attribute and having or not having values for one or more attributes, called hereinafter missing attributes, of the subset of second attributes;
generating a word sequence from the certain value;
identifying the levels of presence corresponding to the generated word sequence;
determining the group associated with the highest identified level of presence;
using the distinct value of the subset of second attributes associated with the determined group for determining missing values of the missing attributes;
associating the missing values with the received record, resulting in a modified record;
comparing the modified record with records of the dataset; and
providing the result of the comparison.

### Claim 3 (depends on 2)

The method of claim 2, wherein the comparing of the modified record with a record of the records of the dataset comprises assigning a contribution weight to the generated word sequence and comparing the value of the first attribute of the record with the certain value, providing the comparison result based on the contribution weight.

### Claim 4 (depends on 2)

The method of claim 2, wherein the received record is a structured or an unstructured record.

### Claim 5 (depends on 1)

The method of claim 1, further comprising processing an incomplete dataset comprising one or more incomplete records, the processing comprising:
identifying a record of the incomplete dataset having a certain value of the first attribute and having no values for one or more attributes, called missing attributes, of the subset of second attributes;
generating a word sequence from the certain value;
identifying levels of presence corresponding to the generated word sequence;
determining the group associated with the highest identified level of presence; and
using the distinct value of the subset of second attributes associated with the determined group for determining the values of the missing attributes.

### Claim 6 (depends on 1)

The method of claim 1, wherein the level of presence of the word sequence, w, in each group, d, of the formed groups, D, is a weighted fraction tf (w,d)*idf (w,D) of words of the group d, wherein
tf
⁡
(
w
,
d
)
=
number
⁢
⁢
of
⁢
⁢
occurences
⁢
⁢
in
⁢
⁢
w
⁢
⁢
in
⁢
⁢
d
number
⁢
⁢
of
⁢
⁢
word
⁢
⁢
sequences
⁢
⁢
in
⁢
⁢
d
,
and
⁢
⁢
idf
⁡
(
w
,
D
)
=
log
⁡
(
number
⁢
⁢
of
⁢
⁢
groups
⁢
⁢
in
⁢
⁢
D
number
⁢
⁢
of
⁢
⁢
groups
⁢
⁢
in
⁢
⁢
containing
⁢
⁢
w
)
.

### Claim 7 (depends on 1)

The method of claim 1, wherein forming the group of words comprises: creating a document, and concatenating the values of the first attribute of the identified records in the document using a delimiter.

### Claim 8 (depends on 1)

The method of claim 1, wherein the storing further comprises: providing a matrix of cells being arranged in columns and rows, each column in the matrix representing a respective group of the formed groups and each row in the matrix representing a distinct word sequence of the distinct word sequences, storing the levels of presence in respective cells of the matrix.

### Claim 9 (depends on 1)

The method of claim 1, wherein the storage of the levels of presence comprises: selecting levels of presence higher than a predefined threshold and storing the selected levels of presence.

### Claim 10 (depends on 1)

The method of claim 1, further comprising repeating the method until all first attributes of the set of attributes are selected.

### Claim 11 (depends on 1)

The method of claim 1, repeating the method until possible combinations of the second subset of attributes of the of set of attributes are selected.

### Claim 12

A computer program product comprising a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured to implement a method for enabling a processing of a dataset of records having a set of attributes, the method comprising:
selecting a first attribute of the set of attributes and a subset of one or more second attributes of the set of attributes;
determining from the dataset distinct values of the subset of second attributes;
for each distinct value of the determined distinct values: identifying records of the dataset that have said each distinct value, and forming a group of words from values of the first attribute of the identified records;
identifying distinct word sequences in the formed groups, each word sequence comprising one or more words, and determining a level of presence of each word sequence of the word sequences in each of the formed groups; and
storing at least part of the levels of presence.

### Claim 13 (depends on 12)

The computer program product of claim 12, wherein the method further comprises:
receiving a matching request, the matching request comprising a record having a certain value of the first attribute and having or not having values for one or more attributes, called hereinafter missing attributes, of the subset of second attributes;
generating a word sequence from the certain value;
identifying the levels of presence corresponding to the generated word sequence;
determining the group associated with the highest identified level of presence;
using the distinct value of the subset of second attributes associated with the determined group for determining missing values of the missing attributes;
associating the missing values with the received record, resulting in a modified record;
comparing the modified record with records of the dataset; and
providing the result of the comparison.

### Claim 14 (depends on 13)

The computer program product of claim 13, wherein the comparing of the modified record with a record of the records of the dataset comprises assigning a contribution weight to the generated word sequence and comparing the value of the first attribute of the record with the certain value, providing the comparison result based on the contribution weight.

### Claim 15 (depends on 13)

The computer program product of claim 13, wherein the received record is a structured or an unstructured record.

### Claim 16 (depends on 12)

The computer program product of claim 12, wherein the method further comprises processing an incomplete dataset comprising one or more incomplete records, the processing comprising:
identifying a record of the incomplete dataset having a certain value of the first attribute and having no values for one or more attributes, called missing attributes, of the subset of second attributes;
generating a word sequence from the certain value;
identifying levels of presence corresponding to the generated word sequence;
determining the group associated with the highest identified level of presence; and
using the distinct value of the subset of second attributes associated with the determined group for determining the values of the missing attributes.

### Claim 17 (depends on 12)

The computer program product of claim 12, wherein forming the group of words comprises: creating a document, and concatenating the values of the first attribute of the identified records in the document using a delimiter.

### Claim 18 (depends on 12)

The computer program product of claim 12, wherein the storing further comprises: providing a matrix of cells being arranged in columns and rows, each column in the matrix representing a respective group of the formed groups and each row in the matrix representing a distinct word sequence of the distinct word sequences, storing the levels of presence in respective cells of the matrix.

### Claim 19 (depends on 12)

The computer program product of claim 12, wherein the storage of the levels of presence comprises: selecting levels of presence higher than a predefined threshold and storing the selected levels of presence.

### Claim 20

A computer system for enabling a processing of a dataset of records having a set of attributes, the computer system being configured for:
selecting a first attribute of the set of attributes and a subset of one or more second attributes of the set of attributes;
determining from the dataset distinct values of the subset of second attributes;
for each distinct value of the determined distinct values: identifying records of the dataset that have said each distinct value, and forming a group of words from values of the first attribute of the identified records;
identifying distinct word sequences in the formed groups, each word sequence comprising one or more words, and determining a level of presence of each word sequence of the word sequences in each of the formed groups; and
storing at least part of the levels of presence.
