# US 12,223,278 B2 · Automatic data card generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,278 B2
- Document ID: US12223278B2
- Status: Granted
- Kind: B2
- Application number: 17/860,912
- Filed: 2022-07-08
- Published: 2024-01-11
- Granted: 2025-02-11
- Earliest claimed: 2022-07-08
- Assignees: Sap SE
- Applicants: Sap SE
- Inventors: Hans-Martin Ramsl
- Web page: https://patentgrep.com/patent/US12223278B2

## CPC

- G06F 16/285: {Clustering or classification}
- G06F 40/30: Semantic analysis
- G06F 16/2458: Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F 16/31: Indexing; Data structures therefor; Storage structures
- G06F 16/353: into predefined classes
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Example methods and systems are directed to automatic data card generation for datasets. A data card is a summary that describes quantitative aspects of a dataset, qualitative aspects of a dataset, or both. The data samples and documentation of a dataset are analyzed automatically to determine a number of samples, a primary data type, a license, or any suitable combination thereof. Data formats for data and documentation of the dataset may be automatically recognized. Language of text data may be automatically recognized. The most frequent language for the text data may be identified as the primary language of the dataset. A data card may be created for the dataset. The data card may indicate the number of samples, the data formats used in the data set, the language of text data in the dataset, or any suitable combination thereof.

## Claims

### Claim 1

A method comprising:
analyzing, by one or more processors, a plurality of files of a dataset to determine a primary data type of the dataset;
determining, by the one or more processors, one or more statistical values for the dataset;
determining, by the one or more processors, a count of entries in the dataset;
accessing, by the one or more processors, a data value representing a stored count of the entries;
based on comparing, by the one or more processors, the determined count of the entries with the data value, storing the data value in a data structure;
determining, by the one or more processors, a percentage of the dataset that comprises text in each of a plurality of languages, by performing operations comprising:
splitting each of the plurality of files of the dataset into lines;
for each line, determining a language; and
determining a percentage of lines in each of the plurality of languages;
based on the determined percentages, determining a primary language of the dataset;
storing, in the data structure, an indication of the primary data type and at least a subset of the one or more statistical values and an indication of the primary language;
determining, by the one or more processors, a first subset of the dataset to be used for training;
determining, by the one or more processors, a second subset of the dataset to be used for validation;
storing, in the data structure, an indication of the first subset and an indication of the second subset;
iteratively generating additional data structures for a set of additional datasets;
receiving, via a user interface, user input to select a set of search criteria, wherein the set
of search criteria comprises the primary data type, the count of entries, and the primary language;
based on the set of search criteria and the data structure, displaying information for the dataset in the user interface, the displayed information including the primary data type, the count of entries, and the primary language; and
based on a selection of the dataset by the user, training a neural network using the first subset of the dataset.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining, based on one or more documentation files for the dataset, a license that governs use of the dataset; and
storing, in the data structure, an indication of the license.

### Claim 3 (depends on 1)

The method of claim 1, wherein the primary data type is at least one of an image, text, audio, and video.

### Claim 4 (depends on 1)

The method of claim 1, wherein the analyzing of the plurality of files to determine the primary data type comprises:
determining a file type for each of the plurality of files; and
for each file type, determining a total size for the files of the file type.

### Claim 5 (depends on 1)

The method of claim 1, wherein the determining of the first subset is based on contents of a first file and the determining of the second subset is based on contents of a second file.

### Claim 6 (depends on 1)

The method of claim 1, wherein the determining of the first subset and the second subset are based on searching for one or more keywords in one or more documentation files for the dataset.

### Claim 7

A system comprising:
a memory that stores instructions; and
one or more processors configured by the instructions to perform operations comprising:
analyzing a plurality of files of a dataset to determine a primary data type of the dataset;
determining one or more statistical values for the dataset;
determining a count of entries in the dataset;
accessing a data value representing a stored count of the entries:
based on comparing the determined count of the entries with the data value, storing the data value in a data structure;
determining a percentage of the dataset that comprises text in each of a plurality of languages, by performing operations comprising:
splitting each of the plurality of files of the dataset into lines;
for each line, determining a language; and
determining a percentage of lines in each of the plurality of languages;
based on the determined percentages, determining a primary language of the dataset;
storing, in the data structure, an indication of the primary data type and at least a subset of the one or more statistical values and an indication of the primary language;
determining, by the one or more processors, a first subset of the dataset to be used for training;
determining, by the one or more processors, a second subset of the dataset to be used for validation;
storing, in the data structure, an indication of the first subset and an indication of the second subset;
iteratively generating additional data structures for a set of additional datasets;
receiving, via a user interface, user input to select a set of search criteria, wherein the set of search criteria comprises the primary data type, the count of entries, and the primary language;
based on the set of search criteria and the data structure, displaying information for the dataset in the user interface, the displayed information including the primary data type, the count of entries, and the primary language; and
based on a selection of the dataset by the user, training a neural network using the first subset of the dataset.

### Claim 8 (depends on 7)

The system of claim 7, wherein the operations further comprise:
determining, based on one or more documentation files for the dataset, a license that governs use of the dataset; and
storing, in the data structure, an indication of the license.

### Claim 9 (depends on 7)

The system of claim 7, wherein the primary data type is at least one of an image, text, audio, and video.

### Claim 10 (depends on 7)

The system of claim 7, wherein the analyzing of the plurality of files to determine the primary data type comprises:
determining a file type for each of the plurality of files; and
for each file type, determining a total size for the files of the file type.

### Claim 11 (depends on 7)

The system of claim 7, wherein the determining of the first subset is based on contents of a first file and the determining of the second subset is based on contents of a second file.

### Claim 12 (depends on 7)

The system of claim 7, wherein the determining of the first subset and the second subset are based on searching for one or more keywords in one or more documentation files for the dataset.

### Claim 13

A non-transitory computer-readable medium that stores instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
analyzing a plurality of files of a dataset to determine a primary data type of the dataset;
determining one or more statistical values for the dataset;
determining a count of entries in the dataset;
accessing a data value representing a stored count of the entries;
based on comparing the determined count of the entries with the data value, storing the data value in a data structure;
determining a percentage of the dataset that comprises text in each of a plurality of languages, by performing operations comprising:
splitting each of the plurality of files of the dataset into lines;
for each line, determining a language; and
determining a percentage of lines in each of the plurality of languages;
based on the determined percentages, determining a primary language of the dataset;
storing, in the data structure, an indication of the primary data type and at least a subset of the one or more statistical values and an indication of the primary language;
determining, by the one or more processors, a first subset of the dataset to be used for training;
determining, by the one or more processors, a second subset of the dataset to be used for validation;
storing, in the data structure, an indication of the first subset and an indication of the second subset;
iteratively generating additional data structures for a set of additional datasets;
receiving, via a user interface, user input to select a set of search criteria, wherein the set of search criteria comprises the primary data type, the count of entries, and the primary language;
based on the set of search criteria and the data structure, displaying information for the dataset in the user interface, the displayed information including the primary data type, the count of entries, and the primary language; and
based on a selection of the dataset by the user, training a neural network using the first subset of the dataset.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the operations further comprise:
determining, based on one or more documentation files for the dataset, a license that governs use of the dataset; and
storing, in the data structure, an indication of the license.

### Claim 15 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the primary data type is at least one of an image, text, audio, and video.

### Claim 16 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the analyzing of the plurality of files to determine the primary data type comprises:
determining a file type for each of the plurality of files; and
for each file type, determining a total size for the files of the file type.

### Claim 17 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the determining of the first subset is based on contents of a first file and the determining of the second subset is based on contents of a second file.