# US 2025/0094828 A1 · Knowledge graph for semantic searching of handwritten documents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094828 A1
- Document ID: US20250094828A1
- Status: Published application
- Kind: A1
- Application number: 18/355,948
- Filed: 2023-07-20
- Published: 2025-03-20
- Earliest claimed: 2023-07-20
- Assignees: Wacom Co., Ltd.
- Applicants: Wacom Co., Ltd.
- Inventors: Markus WEBER; Dilyana BODOVSKA; Nikolay VASILEV
- Web page: https://patentgrep.com/patent/US20250094828A1

## CPC

- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 5/025: Extracting rules from data
- G06V 30/24: characterised by the processing or recognition method (segmentation of character regions G06V30/148)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for building a knowledge graph from handwritten documents is disclosed. The method includes receiving a handwritten document along with dynamic handwritten data from an electronic device and recognizing potential terms for objects in the handwritten document by employing a handwriting recognition technique. The potential terms for each object include a closest recognized term and at least one alternative recognized term. The method also includes determining conceptual terms from the potential recognized terms. Furthermore, the method includes determining a multi-level relation between one or more potential recognized terms and the handwritten document. Thereafter, the method includes building a knowledge graph based on the plurality of potential terms, the one or more conceptual terms, the determined multi-level relation between the one or more potential recognized terms and the handwritten document, or a combination thereof.

## Claims

### Claim 1

A system for building a knowledge graph from handwritten document, the system comprising:
a receiver module configured to receive a handwritten document along with dynamic handwritten data from an electronic device;
a recognition module configured to recognize a plurality of potential terms for one or more objects in the handwritten document by employing a handwriting recognition technique, wherein the plurality of potential terms for each of the one or more objects includes a closest recognized term and at least one alternative recognized term;
a concept building module configured to:
determine one or more conceptual terms from one or more potential recognized terms of the plurality of potential terms; and
determine a multi-level relation between one or more potential recognized terms, out of the plurality of potential terms, and the handwritten document; and
a knowledge graph building module configured to build a knowledge graph, wherein the knowledge graph is built based at least on one of: the plurality of potential terms, the one or more conceptual terms, the determined multi-level relation between the one or more potential recognized terms and the handwritten document,
wherein the knowledge graph is used to enable at least a semantic searching of one or more handwritten documents.

### Claim 2 (depends on 1)

The system of claim 1, wherein the dynamic handwritten data is received in the form of one or more tuples having at least one of: data on x-axis, data on y-axis, pressure, speed of writing, or orientation.

### Claim 3 (depends on 2)

The system of claim 2, wherein the handwriting recognition techniques analyze each of the received one or more tuples to identify the closest recognized term along with one or more alternative recognized terms that each of the received one or more tuples potentially represents.

### Claim 4 (depends on 1)

The system of claim 1, wherein the concept building module is configured to perform a named entity linking on the plurality of potential terms to determine the corresponding one or more conceptual terms.

### Claim 5 (depends on 1)

The system of claim 1, wherein the one or more conceptual terms include at least one of: terms corresponding to the recognized text in one or more languages, one or more synonym terms corresponding to the recognized text, one or more abbreviation terms corresponding to the recognized text, or one or more internally defined terms corresponding to the recognized text.

### Claim 6 (depends on 1)

The system of claim 1, wherein each of the plurality of potential terms along with the one or more corresponding conceptual terms are placed as a node in the built knowledge graph, such that one node is connected to another node based on the determined multi-level relation between the corresponding potential recognized terms and the handwritten document.

### Claim 7 (depends on 1)

The system of claim 1, wherein the knowledge graph building module is further configured to facilitate the user to set visibility of newly added nodes and their relationships in the knowledge graph.

### Claim 8 (depends on 1)

The system of claim 1, wherein the knowledge graph building module is further configured to automatically set visibility of newly added nodes and their relationships in the knowledge graph based on historical visibilities of nodes and their relationships.

### Claim 9 (depends on 8)

The system of claim 8, wherein based on the set visibility, the one or more nodes and their relationships are divided into at least one of: one or more public nodes and relationships corresponding to the documents publicly available to each user, one or more shared nodes and relationships corresponding to the documents on a subject to which the user is invited, and one or more private nodes and relationships corresponding to the documents that are specific to one user.

### Claim 10

A method for building a knowledge graph from handwritten documents, the method comprising:
receiving a handwritten document along with dynamic handwritten data from an electronic device;
recognizing a plurality of potential terms for one or more objects in the handwritten document by employing a handwriting recognition technique, wherein the plurality of potential terms for each of the one or more objects includes a closest recognized term and at least one alternative recognized term;
determining one or more conceptual terms from one or more potential recognized terms of the plurality of potential terms;
determining a multi-level relation between one or more potential recognized terms, out of the plurality of potential terms, and the handwritten document; and
building a knowledge graph, wherein the knowledge graph is built based at least on one of: the plurality of potential terms, the one or more conceptual terms, and the determined multi-level relation between the one or more potential recognized terms and the handwritten document,
wherein the knowledge graph is used to enable at least a semantic searching of the one or more handwritten documents.

### Claim 11 (depends on 10)

The method of claim 10, wherein the handwriting recognition techniques analyze each of received one or more tuples to identify the closest recognized term along with one or more alternative recognized terms that each of the received one or more tuples potentially represents.

### Claim 12 (depends on 10)

The method of claim 10, which further comprises performing a named entity linking on the plurality of potential terms to determine the corresponding one or more conceptual terms.

### Claim 13 (depends on 10)

The method of claim 10, wherein each of the plurality of potential terms along with the one or more corresponding conceptual terms are placed as a node in the built knowledge graph, such that one node is connected to another node based on the determined multi-level relation between the corresponding potential recognized terms and the handwritten document.

### Claim 14 (depends on 10)

The method of claim 10, which further comprises facilitating the user to set visibility of the one or more newly added nodes and their relationships in the knowledge graph.

### Claim 15 (depends on 10)

The method of claim 10, which further comprises automatically setting visibility of the one or more newly added nodes and their relationships in the knowledge graph based on historical visibilities of nodes and their relationships.

### Claim 16

A semantic searching system for searching handwritten documents using a knowledge graph, the semantic searching system comprising:
a receiver module configured to receive, from an electronic device, text data having one or more terms associated with a user's intended search;
an entity recognition module configured to perform entity recognition from the text data to determine one or more entities present in the text data;
a concept determination module configured to determine one or more conceptual terms for each of the determined one or more entities via a named entity linking;
an activation graph creation module configured to create an activation graph based on the determined one or more conceptual terms by adding nodes and their relationships corresponding to at least of: terms corresponding to the recognized entity in one or more languages, one or more synonym terms corresponding to the recognized entity, one or more abbreviation terms corresponding to the recognized entity, or one or more internally defined terms corresponding to the recognized entity;
an associative searching module configured to perform an associated searching for obtaining one or more search results based on matching of the one or more nodes of the activation graph with one or more nodes of the knowledge graph; and
a rendering module configured to render ranked and selected search results to the user,
wherein the one or more ranked and selected search results include at least one of: shortcuts to open a handwritten document associated with the search results, or online links associated with the search results.

### Claim 17 (depends on 16)

The semantic searching system of claim 16, wherein the associative searching module selects the search results based on an accessibility level of the user and visibility level of the one or more nodes and their relationships.

### Claim 18 (depends on 16)

The semantic searching system of claim 16, wherein the visibility level of the one or more nodes and their relationships is at least one of: automatically defined based on historical visibilities of nodes and their relationships, or manually defined based on user inputs in a documents database.

### Claim 19 (depends on 18)

The semantic searching system of claim 18, wherein based on the pre-defined visibility, the documents database includes at least: one or more public documents corresponding to the documents publicly available to each user, one or more shared documents corresponding to the documents on a subject to which the user is invited, and one or more private documents corresponding to the documents that are specific to one user.

### Claim 20 (depends on 16)

The semantic searching system of claim 16, which further comprises a direct searching module configured to:
pre-process the received text data by performing at least one of: tokenization, removal of stop words, removal of punctuation marks, or removal of spaces; and
perform a direct searching by matching the pre-processed received text data with one or more nodes of the comprehensive knowledge graph for obtaining the one or more search results.
