Method, apparatus, device, and storage medium for learning knowledge representation

An electronic device, comprising:

• one or more processors;
• and
• a storage apparatus, storing one or more programs,
• wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
• sampling a sub-graph of a knowledge graph from a knowledge base;
• serializing the sub-graph of the knowledge graph to obtain a serialized text;
• and
• reading, using a pre-trained language model, the serialized text in an order in the sub-graph of the knowledge graph, to perform learning to obtain a knowledge representation of each word in the serialized text by:
• generating a mask matrix and a hierarchical code corresponding to the serialized text, based on the serialized text and the sub-graph of the knowledge graph;
• and
• inputting the serialized text into the pre-trained language model, and reading the serialized text using the pre-trained language model based on the mask matrix and the hierarchical code corresponding to the serialized text, to perform learning to obtain the knowledge representation of each word in the serialized text.

A non-transitory computer readable medium, storing a computer program thereon, wherein the program, when executed by a processor, causes the processor to perform operations, the operations comprising:

• sampling a sub-graph of a knowledge graph from a knowledge base;
• serializing the sub-graph of the knowledge graph to obtain a serialized text;
• and
• reading, using a pre-trained language model, the serialized text in an order in the sub-graph of the knowledge graph, to perform learning to obtain a knowledge representation of each word in the serialized text by:
• generating a mask matrix and a hierarchical code corresponding to the serialized text, based on the serialized text and the sub-graph of the knowledge graph;
• and
• inputting the serialized text into the pre-trained language model, and reading the serialized text using the pre-trained language model based on the mask matrix and the hierarchical code corresponding to the serialized text, to perform learning to obtain the knowledge representation of each word in the serialized text.