Knowledge grap pre-training method based on structural context infor

A knowledge graph pre-training method based on structural context information, wherein a pre-training model comprising a triple integration module, a structural information module and a general task module is adopted to train triples in the knowledge graph, and the specific training process thereof comprises:

• for a target triple, constructing an instance comprising context triples from the knowledge graph, and adopting the triple integration module to encode each of the context triples in the instance to obtain an integration vector;
• combining integration vectors for all context triples in the instance into a context vector sequence, and adopting the structural information module to encode the context vector sequence to obtain a structural representation vector for the triple;
• adopting the general task module to calculate the structural representation vector for the triple, and obtaining a label prediction value for the triple, and updating parameters of the triple integration module, parameters of the structural information module, parameters of the general task module, and the structural representation vector for the triple based on cross-entropy loss of the label prediction value for the triple and a label truth value for the triple until the completion of the training, so as to obtain an optimized structural representation vector for the target triple;
• assigning a triple tag [TRI] to each of the context triples, and combining the triple tag representation k[TRI] and a head entity representation h′, a tail entity representation r′, and a relation representation t′ of the respective context triple into a representation sequence <k[TRI]h′,r′,t′> as an input of the triple integration module, which is calculated by the triple integration module and an output corresponding to the triple tag k[TRI] is the integration vector;
• representing the context vector sequence as <seqh, seqr, seqt>, where, seqh, seqr, seqt are the sequences for the context triples of a head entity h, a relation r, and a tail entity t respectively, and the specific form is:
• seqh=<chi,i∈[1,n]>
• seqr=<cri,i∈[1,n]>
• seqt=<cti,i ∈[1,n]>
• wherein, chi represents an i integration vector of the head entity h, cri i represents an i integration vector of the relation r, and cti represents an i integration vector of the tail entity t.
• assigning a head entity tag [HEA], a relation tag [REL], a tail entity tag [TAI] for seqh, seqr, seqt, and combining the seqh, seqr, seqt and a head entity tag vector k[HEA], a relation tag vector k[REL], and a tail entity tag vector k[TAI] into a sequence <k[HEA], seqhk[REL], seqr,k[TAI], seqt> as an input of the structural information module.