Trending topic discovery with keyword-based topic model

A method for topic discovery with a keyword-based topic model, the method comprising:

• obtaining, by a device comprising a memory storing instructions and a processor in communication with the memory, text data,
• conducting, by the device, pre-processing on the text data to obtain preprocessed text data,
• extracting, by the device, an entity list and a keyword list based on the preprocessed text data,
• generating, by the device, an entity embedding list based on the entity list,
• wherein the generating the entity embedding list based on the entity list comprises:
• generating the entity embedding list with an embedding model based on the entity list according to semantic meaning, wherein each member of the entity embedding list is an embedding vector in a k-dimension embedding space, and k is a positive integer;
• clusterizing, by the device, the entity list based on the entity embedding list to obtain a plurality of entity clusters, each entity cluster comprising at least one entity,
• wherein clusterizing the entity list based on the entity embedding list to obtain the plurality of entity clusters comprises:
• generating a plurality of first entity clusters based on the entity embedding list with a first layer clustering model and a first similarity,
• generating a plurality of second entity clusters based on a largest cluster in the plurality of the first entity clusters with a second layer clustering model and a second similarity,
• combining a first subset of the plurality of the first entity clusters and a second subset of the plurality of the second entity clusters to obtain the plurality of entity clusters,
• wherein the first layer clustering model and the second layer clustering model are based on a clustering algorithm of density-based spatial clustering of applications with noise (DBSCAN), and
• wherein the first similarity is smaller than the second similarity;
• retrieving, by the device, a co-occurring keyword list based on the plurality of entity clusters, the entity list, and the keyword list, and
• obtaining, by the device, a topic for each entity cluster of the plurality of entity clusters based on the co-occurring keyword list.