Method and system to optimize a plurality of topics by evaluation metrics

A system (100), comprising:

• a memory (104) storing instructions;
• one or more communication interfaces (106);
• and
• one or more hardware processors (102) coupled to the memory (104) via the one or more communication interfaces (106), wherein the one or more hardware processors (102) are configured by the instructions to:
• receive, dataset obtained from a raw dataset as an input, wherein the processed dataset corresponds to at least one topic model, wherein the at least one topic model comprises a plurality of topics (k);
• map, the raw dataset, and the processed dataset to obtain a relationship between at least one token and at least one topic from the plurality of topics (k), wherein the at least one token corresponds to at least one word (n) present in at least one sentence from the raw dataset;
• generate, at least one construct in the at least one sentence from the raw dataset, wherein the at least one construct corresponds to at least one closely connected entity;
• determine, a word score based on a probability of a number of constructs, wherein the number of constructs comprising a plurality of paired tokens;
• determine, a litany index value for each topic of the plurality of topics (k), wherein the litany index value corresponds to at least one repetitive token in each topic of the plurality of topics (k);
• determine, a topic score for each topic based on a mean value of the word score subtracted by the corresponding litany index value for each topic, wherein the topic score corresponds to a degree of contextual association between the plurality of tokens in each topic;
• and
• obtain, at least one connected topic from the plurality of topics (k) based on a contiguity score.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors causes:

• receiving, a processed dataset obtained from a raw dataset as an input, wherein the processed dataset corresponds to at least one topic model, wherein the at least one topic model comprises a plurality of topics (k);
• mapping, the raw dataset, and the processed dataset to obtain a relationship between at least one token and at least one topic from the plurality of topics (k), wherein the at least one token corresponds to at least one word (n) present in at least one sentence from the raw dataset;
• generating, by a parsing unit, at least one construct in the at least one sentence from the raw dataset, wherein the at least one construct corresponds to at least one closely connected entity;
• determining, a word score based on a probability of a number of constructs, wherein the number of constructs comprising a plurality of paired tokens;
• determining, a litany index value for each topic of the plurality of topics (k), wherein the litany index value corresponds to at least one repetitive token in each topic of the plurality of topics (k);
• determining, a topic score for each topic based on a mean value of the word score subtracted by the corresponding litany index value for each topic, wherein the topic score corresponds to a degree of contextual association between the plurality of tokens in each topic;
• and
• obtaining, at least one connected topic from the plurality of topics (k) based on a contiguity score.