# US 2013/0159236 A1 · Data processing method and system, and relevant apparartus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0159236 A1
- Document ID: US20130159236A1
- Status: Published application
- Kind: A1
- Application number: 13/722,078
- Filed: 2012-12-20
- Published: 2013-06-20
- Earliest claimed: 2011-12-16
- Assignees: Huawei Technologies Co., Ltd.
- Inventors: Kopylov Vladislav; Liufei Wen; Guangyu Shi
- Web page: https://patentgrep.com/patent/US20130159236A1

## CPC

- G06F 16/30: of unstructured textual data (document management systems G06F16/93)
- G06N 5/02: Knowledge representation; Symbolic representation
- G06F 16/355: Creation or modification of classes or clusters
- G06F 40/30: Semantic analysis
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments of the present invention disclose a data processing method including: sending global initial statistical information to each slave node; merging received local statistical information of each slave node, to obtain new global statistical information; if Gibbs sampling performed by a slave node has ended, calculating a probability distribution between a document and topic and a probability distribution between the topic and a word according to the new global statistical information; according to the probability distributions obtained through calculation, establishing a likelihood function of a text set, and maximizing the likelihood function, to obtain a new hLDA hyper-parameter; and if iteration of solving for an hLDA hyper-parameter has converged, and according to the new hLDA hyper-parameter, calculating and outputting the probability distribution between the document and topic and the probability distribution between the topic and word.

## Claims

### Claim 1

A data processing method, comprising:
sending global initial statistical information to each slave node, wherein the global initial statistical information comprises: text subset information divided in advance according to a text set, preset initial hyper-parameter information of a hierarchical Latent Dirichlet Allocation model, a pre-established nested Chinese restaurant process prior of the text set, hierarchical topic path information of a document, document-topic count matrix information, and topic-word count matrix information;
merging received local statistical information of each slave node, to obtain new global statistical information, wherein the local statistical information comprises: a document-topic count matrix, a topic-word count matrix and a document hierarchical topic path of each slave node, and the new global statistical information comprises: global text-topic count matrix information, topic-word count matrix information, topic-word count matrix information of each slave node, and a global document hierarchical topic path;
if Gibbs sampling performed by a slave node has ended, calculating a probability distribution between the document and a topic and a probability distribution between the topic and a word according to the new global statistical information, wherein the Gibbs sampling is used to allocate a topic for each word of each document, and allocate a hierarchical topic path for each document;
according to the probability distributions obtained through calculation, establishing a likelihood function of the text set, and maximizing the likelihood function, to obtain a new hierarchical Latent Dirichlet Allocation model hyper-parameter; and
if iteration of solving for a hierarchical Latent Dirichlet Allocation model hyper-parameter has converged, and according to the new hierarchical Latent Dirichlet Allocation model hyper-parameter, calculating and outputting the probability distribution between the document and topic and the probability distribution between the topic and word.

### Claim 2 (depends on 1)

The method according to claim 1, before the sending the global initial statistical information to the slave node, the method comprises:
setting a different initial value for each hyper-parameter of the hierarchical Latent Dirichlet Allocation model;
dividing the text set into multiple text subsets, wherein the number of the text subsets is the same as the number of nodes; and
allocating one topic path for each document in the text set, allocating one topic for each word in the document, and according to the statistical total number of words in the text set, the total number of words contained in each document, and a word list of the text set, obtaining a document-topic count matrix and a topic-word count matrix.

### Claim 3 (depends on 1)

The method according to claim 1, after the merging the received local statistical information of each slave node, to obtain the new global statistical information, the method comprises:
judging whether the Gibbs sampling performed by the slave node ends; wherein
the judging whether the Gibbs sampling performed by the slave node ends comprises:
according to the number of times of iteration of the Gibbs sampling or a gradient of the likelihood function, judging whether the Gibbs sampling ends.

### Claim 4 (depends on 3)

The method according to claim 3, further comprising:
if the Gibbs sampling performed by the slave node does not end, sending the new global statistical information to the slave node.

### Claim 5 (depends on 4)

The method according to claim 4, after the according to the probability distributions obtained through calculation, establishing the likelihood function of the text set, and maximizing the likelihood function, to obtain the new hierarchical Latent Dirichlet Allocation model hyper-parameter, the method comprises:
judging whether iteration of an expectation-maximization algorithm converges; and
the judging whether the iteration of the expectation-maximization algorithm converges comprises:
when the gradient of a likelihood function value of the text set corresponding to the hierarchical Latent Dirichlet Allocation model hyper-parameter is less than a preset gradient threshold, determining that the iteration of the expectation-maximization algorithm has converged.

### Claim 6 (depends on 5)

The method according to claim 5, further comprising:
if the iteration of solving for a hierarchical Latent Dirichlet Allocation model hyper-parameter does not converge, sending the slave node the new global statistical information having a hierarchical Latent Dirichlet Allocation model hyper-parameter updated.

### Claim 7

A data processing method, comprising:
receiving global initial statistical information sent by a master node, wherein the global initial statistical information comprises: text subset information divided in advance according to a text set, preset initial hyper-parameter information of a hierarchical Latent Dirichlet Allocation model, a pre-established nested Chinese restaurant process prior of the text set, hierarchical topic path information of a document, document-topic count matrix information, and topic-word count matrix information;
according to a hierarchical topic path of each document, reallocating a topic for each word in each document through Gibbs sampling;
according to the nested Chinese restaurant process prior, and an updated document-topic count matrix and topic-word count matrix, reallocating a hierarchical topic path for each document through Gibbs sampling; and
sending local statistical information to the master node, wherein the local statistical information comprises: document-topic count matrix information and topic-word count matrix information and hierarchical topic path information of each document which are updated and are of a present slave node.

### Claim 8 (depends on 7)

The method according to claim 7, after the according to the hierarchical topic path of each document, reallocating a topic for each word in each document through Gibbs sampling, the method comprises:
updating the document-topic count matrix and topic-word count matrix information of each document having the topic reallocated for the word.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the according to the hierarchical topic path of each document, reallocating a topic for each word in each document through Gibbs sampling comprises:
allocating multiple hierarchical sub-topics for each document in the text subset, and in the multiple hierarchical sub-topics, allocating a corresponding topic for each word in the document through Gibbs sampling.

### Claim 10 (depends on 7)

The method according to claim 7, further comprising:
if new global statistical information sent by the master node is received, reallocating a hierarchical topic path for each document and reallocating a topic for each word in each document, through Gibbs sampling and according to the new global statistical information.

### Claim 11

A master node, comprising:
a sending unit, configured to send global initial statistical information to each slave node, wherein the global initial statistical information comprises: text subset information divided in advance according to a text set, preset initial hyper-parameter information of a hierarchical Latent Dirichlet Allocation model, a pre-established nested Chinese restaurant process prior of the text set, hierarchical topic path information of a document, document-topic count matrix information, and topic-word count matrix information; further configured to, if Gibbs sampling performed by a slave node does not end, send new global statistical information to the slave node; and configured to, if iteration of solving for a hierarchical Latent Dirichlet Allocation model hyper-parameter does not converge, send the slave node the new global statistical information having a hierarchical Latent Dirichlet Allocation model hyper-parameter updated;
a merging unit, configured to merge received local statistical information of each slave node, to obtain new global statistical information, wherein the local statistical information comprises: a document-topic count matrix, a topic-word count matrix and a document hierarchical topic path of each slave node, and the new global statistical information comprises: global text-topic count matrix information, topic-word count matrix information, topic-word count matrix information of each slave node, and a global document hierarchical topic path;
a calculating unit, configured to, if the Gibbs sampling performed by the slave node has ended, calculate a probability distribution between the document and a topic and a probability distribution between the topic and a word according to the new global statistical information; further configured to, according to the probability distributions obtained through calculation, establish a likelihood function of the text set, and maximize the likelihood function to obtain new hierarchical Latent Dirichlet Allocation model hyper-parameter; and configured to, if the iteration of solving for a hierarchical Latent Dirichlet Allocation model hyper-parameter converges, and according to the new hierarchical Latent Dirichlet Allocation model hyper-parameter, calculate and output the probability distribution between the document and topic and the probability distribution between the topic and word.

### Claim 12 (depends on 11)

The master node according to claim 11, further comprising:
a setting unit, configured to set a different initial value for each hyper-parameter of the hierarchical Latent Dirichlet Allocation model;
a dividing unit, configured to divide the text set into multiple text subsets, wherein the number of the text subsets is the same as the number of nodes;
an allocating unit, configured to allocate one topic path for each document in the text set, allocate one topic for each word in the document, and according to the statistical total number of words in the text set, the total number of words contained in each document, and a word list of the text set, obtaining a document-topic count matrix and a topic-word count matrix; and
a judging unit, configured to judge whether the Gibbs sampling performed by the slave node ends, wherein whether the Gibbs sampling ends is judged in a manner of judging, according to the number of times of iteration of the Gibbs sampling or a gradient of the likelihood function, whether the Gibbs sampling ends; further configured to, judge, according to an expectation-maximization algorithm, whether the iteration of solving for a hierarchical Latent Dirichlet Allocation model hyper-parameter converges, wherein whether the iteration of solving for the hierarchical Latent Dirichlet Allocation model hyper-parameter converges is judged in a manner of determining, when a gradient of a likelihood function value of the text set corresponding to the hierarchical Latent Dirichlet Allocation model hyper-parameter is less than a preset gradient threshold, that iteration of the expectation-maximization algorithm has converged.

### Claim 13 (depends on 12)

The master node according to claim 12, wherein,
the sending unit is configured to, if the Gibbs sampling performed by the slave node does not end, send the new global statistical information to the slave node, and if the iteration of solving for a hierarchical Latent Dirichlet Allocation model hyper-parameter does not converge, send the slave node the new global statistical information having a hierarchical Latent Dirichlet Allocation model hyper-parameter updated.

### Claim 14

A slave node, comprising:
an information receiving unit, configured to receive global initial statistical information sent by a master node, wherein the global initial statistical information comprises: text subset information divided in advance according to a text set, preset initial hyper-parameter information of a hierarchical Latent Dirichlet Allocation model, a pre-established nested Chinese restaurant process prior of the text set, hierarchical topic path information of a document, document-topic count matrix information, and topic-word count matrix information;
a topic allocating unit, configured to, according to a hierarchical topic path of each document, reallocate a topic for each word in each document through Gibbs sampling;
a path allocating unit, configured to, according to the nested Chinese restaurant process prior, and an updated document-topic count matrix and topic-word count matrix, reallocate a hierarchical topic path for each document through Gibbs sampling; and
an information sending unit, configured to send local statistical information to the master node, wherein the local statistical information comprises: document-topic count matrix information and topic-word count matrix information and hierarchical topic path information of each document which are updated and are of a present slave node.

### Claim 15 (depends on 14)

The slave node according to claim 14, further comprising:
an updating unit, configured to update the document-topic count matrix and topic-word count matrix information of each document having the topic reallocated for the word.

### Claim 16 (depends on 15)

The slave node according to claim 15, wherein
the topic allocating unit is configured to allocate a corresponding topic for each word in the document in the manner of allocating multiple hierarchical sub-topics for each document in the text subset, and in the multiple hierarchical sub-topics, allocating a corresponding topic for each word in the document through Gibbs sampling.

### Claim 17 (depends on 14)

The slave node according to claim 14, wherein
the path allocating unit is further configured to, if new global statistical information sent by the master node is received, reallocate a hierarchical topic path for each document through Gibbs sampling and according to the new global statistical information; and
the topic allocating unit is further configured to, if the new global statistical information sent by the master node is received, reallocate a topic for each word in each document through Gibbs sampling and according to the new global statistical information.
