# US 12,182,516 B2 · Computer-implemented method, device, and computer program product > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,516 B2 - Document ID: US12182516B2 - Status: Granted - Kind: B2 - Application number: 17/527,798 - Filed: 2021-11-16 - Published: 2023-04-27 - Granted: 2024-12-31 - Earliest claimed: 2021-10-21 - Assignees: EMC IP Holding Company LLC - Applicants: EMC IP Holding Company LLC - Inventors: Zijia Wang; Jiacheng Ni; Zhen Jia - Web page: https://patentgrep.com/patent/US12182516B2 ## CPC - G06F 40/30: Semantic analysis - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 5/02: Knowledge representation; Symbolic representation - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Embodiments of the present disclosure relate to a computer-implemented method, a device, and a computer program product. The method includes extracting respective themes of a set of documents with release time within a first period; determining respective semantic information of the themes and frequencies of the themes appearing in the set of documents; and determining the number of documents associated with the themes within a second period according to a prediction model and based on the semantic information and frequencies of the themes. The second period is after the first period. Embodiments of the present disclosure can better predict the tendency of the themes appearing in the future based on the semantic information and frequencies of the themes. ## Claims ### Claim 1 A computer-implemented method, comprising: extracting respective themes of a set of documents with release time within a first period; determining respective semantic information of the themes and frequencies of the themes appearing in the set of documents utilizing at least a first machine learning model, the first machine learning model comprising a transformer-based model; and determining the number of documents associated with the themes within a second period according to a second machine learning model, different than the first machine learning model, the second machine learning model comprising a time sequence prediction model and based on the semantic information and frequencies of the themes, wherein the second period is after the first period; wherein the first machine learning model is configured to encode the semantic information of the themes and the frequencies of the themes appearing in the set of documents; wherein results of the encoding of the semantic information of the themes and the frequencies of the themes comprises, for each of at least a subset of the themes, (i) a semantic encoding component comprising at least a classification token for the semantic information of the theme, and (ii) a frequency encoding component comprising a plurality of frequency-related values generated for the frequency of appearance of the theme based on a designated probability function; wherein the second machine learning model is configured to process the semantic encoding components and the frequency encoding components to determine the number of documents associated with the themes within the second period; wherein determining the frequencies comprises determining a time sequence of frequency representations of the themes within the first period; wherein determining the time sequence of frequency representations comprises: for a time interval point within the first period, determining a frequency representation of the time sequence of frequency representations at the time interval point based on the number of documents corresponding to the themes; and wherein determining the frequency representation at the time interval point comprises: determining the frequency representation by using a position extending code based on the number of documents corresponding to the themes. ### Claim 2 (depends on 1) The method according to claim 1, wherein determining the semantic information comprises: determining a time sequence of semantic representations of the themes within the first period. ### Claim 3 (depends on 2) The method according to claim 2, wherein determining the time sequence of semantic representations comprises: for a time interval point within the first period, determining a semantic representation of the time sequence of semantic representations at the time interval point according to a semantic encoding model and based on words or words in phrases corresponding to the themes in documents with release time not later than the time interval point in the set of documents. ### Claim 4 (depends on 1) The method according to claim 1, wherein determining the frequency representation of the time sequence of frequency representations at the time interval point is based on the number of documents corresponding to the themes in documents with release time not later than the time interval point in the set of documents. ### Claim 5 (depends on 1) The method according to claim 1, wherein the frequency representation has a predefined dimension which is greater than one dimension. ### Claim 6 (depends on 1) The method according to claim 1, wherein extracting the respective themes of the set of documents comprises: extracting a predefined number of respective themes of the set of documents by using a theme classifying model. ### Claim 7 (depends on 1) The method according to claim 1, wherein determining the number of the documents associated with the themes within the second period comprises: determining a number time sequence of the themes within the second period, wherein the number time sequence comprises the number of documents associated with the themes at each time interval point within the second period. ### Claim 8 An electronic device, comprising: at least one processor; and at least one memory storing computer program instructions, wherein the computer program instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising: extracting respective themes of a set of documents with release time within a first period; determining respective semantic information of the themes and frequencies of the themes appearing in the set of documents utilizing at least a first machine learning model, the first machine learning model comprising a transformer-based model; and determining the number of documents associated with the themes within a second period according to a second machine learning model, different than the first machine learning model, the second machine learning model comprising a time sequence prediction model and based on the semantic information and frequencies of the themes, wherein the second period is after the first period; wherein the first machine learning model is configured to encode the semantic information of the themes and the frequencies of the themes appearing in the set of documents; wherein results of the encoding of the semantic information of the themes and the frequencies of the themes comprises, for each of at least a subset of the themes, (i) a semantic encoding component comprising at least a classification token for the semantic information of the theme, and (ii) a frequency encoding component comprising a plurality of frequency-related values generated for the frequency of appearance of the theme based on a designated probability function; wherein the second machine learning model is configured to process the semantic encoding components and the frequency encoding components to determine the number of documents associated with the themes within the second period; wherein determining the frequencies comprises determining a time sequence of frequency representations of the themes within the first period; wherein determining the time sequence of frequency representations comprises: for a time interval point within the first period, determining a frequency representation of the time sequence of frequency representations at the time interval point based on the number of documents corresponding to the themes; and wherein determining the frequency representation at the time interval point comprises: determining the frequency representation by using a position extending code based on the number of documents corresponding to the themes. ### Claim 9 (depends on 8) The electronic device according to claim 8, wherein determining the semantic information comprises: determining a time sequence of semantic representations of the themes within the first period. ### Claim 10 (depends on 9) The electronic device according to claim 9, wherein determining the time sequence of semantic representations comprises: for a time interval point within the first period, determining a semantic representation of the time sequence of semantic representations at the time interval point according to a semantic encoding model and based on words or words in phrases corresponding to the themes in documents with release time not later than the time interval point in the set of documents. ### Claim 11 (depends on 8) The electronic device according to claim 8, wherein determining the frequency representation of the time sequence of frequency representations at the time interval point is based on the number of documents corresponding to the themes in documents with release time not later than the time interval point in the set of documents. ### Claim 12 (depends on 8) The electronic device according to claim 8, wherein the frequency representation has a predefined dimension which is greater than one dimension. ### Claim 13 (depends on 8) The electronic device according to claim 8, wherein extracting the respective themes of the set of documents comprises: extracting a predefined number of respective themes of the set of documents by using a theme classifying model. ### Claim 14 (depends on 8) The electronic device according to claim 8, wherein determining the number of the documents associated with the themes within the second period comprises: determining a number time sequence of the themes within the second period, wherein the number time sequence comprises the number of documents associated with the themes at each time interval point within the second period. ### Claim 15 A computer program product tangibly stored in a non-transitory computer-readable storage medium and including machine-executable instructions, wherein the machine-executable instructions, when executed, cause a device to execute a method, the method comprising: extracting respective themes of a set of documents with release time within a first period; determining respective semantic information of the themes and frequencies of the themes appearing in the set of documents utilizing at least a first machine learning model, the first machine learning model comprising a transformer-based model; and determining the number of documents associated with the themes within a second period according to a second machine learning model, different than the first machine learning model, the second machine learning model comprising a time sequence prediction model and based on the semantic information and frequencies of the themes, wherein the second period is after the first period; wherein the first machine learning model is configured to encode the semantic information of the themes and the frequencies of the themes appearing in the set of documents; wherein results of the encoding of the semantic information of the themes and the frequencies of the themes comprises, for each of at least a subset of the themes, (i) a semantic encoding component comprising at least a classification token for the semantic information of the theme, and (ii) a frequency encoding component comprising a plurality of frequency-related values generated for the frequency of appearance of the theme based on a designated probability function; wherein the second machine learning model is configured to process the semantic encoding components and the frequency encoding components to determine the number of documents associated with the themes within the second period; wherein determining the frequencies comprises determining a time sequence of frequency representations of the themes within the first period; wherein determining the time sequence of frequency representations comprises: for a time interval point within the first period, determining a frequency representation of the time sequence of frequency representations at the time interval point based on the number of documents corresponding to the themes; and wherein determining the frequency representation at the time interval point comprises: determining the frequency representation by using a position extending code based on the number of documents corresponding to the themes. ### Claim 16 (depends on 15) The computer program product according to claim 15, wherein determining the semantic information comprises: determining a time sequence of semantic representations of the themes within the first period. ### Claim 17 (depends on 16) The computer program product according to claim 16, wherein determining the time sequence of semantic representations comprises: for a time interval point within the first period, determining a semantic representation of the time sequence of semantic representations at the time interval point according to a semantic encoding model and based on words or words in phrases corresponding to the themes in documents with release time not later than the time interval point in the set of documents. ### Claim 18 (depends on 15) The computer program product according to claim 15, wherein determining the frequency representation of the time sequence of frequency representations at the time interval point is based on the number of documents corresponding to the themes in documents with release time not later than the time interval point in the set of documents. ### Claim 19 (depends on 15) The computer program product according to claim 15, wherein the frequency representation has a predefined dimension which is greater than one dimension. ### Claim 20 (depends on 15) The computer program product according to claim 15, wherein determining the number of the documents associated with the themes within the second period comprises: determining a number time sequence of the themes within the second period, wherein the number time sequence comprises the number of documents associated with the themes at each time interval point within the second period.