Back to results
US 10,380,240 B2Granted
Apparatus and method for data compression extension
AssigneeFujitsu Limited
InventorsMasao Ideuchi, Masahiro Kataoka, Kazuo Mineno
Life of this patent
Estimating the term…
Abstract
An information processing apparatus encodes text data by using a word dictionary that allocates each of a plurality of words in a word group to a corresponding code and generates a count map indicating an appearance frequency of the plurality of words in the word group in the text data. The information processing apparatus associates the count map with encoded text data and stores it to a storage unit.
Claims5
A non-transitory computer-readable recording medium having stored therein a data management program that causes a computer to execute a process comprising:
- encoding text data by using an encoding dictionary that allocates each of a plurality of words in a word group to a corresponding code to generate an encoded file;
- simultaneously with the encoding of the text data, generating frequency information indicating an appearance frequency of the plurality of words in the word group in the text data;
- simultaneously with the encoding of the text data, storing the frequency information to a storage unit associating the frequency information with the encoded text data;
- storing the generated encoded file in a file management system;
- causing applications including MapReduce to calculate, in response to an instruction, a similarity between documents included in the encoded file using the frequency information stored in the storage unit without converting the encoded text data into a non-encoded state;
- and
- performing syntax analysis and term frequency/inverse document frequency (tf/idf) in text mining without extension of the encoded file and without a repeated execution of lexical analysis and frequency count.
4 more elements
An information processing apparatus, comprising:
- a processor, wherein the processor executes:
- encoding text data by using an encoding dictionary that allocates each of a plurality of words in a word group to a corresponding code to generate an encoded file;
- simultaneously with the encoding of the text data, generating frequency information indicating an appearance frequency of the plurality of words in the word group in the text data;
- simultaneously with the encoding of the text data, storing the frequency information to a storage unit associating the frequency information with the encoded text data;
- storing the generated encoded file in a file management system;
- causing applications including MapReduce to calculate, in response to an instruction, a similarity between documents included in the encoded file using the frequency information stored in the storage unit without converting the encoded text data into a non-encoded state;
- and
- performing syntax analysis and term frequency/inverse document frequency (tf/idf) in text mining without extension of the encoded file and without a repeated execution of lexical analysis and frequency count.
5 more elements
A data management method executed by a computer, the data management method comprising:
- encoding, by a processor, text data by using an encoding dictionary that allocates each of a plurality of words in a word group to a corresponding code to generate an encoded file;
- simultaneously with the encoding of the text data, generating, by the processor, frequency information indicating an appearance frequency of the plurality of words in the word group in the text data;
- simultaneously with the encoding of the text data, storing the frequency information to a storage unit associating the frequency information with the encoded text data;
- storing the generated encoded file in a file management system;
- causing applications including MapReduce to calculate, in response to an instruction, a similarity between documents included in the encoded file using the frequency information stored in the storage unit without converting the encoded text data into a non-encoded state;
- and
- performing syntax analysis and term frequency/inverse document frequency (tf/idf) in text mining without extension of the encoded file and without a repeated execution of lexical analysis and frequency count.
4 more elements
Dependencies are inferred from the claim text.