com.hankcs.hanlp.mining.word2vec

Class TextFileCorpus

java.lang.Object

com.hankcs.hanlp.mining.word2vec.Corpus

com.hankcs.hanlp.mining.word2vec.TextFileCorpus

public class TextFileCorpus
extends Corpus

Field Summary

Fields inherited from class com.hankcs.hanlp.mining.word2vec.Corpus

cacheFile, config, encoding, eoc, table, trainWords, vocab, vocabIndexMap, vocabMaxSize, vocabSize

Constructor Summary

Constructors

Constructor and Description
TextFileCorpus(Config config)

Method Summary

Modifier and Type	Method and Description
void	learnVocab()
String	nextWord() Read the next word from the corpus
void	rewind(int numThreads, int id) reset current corpus to initial status
void	shutdown()

Methods inherited from class com.hankcs.hanlp.mining.word2vec.Corpus

addWordToVocab, close, endOfCorpus, getTrainWords, getVocab, getVocabIndexMap, getVocabSize, readWordIndex

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextFileCorpus

public TextFileCorpus(Config config)
               throws IOException

Throws:

IOException

Method Detail

shutdown

public void shutdown()
              throws IOException

Overrides:

shutdown in class Corpus

Throws:

IOException

rewind

public void rewind(int numThreads,
                   int id)
            throws IOException

Description copied from class: Corpus

reset current corpus to initial status

Overrides:

rewind in class Corpus

Throws:

IOException

nextWord

public String nextWord()
                throws IOException

Description copied from class: Corpus

Read the next word from the corpus

Specified by:

nextWord in class Corpus

Returns:

next word that is read from the corpus. null will be returned

Throws:

IOException

learnVocab

public void learnVocab()
                throws IOException

Throws:

IOException