com.hankcs.hanlp.mining.word2vec

Class CacheCorpus

java.lang.Object

com.hankcs.hanlp.mining.word2vec.Corpus

com.hankcs.hanlp.mining.word2vec.CacheCorpus

public class CacheCorpus
extends Corpus

load corpus from disk cache

Author:

hankcs

Field Summary

Fields inherited from class com.hankcs.hanlp.mining.word2vec.Corpus

cacheFile, config, encoding, eoc, table, trainWords, vocab, vocabIndexMap, vocabMaxSize, vocabSize

Constructor Summary

Constructors

Constructor and Description
CacheCorpus(Corpus cloneSrc)

Method Summary

Modifier and Type	Method and Description
String	nextWord() Read the next word from the corpus
int	readWordIndex()
void	rewind(int numThreads, int id) reset current corpus to initial status

Methods inherited from class com.hankcs.hanlp.mining.word2vec.Corpus

addWordToVocab, close, endOfCorpus, getTrainWords, getVocab, getVocabIndexMap, getVocabSize, shutdown

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CacheCorpus

public CacheCorpus(Corpus cloneSrc)
            throws IOException

Throws:

IOException

Method Detail

nextWord

public String nextWord()
                throws IOException

Description copied from class: Corpus

Read the next word from the corpus

Specified by:

nextWord in class Corpus

Returns:

next word that is read from the corpus. null will be returned

Throws:

IOException

readWordIndex

public int readWordIndex()
                  throws IOException

Overrides:

readWordIndex in class Corpus

Returns:

-4 if is dropped, -3 if end of sentence, -2 if end of corpus, -1 if word not found or index value of the word

Throws:

IOException

rewind

public void rewind(int numThreads,
                   int id)
            throws IOException

Description copied from class: Corpus

reset current corpus to initial status

Overrides:

rewind in class Corpus

Throws:

IOException