org.deeplearning4j.spark.models.embeddings.word2vec

Class FirstIterationFunction

java.lang.Object

org.deeplearning4j.spark.models.embeddings.word2vec.FirstIterationFunction

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.FlatMapFunction<Iterator<scala.Tuple2<List<VocabWord>,Long>>,Map.Entry<VocabWord,org.nd4j.linalg.api.ndarray.INDArray>>

public class FirstIterationFunction
extends Object
implements org.apache.spark.api.java.function.FlatMapFunction<Iterator<scala.Tuple2<List<VocabWord>,Long>>,Map.Entry<VocabWord,org.nd4j.linalg.api.ndarray.INDArray>>

Author:

jeffreytang, raver119@gmail.com

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FirstIterationFunction(org.apache.spark.broadcast.Broadcast<Map<String,Object>> word2vecVarMapBroadcast, org.apache.spark.broadcast.Broadcast<double[]> expTableBroadcast, org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast)

Method Summary

Modifier and Type	Method and Description
Iterator<Map.Entry<VocabWord,org.nd4j.linalg.api.ndarray.INDArray>>	call(Iterator<scala.Tuple2<List<VocabWord>,Long>> pairIter)
void	iterateSample(VocabWord w1, VocabWord w2, double currentSentenceAlpha)
void	skipGram(int ithWordInSentence, List<VocabWord> vocabWordsList, int b, double currentSentenceAlpha)
void	trainSentence(List<VocabWord> vocabWordsList, double currentSentenceAlpha)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FirstIterationFunction

public FirstIterationFunction(org.apache.spark.broadcast.Broadcast<Map<String,Object>> word2vecVarMapBroadcast,
                              org.apache.spark.broadcast.Broadcast<double[]> expTableBroadcast,
                              org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast)

Method Detail

call

public Iterator<Map.Entry<VocabWord,org.nd4j.linalg.api.ndarray.INDArray>> call(Iterator<scala.Tuple2<List<VocabWord>,Long>> pairIter)

Specified by:

call in interface org.apache.spark.api.java.function.FlatMapFunction<Iterator<scala.Tuple2<List<VocabWord>,Long>>,Map.Entry<VocabWord,org.nd4j.linalg.api.ndarray.INDArray>>

trainSentence

public void trainSentence(List<VocabWord> vocabWordsList,
                          double currentSentenceAlpha)

skipGram

public void skipGram(int ithWordInSentence,
                     List<VocabWord> vocabWordsList,
                     int b,
                     double currentSentenceAlpha)

iterateSample

public void iterateSample(VocabWord w1,
                          VocabWord w2,
                          double currentSentenceAlpha)