com.wcohen.ss.abbvGapsHmm

Class AbbvGapsHMM

java.lang.Object

com.wcohen.ss.abbvGapsHmm.AbbvGapsHMM

public class AbbvGapsHMM
extends Object

Author:

Dana Movshovitz-Attias

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AbbvGapsHMM.Emissions
static class	AbbvGapsHMM.States
static class	AbbvGapsHMM.Transitions

Constructor Summary

Constructors

Constructor and Description
AbbvGapsHMM()
AbbvGapsHMM(String modelParamFile)
AbbvGapsHMM(String modelParamFile, boolean allowVowelsMatch)

Method Summary

Modifier and Type	Method and Description
protected void	expectationStep(Acronym acronym, String trueLongForm)
Double	getDF(String word)
List<Double>	getEmmisionParams()
protected double	getNewStateVal(double current, double total)
List<Double>	getTransitionParams()
void	initModelParamsAndCounters()
protected void	initStartProbs() Initialize the starting probabilities for each state (hard coded).
boolean	loadModelParams()
protected Double	maximizationStep() Returns the total change in model parameter values.
protected Double	maximizationStepForEmissions()
protected Double	maximizationStepForTransitions()
void	saveModelParams()
void	setParamFile(String paramFile)
void	setStartingParams(List<Double> emmisions, List<Double> transitions)
void	setTfIdfData(String dataFile)
protected double	smoothCounter(int i, List<Double> counters, List<Double> params) Dirichlet smoothing ------------------- Without a prior: P(data | theta) = theta(i)^beta(i) = counters(i) With a dirichlet prior: P(data | theta)*p(theta) = theta(i)^(beta(i) + alpha(i)) = theta(i)^beta(i) + theta(i)^alpha(i) counters(i) + params(i)^alpha(i)
boolean	train(List<List<Acronym>> corpus, List<Map<String,String>> trueLabels)
boolean	train(List<List<Acronym>> corpus, List<Map<String,String>> trueLabels, boolean force)
protected boolean	trainCorpus(List<List<Acronym>> corpus, List<Map<String,String>> trueLabels)
boolean	useTDIDF()
AbbreviationAlignmentContainer<AbbvGapsHMM.Emissions,AbbvGapsHMM.States>	viterbi(Acronym acronym)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbbvGapsHMM

public AbbvGapsHMM()

AbbvGapsHMM

public AbbvGapsHMM(String modelParamFile)

Parameters:

modelParamFile - After training, the model parameters will be saved to this file.

AbbvGapsHMM

public AbbvGapsHMM(String modelParamFile,
                   boolean allowVowelsMatch)

Parameters:

modelParamFile - After training, the model parameters will be saved to this file.

Method Detail

getEmmisionParams

public List<Double> getEmmisionParams()

getTransitionParams

public List<Double> getTransitionParams()

useTDIDF

public boolean useTDIDF()

getDF

public Double getDF(String word)

setTfIdfData

public void setTfIdfData(String dataFile)
                  throws IOException

Throws:

IOException

initStartProbs

protected void initStartProbs()

Initialize the starting probabilities for each state (hard coded).

setParamFile

public void setParamFile(String paramFile)

train

public boolean train(List<List<Acronym>> corpus,
                     List<Map<String,String>> trueLabels)

train

public boolean train(List<List<Acronym>> corpus,
                     List<Map<String,String>> trueLabels,
                     boolean force)

setStartingParams

public void setStartingParams(List<Double> emmisions,
                              List<Double> transitions)

initModelParamsAndCounters

public void initModelParamsAndCounters()

trainCorpus

protected boolean trainCorpus(List<List<Acronym>> corpus,
                              List<Map<String,String>> trueLabels)

expectationStep

protected void expectationStep(Acronym acronym,
                               String trueLongForm)

viterbi

public AbbreviationAlignmentContainer<AbbvGapsHMM.Emissions,AbbvGapsHMM.States> viterbi(Acronym acronym)

saveModelParams

public void saveModelParams()

loadModelParams

public boolean loadModelParams()

maximizationStep

protected Double maximizationStep()

Returns the total change in model parameter values.

maximizationStepForTransitions

protected Double maximizationStepForTransitions()

smoothCounter

protected double smoothCounter(int i,
                               List<Double> counters,
                               List<Double> params)

Dirichlet smoothing ------------------- Without a prior: P(data | theta) = theta(i)^beta(i) = counters(i) With a dirichlet prior: P(data | theta)*p(theta) = theta(i)^(beta(i) + alpha(i)) = theta(i)^beta(i) + theta(i)^alpha(i) counters(i) + params(i)^alpha(i)

getNewStateVal

protected double getNewStateVal(double current,
                                double total)

maximizationStepForEmissions

protected Double maximizationStepForEmissions()