ai.idylnlp.models.opennlp.training

Class TokenModelOperations

java.lang.Object

ai.idylnlp.models.opennlp.training.TokenModelOperations

All Implemented Interfaces:

ModelCrossValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>, ModelSeparateDataValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>, ModelTrainingOperations

public class TokenModelOperations
extends Object
implements ModelTrainingOperations, ModelSeparateDataValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>, ModelCrossValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>

Operations for training and validating token models.

Author:

Mountain Fog, Inc.

Constructor Summary

Constructors

Constructor and Description
TokenModelOperations()

Method Summary

Modifier and Type	Method and Description
static ai.idylnlp.model.training.FMeasureModelValidationResult	crossValidate(TrainingDefinitionReader reader, int folds) Performs cross-validation of a token model.
ai.idylnlp.model.training.FMeasureModelValidationResult	crossValidationEvaluateMaxEntQN(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining, com.neovisionaries.i18n.LanguageCode language, int iterations, int cutOff, int folds, double l1, double l2, int m, int max) Performs model cross validation using the maxent QN algorithm.
ai.idylnlp.model.training.FMeasureModelValidationResult	crossValidationEvaluatePerceptron(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining, com.neovisionaries.i18n.LanguageCode language, int iterations, int cutOff, int folds) Performs model cross validation using the perceptron algorithm.
ai.idylnlp.model.training.FMeasureModelValidationResult	separateDataEvaluate(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining, String modelFileName, String encryptionKey) Performs model validation using separate data.
static String	train(TrainingDefinitionReader reader) Performs token model training using a training definition file.
String	trainMaxEntQN(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining, String modelFile, com.neovisionaries.i18n.LanguageCode language, String encryptionKey, int cutOff, int iterations, int threads, double l1, double l2, int m, int max) Train a maxent model using quasi-newton.
String	trainPerceptron(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining, String modelFile, com.neovisionaries.i18n.LanguageCode language, String encryptionKey, int cutOff, int iterations) Train a perceptron model using.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenModelOperations

public TokenModelOperations()

Method Detail

train

public static String train(TrainingDefinitionReader reader)
                    throws IOException

Performs token model training using a training definition file.

Parameters:

reader - A TrainingDefinitionReader.

Returns:

The generated model's ID.

Throws:

IOException - Thrown if the model creation fails.

crossValidate

public static ai.idylnlp.model.training.FMeasureModelValidationResult crossValidate(TrainingDefinitionReader reader,
                                                                                    int folds)
                                                                             throws IOException

Performs cross-validation of a token model.

Parameters:

reader - A TrainingDefinitionReader.

folds - The number of cross-validation folds.

Returns:

A FMeasureModelValidationResult.

Throws:

IOException - Thrown if the model cannot be validated.

crossValidationEvaluateMaxEntQN

public ai.idylnlp.model.training.FMeasureModelValidationResult crossValidationEvaluateMaxEntQN(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining,
                                                                                               com.neovisionaries.i18n.LanguageCode language,
                                                                                               int iterations,
                                                                                               int cutOff,
                                                                                               int folds,
                                                                                               double l1,
                                                                                               double l2,
                                                                                               int m,
                                                                                               int max)
                                                                                        throws IOException

Description copied from interface: ModelCrossValidationOperations

Performs model cross validation using the maxent QN algorithm.

Specified by:

crossValidationEvaluateMaxEntQN in interface ModelCrossValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>

Parameters:

subjectOfTraining - The SubjectOfTrainingOrEvaluation.

language - The language of the model.

iterations - The number of iterations.

cutOff - The value of the cutoff.

folds - The number of cross validation folds.

Returns:

A FMeasureModelValidationResult.

Throws:

IOException - Thrown if the cross validation fails.

crossValidationEvaluatePerceptron

public ai.idylnlp.model.training.FMeasureModelValidationResult crossValidationEvaluatePerceptron(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining,
                                                                                                 com.neovisionaries.i18n.LanguageCode language,
                                                                                                 int iterations,
                                                                                                 int cutOff,
                                                                                                 int folds)
                                                                                          throws IOException

Description copied from interface: ModelCrossValidationOperations

Performs model cross validation using the perceptron algorithm.

Specified by:

crossValidationEvaluatePerceptron in interface ModelCrossValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>

Parameters:

subjectOfTraining - The SubjectOfTrainingOrEvaluation.

language - The language of the model.

iterations - The number of iterations.

cutOff - The value of the cutoff.

folds - The number of cross validation folds.

Returns:

A FMeasureModelValidationResult.

Throws:

IOException - Thrown if the cross validation fails.

separateDataEvaluate

public ai.idylnlp.model.training.FMeasureModelValidationResult separateDataEvaluate(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining,
                                                                                    String modelFileName,
                                                                                    String encryptionKey)
                                                                             throws IOException

Description copied from interface: ModelSeparateDataValidationOperations

Performs model validation using separate data. This validation requires a built model file.

Specified by:

separateDataEvaluate in interface ModelSeparateDataValidationOperations<ai.idylnlp.model.training.FMeasureModelValidationResult>

Parameters:

subjectOfTraining - The SubjectOfTrainingOrEvaluation.

modelFileName - The full path to the model file.

encryptionKey - The model's encryption key.

Returns:

The results of the validation as an FMeasure.

Throws:

IOException - Thrown if any of the input files cannot be read.

trainMaxEntQN

public String trainMaxEntQN(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining,
                            String modelFile,
                            com.neovisionaries.i18n.LanguageCode language,
                            String encryptionKey,
                            int cutOff,
                            int iterations,
                            int threads,
                            double l1,
                            double l2,
                            int m,
                            int max)
                     throws IOException

Description copied from interface: ModelTrainingOperations

Train a maxent model using quasi-newton.

Specified by:

trainMaxEntQN in interface ModelTrainingOperations

Parameters:

subjectOfTraining - The SubjectOfTrainingOrEvaluation.

modelFile - The output model file.

language - The language of the model.

encryptionKey - The model's encryption key.

cutOff - The training cutoff.

iterations - The training iterations.

threads - The number of training threads.

Returns:

The generated model's ID.

Throws:

IOException - Thrown if the model cannot be trained.

trainPerceptron

public String trainPerceptron(ai.idylnlp.model.nlp.subjects.SubjectOfTrainingOrEvaluation subjectOfTraining,
                              String modelFile,
                              com.neovisionaries.i18n.LanguageCode language,
                              String encryptionKey,
                              int cutOff,
                              int iterations)
                       throws IOException

Description copied from interface: ModelTrainingOperations

Train a perceptron model using.

Specified by:

trainPerceptron in interface ModelTrainingOperations

Parameters:

subjectOfTraining - The SubjectOfTrainingOrEvaluation.

modelFile - The output model file.

language - The language of the model.

encryptionKey - The model's encryption key.

cutOff - The training cutoff.

iterations - The training iterations.

Returns:

The generated model's ID.

Throws:

IOException - Thrown if the model cannot be trained.