hex

Class Model<M extends Model<M,P,O>,P extends Model.Parameters,O extends Model.Output>

java.lang.Object

water.Iced

water.Keyed

water.Lockable<M>

hex.Model<M,P,O>

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable

Direct Known Subclasses:

SupervisedModel

public abstract class Model<M extends Model<M,P,O>,P extends Model.Parameters,O extends Model.Output>
extends Lockable<M>

A Model models reality (hopefully). A model can be used to 'score' a row (make a prediction), or a collection of rows on any compatible dataset - meaning the row has all the columns with the same names as used to build the mode and any enum (categorical) columns can be adapted.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Model.ModelCategory
static class	Model.Output Model-specific output class.
static class	Model.Parameters Model-specific parameter class.

Field Summary

Fields

Modifier and Type	Field and Description
O	_output
P	_parms
java.lang.String[]	_warnings
long	training_duration_in_ms The duration in mS for model training.
long	training_start_time The start time in mS since the epoch for model training.

Fields inherited from class water.Lockable

_lockers

Fields inherited from class water.Keyed

_key

Constructor Summary

Constructors

Constructor and Description
Model(Key selfKey, P parms, O output) Full constructor

Method Summary

Methods

Modifier and Type	Method and Description
Frame[]	adapt(Frame fr, boolean exact) Build an adapted Frame from the given Frame.
Frame[]	adapt(Frame fr, boolean exact, boolean haveResponse)
protected int[][][]	adapt(java.lang.String[] names, java.lang.String[][] domains, boolean exact) Build an adaption array.
void	addWarning(java.lang.String s)
long	checksum() High-quality 64-bit checksum of the content of the object.
protected ModelMetrics	computeModelMetrics(long start_time, Frame frame, Frame predictions)
static int[][]	getDomainMapping(java.lang.String[] modelDom, java.lang.String[] colDom, boolean exact) Returns a mapping between values of model domains (modelDom) and given column domain.
static int[][]	getDomainMapping(java.lang.String colName, java.lang.String[] modelDom, java.lang.String[] colDom, boolean logNonExactMapping) Returns a mapping for given column according to given modelDom.
boolean	isSupervised() Needs to be set correctly otherwise eg scoring does not work.
protected double	missingColumnsType() Type of missing columns during adaptation between train/test datasets Overload this method for models that have sparse data handling.
protected Futures	remove_impl(Futures fs) Override to remove subparts, but not self, of composite Keyed objects.
abstract ModelSchema	schema() Externally visible default schema TODO: this is in the wrong layer: the internals should not know anything about the schemas!!! This puts a reverse edge into the dependency graph.
double	score(double[] data)
Frame	score(Frame fr) Bulk score for given fr frame.
Frame	score(Frame fr, boolean adapt) Bulk score the frame fr, producing a Frame result; the 1st Vec is the predicted class, the remaining Vecs are the probability distributions.
float[]	score(Frame fr, boolean exact, int row) Single row scoring, on a compatible Frame.
float[]	score(int[][][] map, double[] row, float[] preds) Single row scoring, on a compatible set of data, given an adaption vector
float[]	score(java.lang.String[] names, java.lang.String[][] domains, boolean exact, double[] row) Single row scoring, on a compatible set of data.
protected float[]	score0(Chunk[] chks, int row_in_chunk, double[] tmp, float[] preds) Bulk scoring API for one row.
protected abstract float[]	score0(double[] data, float[] preds) Subclasses implement the scoring logic.
void	start_training(long training_start_time)
void	start_training(Model previous)
void	stop_training()

Methods inherited from class water.Lockable

delete_and_lock, delete, delete, delete, read_lock, read_lock, unlock_all, unlock, update, write_lock

Methods inherited from class water.Keyed

remove, remove, remove, remove

Methods inherited from class water.Iced

clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_parms

public P extends Model.Parameters _parms

_warnings

public java.lang.String[] _warnings

_output

public O extends Model.Output _output

training_start_time

public long training_start_time

The start time in mS since the epoch for model training.

training_duration_in_ms

public long training_duration_in_ms

The duration in mS for model training.

Constructor Detail

Model

public Model(Key selfKey,
     P parms,
     O output)

Full constructor

Method Detail

isSupervised

public boolean isSupervised()

Needs to be set correctly otherwise eg scoring does not work.

Returns:

true if there was a response column used during training.

addWarning

public void addWarning(java.lang.String s)

schema

public abstract ModelSchema schema()

Externally visible default schema TODO: this is in the wrong layer: the internals should not know anything about the schemas!!! This puts a reverse edge into the dependency graph.

start_training

public void start_training(long training_start_time)

start_training

public void start_training(Model previous)

stop_training

public void stop_training()

score

public Frame score(Frame fr)

Bulk score for given fr frame. The frame is always adapted to this model.

Parameters:

fr - frame to be scored

Returns:

frame holding predicted values

See Also:

score(Frame, boolean)

score

public final Frame score(Frame fr,
          boolean adapt)

Bulk score the frame fr, producing a Frame result; the 1st Vec is the predicted class, the remaining Vecs are the probability distributions. For Regression (single-class) models, the 1st and only Vec is the prediction value. The flat adapt

Parameters:

fr - frame which should be scored

adapt - a flag enforcing an adaptation of fr to this model. If flag is false scoring code expect that fr is already adapted.

Returns:

a new frame containing a predicted values. For classification it contains a column with prediction and distribution for all response classes. For regression it contains only one column with predicted values.

score

public final float[] score(Frame fr,
            boolean exact,
            int row)

Single row scoring, on a compatible Frame.

score

public final float[] score(java.lang.String[] names,
            java.lang.String[][] domains,
            boolean exact,
            double[] row)

Single row scoring, on a compatible set of data. Fairly expensive to adapt.

score

public final float[] score(int[][][] map,
            double[] row,
            float[] preds)

Single row scoring, on a compatible set of data, given an adaption vector

adapt

protected int[][][] adapt(java.lang.String[] names,
              java.lang.String[][] domains,
              boolean exact)

Build an adaption array. The length is equal to the Model's vector length. Each inner 2D-array is a compressed domain map from data domains to model domains - or null for non-enum columns, or null for identity mappings. The extra final int[] is the column mapping itself, mapping from model columns to data columns. or -1 if missing. If 'exact' is true, will throw if there are: any columns in the model but not in the input set; any enums in the data that the model does not understand any enums returned by the model that the data does not have a mapping for. If 'exact' is false, these situations will use or return NA's instead.

computeModelMetrics

protected ModelMetrics computeModelMetrics(long start_time,
                               Frame frame,
                               Frame predictions)

missingColumnsType

protected double missingColumnsType()

Type of missing columns during adaptation between train/test datasets Overload this method for models that have sparse data handling. Otherwise, NaN is used.

Returns:

real-valued number (can be NaN)

adapt

public Frame[] adapt(Frame fr,
            boolean exact)

Build an adapted Frame from the given Frame. Useful for efficient bulk scoring of a new dataset to an existing model. Same adaption as above, but expressed as a Frame instead of as an int[][]. The returned Frame does not have a response column. It returns a two element array containing an adapted frame and a frame which contains only vectors which where adapted (the purpose of the second frame is to delete all adapted vectors with deletion of the frame).

adapt

public Frame[] adapt(Frame fr,
            boolean exact,
            boolean haveResponse)

getDomainMapping

public static int[][] getDomainMapping(java.lang.String[] modelDom,
                       java.lang.String[] colDom,
                       boolean exact)

Returns a mapping between values of model domains (modelDom) and given column domain.

See Also:

getDomainMapping(String, String[], String[], boolean)

getDomainMapping

public static int[][] getDomainMapping(java.lang.String colName,
                       java.lang.String[] modelDom,
                       java.lang.String[] colDom,
                       boolean logNonExactMapping)

Returns a mapping for given column according to given modelDom. In this case, modelDom is

Parameters:

colName - name of column which is mapped, can be null.

modelDom -

logNonExactMapping -

Returns:

A mapping for given column according to given modelDom.

score0

protected float[] score0(Chunk[] chks,
             int row_in_chunk,
             double[] tmp,
             float[] preds)

Bulk scoring API for one row. Chunks are all compatible with the model, and expect the last Chunks are for the final distribution and prediction. Default method is to just load the data into the tmp array, then call subclass scoring logic.

score0

protected abstract float[] score0(double[] data,
             float[] preds)

Subclasses implement the scoring logic. The data is pre-loaded into a re-used temp array, in the order the model expects. The predictions are loaded into the re-used temp array, which is also returned.

score

public double score(double[] data)

remove_impl

protected Futures remove_impl(Futures fs)

Description copied from class: Keyed

Override to remove subparts, but not self, of composite Keyed objects. Examples include Vec (removing associated Chunk keys) and Frame (removing associated Vec keys.)

Overrides:

remove_impl in class Keyed

checksum

public long checksum()

Description copied from class: Keyed

High-quality 64-bit checksum of the content of the object. Similar to hashcode(), but a long to reduce the chance of hash clashes. For composite objects this should be defined using the subcomponents' checksums (or hashcodes if not available). If two Keyed objects have the same checksum() there should be a 1 - 1/2^64 chance that they are the same object by value.

Specified by:

checksum in class Keyed