ai.h2o.targetencoding

Class TargetEncoder

java.lang.Object

water.Iced<TargetEncoder>

ai.h2o.targetencoding.TargetEncoder

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable<TargetEncoder>

public class TargetEncoder
extends water.Iced<TargetEncoder>

Status: alpha version This is a core class for target encoding related logic. In general target encoding could be applied to three types of problems, namely: 1) Binary classification (supported) 2) Multi-class classification (not supported yet) 3) Regression (not supported yet) In order to differentiate between above-mentioned types of problems at hand and enable algorithm to do encodings correctly user should explicitly set corresponding type for a response column: 1) Binary classification: response column should be of a categorical type with cardinality = 2 2) Multi-class: response column should be of a categorical type with cardinality > 2 3) Regression: response column should be of a numerical type Usage: see TargetEncodingTitanicBenchmark.java

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TargetEncoder.AddNoiseTask
static class	TargetEncoder.DataLeakageHandlingStrategy
static class	TargetEncoder.SubtractCurrentRowForLeaveOneOutTask

Field Summary

Fields

Modifier and Type	Field and Description
static BlendingParams	DEFAULT_BLENDING_PARAMS
static java.lang.String	DENOMINATOR_COL_NAME
static java.lang.String	ENCODED_COLUMN_POSTFIX
static java.lang.String	NUMERATOR_COL_NAME

Constructor Summary

Constructors

Constructor and Description
TargetEncoder(java.lang.String[] columnNamesToEncode)

Method Summary

Modifier and Type	Method and Description
water.fvec.Frame	applyTargetEncoding(water.fvec.Frame data, java.lang.String targetColumnName, java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap, TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy, boolean withBlendedAvg, boolean imputeNAsWithNewCategory, BlendingParams blendingParams, long seed)
water.fvec.Frame	applyTargetEncoding(water.fvec.Frame data, java.lang.String targetColumnName, java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap, TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy, boolean withBlendedAvg, double noiseLevel, boolean imputeNAsWithNewCategory, BlendingParams blendingParams, long seed)
water.fvec.Frame	applyTargetEncoding(water.fvec.Frame data, java.lang.String targetColumnName, java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap, TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy, java.lang.String foldColumn, boolean withBlendedAvg, boolean imputeNAsWithNewCategory, BlendingParams blendingParams, long seed)
water.fvec.Frame	applyTargetEncoding(water.fvec.Frame data, java.lang.String targetColumnName, java.util.Map<java.lang.String,water.fvec.Frame> columnToEncodingMap, TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy, java.lang.String foldColumnName, boolean withBlendedAvg, double noiseLevel, boolean imputeNAsWithNewCategory, BlendingParams blendingParams, long seed)
water.fvec.Frame	applyTargetEncoding(water.fvec.Frame data, java.lang.String targetColumnName, java.util.Map<java.lang.String,water.fvec.Frame> columnToEncodingMap, TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy, java.lang.String foldColumnName, boolean useBlending, double noiseLevel, long seed, water.Key<water.fvec.Frame> encodedFrameKey, BlendingParams blendingParams) Core method for applying pre-calculated encodings to the dataset.
water.fvec.Frame	applyTargetEncoding(water.fvec.Frame data, java.lang.String targetColumnName, java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap, TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy, java.lang.String foldColumn, boolean withBlendedAvg, long seed, boolean imputeNAsWithNewCategory, water.Key<water.fvec.Frame> encodedColumnName, BlendingParams blendingParams)
static water.fvec.Frame	groupingIgnoringFoldColumn(java.lang.String foldColumnName, water.fvec.Frame targetEncodingMap, java.lang.String teColumnName)
water.util.IcedHashMapGeneric<java.lang.String,water.fvec.Frame>	prepareEncodingMap(water.fvec.Frame data, java.lang.String targetColumnName, java.lang.String foldColumnName)
water.util.IcedHashMapGeneric<java.lang.String,water.fvec.Frame>	prepareEncodingMap(water.fvec.Frame data, java.lang.String targetColumnName, java.lang.String foldColumnName, boolean imputeNAsWithNewCategory)

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENCODED_COLUMN_POSTFIX

public static final java.lang.String ENCODED_COLUMN_POSTFIX

See Also:

Constant Field Values

DEFAULT_BLENDING_PARAMS

public static final BlendingParams DEFAULT_BLENDING_PARAMS

NUMERATOR_COL_NAME

public static java.lang.String NUMERATOR_COL_NAME

DENOMINATOR_COL_NAME

public static java.lang.String DENOMINATOR_COL_NAME

Constructor Detail

TargetEncoder

public TargetEncoder(java.lang.String[] columnNamesToEncode)

Parameters:

columnNamesToEncode - names of columns to apply target encoding to

Method Detail

prepareEncodingMap

public water.util.IcedHashMapGeneric<java.lang.String,water.fvec.Frame> prepareEncodingMap(water.fvec.Frame data,
                                                                                           java.lang.String targetColumnName,
                                                                                           java.lang.String foldColumnName,
                                                                                           boolean imputeNAsWithNewCategory)

Parameters:

targetColumnName - name of the target column

foldColumnName - name of the column that contains fold number the row is belong to

imputeNAsWithNewCategory - set to `true` to impute NAs with new category. // TODO probably we need to always set it to true bc we do not support null values on the right side of merge operation.

prepareEncodingMap

public water.util.IcedHashMapGeneric<java.lang.String,water.fvec.Frame> prepareEncodingMap(water.fvec.Frame data,
                                                                                           java.lang.String targetColumnName,
                                                                                           java.lang.String foldColumnName)

applyTargetEncoding

public water.fvec.Frame applyTargetEncoding(water.fvec.Frame data,
                                            java.lang.String targetColumnName,
                                            java.util.Map<java.lang.String,water.fvec.Frame> columnToEncodingMap,
                                            TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy,
                                            java.lang.String foldColumnName,
                                            boolean withBlendedAvg,
                                            double noiseLevel,
                                            boolean imputeNAsWithNewCategory,
                                            BlendingParams blendingParams,
                                            long seed)

applyTargetEncoding

public water.fvec.Frame applyTargetEncoding(water.fvec.Frame data,
                                            java.lang.String targetColumnName,
                                            java.util.Map<java.lang.String,water.fvec.Frame> columnToEncodingMap,
                                            TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy,
                                            java.lang.String foldColumnName,
                                            boolean useBlending,
                                            double noiseLevel,
                                            long seed,
                                            water.Key<water.fvec.Frame> encodedFrameKey,
                                            BlendingParams blendingParams)

Core method for applying pre-calculated encodings to the dataset. There are multiple overloaded methods that we will probably be able to get rid off if we are not going to expose Java API for TE. We can just stick to one signature that will suit internal representations of the AutoML's pipeline.

Parameters:

data - dataset that will be used as a base for creation of encodings .

targetColumnName - name of the column with respect to which we were computing encodings.

columnToEncodingMap - map of the prepared encodings with the keys being the names of the columns.

dataLeakageHandlingStrategy - see TargetEncoding.DataLeakageHandlingStrategy //TODO use common interface for stronger type safety.

foldColumnName - column's name that contains fold number the row is belong to.

useBlending - whether to apply blending or not.

noiseLevel - amount of noise to add to the final encodings.

seed - we might want to specify particular values for reproducibility in tests.

Returns:

copy of the `data` frame with encodings

groupingIgnoringFoldColumn

public static water.fvec.Frame groupingIgnoringFoldColumn(java.lang.String foldColumnName,
                                                          water.fvec.Frame targetEncodingMap,
                                                          java.lang.String teColumnName)

applyTargetEncoding

public water.fvec.Frame applyTargetEncoding(water.fvec.Frame data,
                                            java.lang.String targetColumnName,
                                            java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap,
                                            TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy,
                                            java.lang.String foldColumn,
                                            boolean withBlendedAvg,
                                            boolean imputeNAsWithNewCategory,
                                            BlendingParams blendingParams,
                                            long seed)

applyTargetEncoding

public water.fvec.Frame applyTargetEncoding(water.fvec.Frame data,
                                            java.lang.String targetColumnName,
                                            java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap,
                                            TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy,
                                            java.lang.String foldColumn,
                                            boolean withBlendedAvg,
                                            long seed,
                                            boolean imputeNAsWithNewCategory,
                                            water.Key<water.fvec.Frame> encodedColumnName,
                                            BlendingParams blendingParams)

applyTargetEncoding

public water.fvec.Frame applyTargetEncoding(water.fvec.Frame data,
                                            java.lang.String targetColumnName,
                                            java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap,
                                            TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy,
                                            boolean withBlendedAvg,
                                            boolean imputeNAsWithNewCategory,
                                            BlendingParams blendingParams,
                                            long seed)

applyTargetEncoding

public water.fvec.Frame applyTargetEncoding(water.fvec.Frame data,
                                            java.lang.String targetColumnName,
                                            java.util.Map<java.lang.String,water.fvec.Frame> targetEncodingMap,
                                            TargetEncoder.DataLeakageHandlingStrategy dataLeakageHandlingStrategy,
                                            boolean withBlendedAvg,
                                            double noiseLevel,
                                            boolean imputeNAsWithNewCategory,
                                            BlendingParams blendingParams,
                                            long seed)