hex.splitframe

Class SplitFrame

java.lang.Object

water.Iced<T>

water.Keyed

water.Job<M>

hex.ModelBuilder<SplitFrameModel,SplitFrameModel.SplitFrameParameters,SplitFrameModel.SplitFrameOutput>

hex.splitframe.SplitFrame

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable

public class SplitFrame
extends hex.ModelBuilder<SplitFrameModel,SplitFrameModel.SplitFrameParameters,SplitFrameModel.SplitFrameOutput>

Frame splitter function to divide given frame into multiple partitions based on given ratios.

The task creates ratios.length+1 output frame each containing a demanded fraction of rows from source dataset

The tasks internally extract data from source chunks and create output chunks in preserving order of parts. I.e., the 1st partition contains the first P1-rows, the 2nd partition contains following P2-rows, ...

Assumptions and invariants

• number of demanding split parts is reasonable number, i.e., <10. The task is not designed to split into many small parts.
• the worker DOES NOT preserves distribution of new chunks over the cloud according to source dataset chunks.
• rows inside one output chunk are not shuffled, they are extracted deterministically in the same order as they appear in source chunk.
• workers can enforce data transfers if they need to obtain data from remote chunks.

NOTE: the implementation is data-transfer expensive and in some cases it would be beneficial to use original implementation from 9af3f4e.

.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class hex.ModelBuilder

hex.ModelBuilder.ValidationMessage

Nested classes/interfaces inherited from class water.Job

water.Job.JobCancelledException, water.Job.JobState, water.Job.Progress, water.Job.ProgressUpdate

Field Summary

Fields inherited from class hex.ModelBuilder

_messages, _parms, _train, _valid

Fields inherited from class water.Job

_description, _dest, _end_time, _exception, _progressKey, _start_time, _state

Fields inherited from class water.Keyed

_key

Constructor Summary

Constructors

Constructor and Description
SplitFrame(SplitFrameModel.SplitFrameParameters parms)

Method Summary

Methods

Modifier and Type	Method and Description
hex.Model.ModelCategory[]	can_build()
void	init(boolean expensive) Initialize the ModelBuilder, validating all arguments and preparing the training frame.
hex.schemas.ModelBuilderSchema	schema()
SplitFrame	trainModel()

Methods inherited from class hex.ModelBuilder

clearInitState, createModelBuilder, error_count, error, getAlgo, getModelBuilder, getModelBuilderName, getModelBuilders, getModelClass, hide, info, registerModelBuilder, train, valid, validationErrors, warn

Methods inherited from class water.Job

cancel, cancel, checksum_impl, dest, done, failed, get, isCancelledOrCrashed, isDone, isRunning, isRunning, isStopped, jobs, msec, onCancelled, progress, remove_impl, start, update, update

Methods inherited from class water.Keyed

checksum, remove, remove, remove, remove

Methods inherited from class water.Iced

clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitFrame

public SplitFrame(SplitFrameModel.SplitFrameParameters parms)

Method Detail

schema

public hex.schemas.ModelBuilderSchema schema()

Specified by:

schema in class hex.ModelBuilder<SplitFrameModel,SplitFrameModel.SplitFrameParameters,SplitFrameModel.SplitFrameOutput>

trainModel

public SplitFrame trainModel()

Specified by:

trainModel in class hex.ModelBuilder<SplitFrameModel,SplitFrameModel.SplitFrameParameters,SplitFrameModel.SplitFrameOutput>

can_build

public hex.Model.ModelCategory[] can_build()

Specified by:

can_build in class hex.ModelBuilder<SplitFrameModel,SplitFrameModel.SplitFrameParameters,SplitFrameModel.SplitFrameOutput>

init

public void init(boolean expensive)

Initialize the ModelBuilder, validating all arguments and preparing the training frame. This call is expected to be overridden in the subclasses and each subclass will start with "super.init();". This call is made by the front-end whenever the GUI is clicked, and needs to be fast; heavy-weight prep needs to wait for the trainModel() call. Validate the ratios.

Overrides:

init in class hex.ModelBuilder<SplitFrameModel,SplitFrameModel.SplitFrameParameters,SplitFrameModel.SplitFrameOutput>