org.datavec.api.transform.reduce

Class Reducer

java.lang.Object

org.datavec.api.transform.reduce.Reducer

All Implemented Interfaces:

Serializable, IReducer

public class Reducer
extends Object
implements IReducer

A Reducer is used to take a set of examples and reduce them. The idea: suppose you have a large number of columns, and you want to combine/reduce the values in each column.
Reducer allows you to specify different reductions for differently for different columns: min, max, sum, mean etc. See Reducer.Builder and ReduceOp for the full list.

Uses are: (1) Reducing examples by a key (2) Reduction operations in time series (windowing ops, etc)

Author:

Alex Black

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Reducer.Builder
static class	Reducer.ConditionalReduction

Constructor Summary

Constructors

Constructor and Description
Reducer(List<String> keyColumns, ReduceOp defaultOp, Map<String,ReduceOp> opMap, Map<String,ColumnReduction> customReductions, Map<String,Reducer.ConditionalReduction> conditionalReductions, Set<String> ignoreInvalidInColumns)

Method Summary

Modifier and Type	Method and Description
Schema	getInputSchema()
List<String>	getKeyColumns()
List<Writable>	reduce(List<List<Writable>> examplesList)
static Writable	reduceBytesColumn(ReduceOp op, List<Writable> list)
static Writable	reduceColumn(ReduceOp op, ColumnType type, List<Writable> values, boolean ignoreInvalid, ColumnMetaData metaData)
static Writable	reduceDoubleColumn(ReduceOp op, List<Writable> values, boolean ignoreInvalid, ColumnMetaData metaData)
static Writable	reduceLongColumn(ReduceOp op, List<Writable> values, boolean ignoreInvalid, ColumnMetaData metaData)
static Writable	reduceStringOrCategoricalColumn(ReduceOp op, List<Writable> values, boolean ignoreInvalid, ColumnMetaData metaData)
static Writable	reduceTimeColumn(ReduceOp op, List<Writable> values, boolean ignoreInvalid, ColumnMetaData metaData)
void	setInputSchema(Schema schema)
String	toString()
Schema	transform(Schema schema) Get the output schema, given the input schema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Reducer

public Reducer(List<String> keyColumns,
               ReduceOp defaultOp,
               Map<String,ReduceOp> opMap,
               Map<String,ColumnReduction> customReductions,
               Map<String,Reducer.ConditionalReduction> conditionalReductions,
               Set<String> ignoreInvalidInColumns)

Method Detail

setInputSchema

public void setInputSchema(Schema schema)

Specified by:

setInputSchema in interface IReducer

getInputSchema

public Schema getInputSchema()

Specified by:

getInputSchema in interface IReducer

Returns:

getKeyColumns

public List<String> getKeyColumns()

Specified by:

getKeyColumns in interface IReducer

Returns:

transform

public Schema transform(Schema schema)

Get the output schema, given the input schema

Specified by:

transform in interface IReducer

Returns:

reduce

public List<Writable> reduce(List<List<Writable>> examplesList)

Specified by:

reduce in interface IReducer

Returns:

reduceColumn

public static Writable reduceColumn(ReduceOp op,
                                    ColumnType type,
                                    List<Writable> values,
                                    boolean ignoreInvalid,
                                    ColumnMetaData metaData)

reduceLongColumn

public static Writable reduceLongColumn(ReduceOp op,
                                        List<Writable> values,
                                        boolean ignoreInvalid,
                                        ColumnMetaData metaData)

reduceDoubleColumn

public static Writable reduceDoubleColumn(ReduceOp op,
                                          List<Writable> values,
                                          boolean ignoreInvalid,
                                          ColumnMetaData metaData)

reduceStringOrCategoricalColumn

public static Writable reduceStringOrCategoricalColumn(ReduceOp op,
                                                       List<Writable> values,
                                                       boolean ignoreInvalid,
                                                       ColumnMetaData metaData)

reduceTimeColumn

public static Writable reduceTimeColumn(ReduceOp op,
                                        List<Writable> values,
                                        boolean ignoreInvalid,
                                        ColumnMetaData metaData)

reduceBytesColumn

public static Writable reduceBytesColumn(ReduceOp op,
                                         List<Writable> list)

toString

public String toString()

Overrides:

toString in class Object