hex.Infogram

Class InfogramUtils

java.lang.Object

hex.Infogram.InfogramUtils

public class InfogramUtils
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description
InfogramUtils()

Method Summary

Modifier and Type	Method and Description
static hex.ModelBuilder[]	buildModelBuilders(hex.Model.Parameters[] modelParams)
static hex.Model.Parameters[]	buildModelParameters(water.fvec.Frame[] trainingFrames, hex.Model.Parameters infoParams, int numModels, InfogramModel.InfogramParameters.Algorithm algoName) Build model parameters for model specified in infogram_algorithm.
static double[]	calculateFinalCMI(double[] cmiRaw, boolean buildCore) To calculate the CMI, refer to https://h2oai.atlassian.net/browse/PUBDEV-8075 section I step 2 for core infogram, section II step 2 for fair infogram.
static void	extractInfogramInfo(InfogramModel infoModel, double[][] cmiRaw, java.util.List<java.util.List<java.lang.String>> columns, int foldIndex)
static java.lang.String[]	extractPredictors(InfogramModel.InfogramParameters parms, water.fvec.Frame train, java.lang.String foldColumnName) This method will take the columns of _parms.train().
static java.lang.String[]	extractTopKPredictors(InfogramModel.InfogramParameters parms, water.fvec.Frame trainFrame, java.lang.String[] eligiblePredictors, water.Key<water.fvec.Frame>[] generatedFrameKeys) Method to run infogram model once in order to get the variable importance of the topK predictors
static water.fvec.Frame	extractTrainingFrame(InfogramModel.InfogramParameters parms, java.lang.String[] sensitivePredictors, double dataFraction, water.fvec.Frame trainFrame) This method will perform two functions: - if user only wants a fraction of the training dataset to be used for infogram calculation, we will split the training frame and only use a fraction of it for infogram training purposes; - next, a new training dataset will be generated containing only the predictors in predictors2Use array.
static int	findstart(water.Key<water.fvec.Frame>[] generatedFrameKeys)
static water.fvec.Frame	generateCMIRelevance(java.lang.String[] allPredictorNames, double[] admissible, double[] admissibleIndex, double[] relevance, double[] cmi, double[] cmiRaw, boolean buildCore)
static java.lang.String[]	generateModelDescription(java.lang.String[] topKPredictors, java.lang.String[] sensitive_attributes)
static void	removeFromDKV(water.Key<water.fvec.Frame>[] generatedFrameKeys)
static water.fvec.Frame	subtractAdd2Frame(water.fvec.Frame base, water.fvec.Frame featureFrame, java.lang.String[] removeFeatures, java.lang.String[] addFeatures)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InfogramUtils

public InfogramUtils()

Method Detail

extractPredictors

public static java.lang.String[] extractPredictors(InfogramModel.InfogramParameters parms,
                                                   water.fvec.Frame train,
                                                   java.lang.String foldColumnName)

This method will take the columns of _parms.train(). It will then remove the response, any columns in _parms._sensitive_attributes from the columns of _parms.train(), weights_column, offset_column. Then, the columns that are left are the columns that are eligible to get their InfoGram.

extractTopKPredictors

public static java.lang.String[] extractTopKPredictors(InfogramModel.InfogramParameters parms,
                                                       water.fvec.Frame trainFrame,
                                                       java.lang.String[] eligiblePredictors,
                                                       water.Key<water.fvec.Frame>[] generatedFrameKeys)

Method to run infogram model once in order to get the variable importance of the topK predictors

findstart

public static int findstart(water.Key<water.fvec.Frame>[] generatedFrameKeys)

extractTrainingFrame

public static water.fvec.Frame extractTrainingFrame(InfogramModel.InfogramParameters parms,
                                                    java.lang.String[] sensitivePredictors,
                                                    double dataFraction,
                                                    water.fvec.Frame trainFrame)

This method will perform two functions: - if user only wants a fraction of the training dataset to be used for infogram calculation, we will split the training frame and only use a fraction of it for infogram training purposes; - next, a new training dataset will be generated containing only the predictors in predictors2Use array.

generateModelDescription

public static java.lang.String[] generateModelDescription(java.lang.String[] topKPredictors,
                                                          java.lang.String[] sensitive_attributes)

buildModelParameters

public static hex.Model.Parameters[] buildModelParameters(water.fvec.Frame[] trainingFrames,
                                                          hex.Model.Parameters infoParams,
                                                          int numModels,
                                                          InfogramModel.InfogramParameters.Algorithm algoName)

Build model parameters for model specified in infogram_algorithm. Any model specific parameters can be specified in infogram_algorithm_params.

buildModelBuilders

public static hex.ModelBuilder[] buildModelBuilders(hex.Model.Parameters[] modelParams)

generateCMIRelevance

public static water.fvec.Frame generateCMIRelevance(java.lang.String[] allPredictorNames,
                                                    double[] admissible,
                                                    double[] admissibleIndex,
                                                    double[] relevance,
                                                    double[] cmi,
                                                    double[] cmiRaw,
                                                    boolean buildCore)

removeFromDKV

public static void removeFromDKV(water.Key<water.fvec.Frame>[] generatedFrameKeys)

calculateFinalCMI

public static double[] calculateFinalCMI(double[] cmiRaw,
                                         boolean buildCore)

To calculate the CMI, refer to https://h2oai.atlassian.net/browse/PUBDEV-8075 section I step 2 for core infogram, section II step 2 for fair infogram. Note that the last model is built with all predictors for core infogram or built with protected columns for fair infogram.

subtractAdd2Frame

public static water.fvec.Frame subtractAdd2Frame(water.fvec.Frame base,
                                                 water.fvec.Frame featureFrame,
                                                 java.lang.String[] removeFeatures,
                                                 java.lang.String[] addFeatures)

extractInfogramInfo

public static void extractInfogramInfo(InfogramModel infoModel,
                                       double[][] cmiRaw,
                                       java.util.List<java.util.List<java.lang.String>> columns,
                                       int foldIndex)