org.apache.hudi.hadoop

Class HoodieColumnProjectionUtils

java.lang.Object

org.apache.hudi.hadoop.HoodieColumnProjectionUtils

public class HoodieColumnProjectionUtils
extends Object

Utility functions copied from Hive ColumnProjectionUtils.java. Needed to copy as we see NoSuchMethod errors when directly using these APIs with/without Spark. Some of these methods are not available across hive versions.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG
static String	READ_COLUMN_IDS_CONF_STR
static String	READ_COLUMN_NAMES_CONF_STR the nested column path is the string from the root to the leaf e.g.

Constructor Summary

Constructors

Constructor and Description
HoodieColumnProjectionUtils()

Method Summary

Modifier and Type	Method and Description
static List<Pair<String,String>>	getIOColumnNameAndTypes(org.apache.hadoop.conf.Configuration conf)
static List<String>	getIOColumns(org.apache.hadoop.conf.Configuration conf)
static List<String>	getIOColumnTypes(org.apache.hadoop.conf.Configuration conf)
static List<Integer>	getReadColumnIDs(org.apache.hadoop.conf.Configuration conf) Returns an array of column ids(start from zero) which is set in the given parameter conf.
static String[]	getReadColumnNames(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

READ_COLUMN_IDS_CONF_STR

public static final String READ_COLUMN_IDS_CONF_STR

See Also:

Constant Field Values

READ_COLUMN_NAMES_CONF_STR

public static final String READ_COLUMN_NAMES_CONF_STR

the nested column path is the string from the root to the leaf e.g. c:struct_of (a:string,b:string). the column a's path is c.a and b's path is c.b

See Also:

Constant Field Values

Constructor Detail

HoodieColumnProjectionUtils

public HoodieColumnProjectionUtils()

Method Detail

getReadColumnIDs

public static List<Integer> getReadColumnIDs(org.apache.hadoop.conf.Configuration conf)

Returns an array of column ids(start from zero) which is set in the given parameter conf.

getReadColumnNames

public static String[] getReadColumnNames(org.apache.hadoop.conf.Configuration conf)

getIOColumns

public static List<String> getIOColumns(org.apache.hadoop.conf.Configuration conf)

getIOColumnTypes

public static List<String> getIOColumnTypes(org.apache.hadoop.conf.Configuration conf)

getIOColumnNameAndTypes

public static List<Pair<String,String>> getIOColumnNameAndTypes(org.apache.hadoop.conf.Configuration conf)