org.apache.carbondata.hadoop.api

Class CarbonFileInputFormat<T>

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<Void,T>

org.apache.carbondata.hadoop.api.CarbonInputFormat<T>

org.apache.carbondata.hadoop.api.CarbonFileInputFormat<T>

Type Parameters:

T -

All Implemented Interfaces:

Serializable

@InterfaceAudience.User
 @InterfaceStability.Evolving
public class CarbonFileInputFormat<T>
extends CarbonInputFormat<T>
implements Serializable

InputFormat for reading carbondata files without table level metadata support, schema is inferred as following steps: 1. read from schema file is exists 2. read from data file footer

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

Field Summary

Fields inherited from class org.apache.carbondata.hadoop.api.CarbonInputFormat

DATABASE_NAME, hitedStreamFiles, INPUT_SEGMENT_NUMBERS, numBlocks, numSegments, numStreamFiles, numStreamSegments, TABLE_NAME

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

Constructors

Constructor and Description
CarbonFileInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.carbondata.core.metadata.schema.table.CarbonTable	getOrCreateCarbonTable(org.apache.hadoop.conf.Configuration configuration) Get the cached CarbonTable or create it by TableInfo in `configuration`
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext job) Configurations FileInputFormat.INPUT_DIR are used to get table path to read.

Methods inherited from class org.apache.carbondata.hadoop.api.CarbonInputFormat

createQueryModel, createRecordReader, getAbsoluteTableIdentifier, getAccessStreamingSegments, getColumnProjection, getDatabaseName, getDataBlocksOfSegment, getDataTypeConverter, getFilterPredicates, getHitedStreamFiles, getNumBlocks, getNumSegments, getNumStreamFiles, getNumStreamSegments, getPartitionsToPrune, getReadCommittedScope, getReadSupportClass, getTableInfo, getTableName, getValidateSegmentsToAccess, isFgDataMapPruningEnable, isSplitable, projectAllColumns, setAccessStreamingSegments, setCarbonReadSupport, setColumnProjection, setColumnProjection, setDatabaseName, setDataTypeConverter, setFgDataMapPruning, setFilterPredicates, setPartitionIdList, setPartitionsToPrune, setQuerySegment, setQuerySegment, setReadCommittedScope, setSegmentsToAccess, setTableInfo, setTableName, setTablePath, setTransactionalTable, setValidateSegmentsToAccess

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, listStatus, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CarbonFileInputFormat

public CarbonFileInputFormat()

Method Detail

getOrCreateCarbonTable

public org.apache.carbondata.core.metadata.schema.table.CarbonTable getOrCreateCarbonTable(org.apache.hadoop.conf.Configuration configuration)
                                                                                    throws IOException

Description copied from class: CarbonInputFormat

Get the cached CarbonTable or create it by TableInfo in `configuration`

Specified by:

getOrCreateCarbonTable in class CarbonInputFormat<T>

Throws:

IOException

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                       throws IOException

Configurations FileInputFormat.INPUT_DIR are used to get table path to read. Configurations FileInputFormat.INPUT_DIR are used to get table path to read.

Specified by:

getSplits in class CarbonInputFormat<T>

Parameters:

job -

Returns:

List list of CarbonInputSplit

Throws:

IOException