org.apache.hadoop.hive.ql.io

Class AcidUtils.ParsedBase

java.lang.Object

org.apache.hadoop.hive.ql.io.AcidUtils.ParsedBaseLight

org.apache.hadoop.hive.ql.io.AcidUtils.ParsedBase

All Implemented Interfaces:

AcidUtils.ParsedDirectory

Enclosing class:

AcidUtils

public static final class AcidUtils.ParsedBase
extends AcidUtils.ParsedBaseLight
implements AcidUtils.ParsedDirectory

In addition to AcidUtils.ParsedBaseLight this knows if the data is in raw format, i.e. doesn't have acid metadata columns embedded in the files. To determine this in some cases requires looking at the footer of the data file which can be expensive so if this info is not needed AcidUtils.ParsedBaseLight should be used.

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.io.AcidUtils.ParsedBaseLight

baseDirPath, visibilityTxnId, writeId

Method Summary

Modifier and Type	Method and Description
List<org.apache.hadoop.hive.shims.HadoopShims.HdfsFileStatusWithId>	getFiles(org.apache.hadoop.fs.FileSystem fs, Ref<Boolean> useFileIds) Returns the files from the base directory.
boolean	isRawFormat()
void	setFiles(List<org.apache.hadoop.hive.shims.HadoopShims.HdfsFileStatusWithId> files)
void	setRawFormat(boolean rawFormat)
String	toString()

Methods inherited from class org.apache.hadoop.hive.ql.io.AcidUtils.ParsedBaseLight

getBaseDirPath, getVisibilityTxnId, getWriteId, parseBase

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

isRawFormat

public boolean isRawFormat()

setRawFormat

public void setRawFormat(boolean rawFormat)

getFiles

public List<org.apache.hadoop.hive.shims.HadoopShims.HdfsFileStatusWithId> getFiles(org.apache.hadoop.fs.FileSystem fs,
                                                                                    Ref<Boolean> useFileIds)
                                                                             throws IOException

Returns the files from the base directory. The list is either populated by AcidUtils or it will be listed through the provided FileSystem object. If the list was not filled and no FS is provided, returns null.

Specified by:

getFiles in interface AcidUtils.ParsedDirectory

Parameters:

fs - FileSystem optional

useFileIds - to use fileId based listing or not. Optional

Returns:

list of files in the base directory

Throws:

IOException - ex

setFiles

public void setFiles(List<org.apache.hadoop.hive.shims.HadoopShims.HdfsFileStatusWithId> files)

toString

public String toString()

Overrides:

toString in class AcidUtils.ParsedBaseLight