org.apache.hadoop.hive.ql.io

Class AcidUtils.MetaDataFile

java.lang.Object

org.apache.hadoop.hive.common.AcidMetaDataFile

org.apache.hadoop.hive.ql.io.AcidUtils.MetaDataFile

Enclosing class:

AcidUtils

public static class AcidUtils.MetaDataFile
extends AcidMetaDataFile

General facility to place a metadata file into a dir created by acid/compactor write. Load Data commands against Acid tables write AcidUtils.AcidBaseFileType.ORIGINAL_BASE type files into delta_x_x/ (or base_x in case there is Overwrite clause). AcidUtils.MetaDataFile is a small JSON file in this directory that indicates that these files don't have Acid metadata columns and so the values for these columns need to be assigned at read time/compaction.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.common.AcidMetaDataFile

AcidMetaDataFile.DataFormat, AcidMetaDataFile.Field

Field Summary

Fields inherited from class org.apache.hadoop.hive.common.AcidMetaDataFile

CURRENT_VERSION, METADATA_FILE

Constructor Summary

Constructors

Constructor and Description
MetaDataFile()

Method Summary

Modifier and Type	Method and Description
static boolean	isRawFormat(org.apache.hadoop.fs.Path baseOrDeltaDir, org.apache.hadoop.fs.FileSystem fs, AcidUtils.HdfsDirSnapshot dirSnapshot) Checks if the files in base/delta dir are a result of Load Data/Add Partition statement and thus do not have ROW_IDs embedded in the data.
static boolean	isRawFormatFile(org.apache.hadoop.fs.Path dataFile, org.apache.hadoop.fs.FileSystem fs)

Methods inherited from class org.apache.hadoop.hive.common.AcidMetaDataFile

writeToFile

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MetaDataFile

public MetaDataFile()

Method Detail

isRawFormat

public static boolean isRawFormat(org.apache.hadoop.fs.Path baseOrDeltaDir,
                                  org.apache.hadoop.fs.FileSystem fs,
                                  AcidUtils.HdfsDirSnapshot dirSnapshot)
                           throws IOException

Checks if the files in base/delta dir are a result of Load Data/Add Partition statement and thus do not have ROW_IDs embedded in the data. This is only meaningful for full CRUD tables - Insert-only tables have all their data in raw format by definition.

Parameters:

baseOrDeltaDir - base or delta file.

dirSnapshot -

Throws:

IOException

isRawFormatFile

public static boolean isRawFormatFile(org.apache.hadoop.fs.Path dataFile,
                                      org.apache.hadoop.fs.FileSystem fs)
                               throws IOException

Throws:

IOException