org.apache.hadoop.hive.ql.metadata

Interface HiveStorageHandler

All Superinterfaces:

org.apache.hadoop.conf.Configurable

All Known Implementing Classes:

DefaultStorageHandler, SyslogStorageHandler

@InterfaceAudience.Public
 @InterfaceStability.Stable
public interface HiveStorageHandler
extends org.apache.hadoop.conf.Configurable

HiveStorageHandler defines a pluggable interface for adding new storage handlers to Hive. A storage handler consists of a bundle of the following:

• input format
• output format
• serde
• metadata hooks for keeping an external catalog in sync with Hive's metastore
• rules for setting up the configuration properties on map/reduce jobs which access tables stored by this handler

Storage handler classes are plugged in using the STORED BY 'classname' clause in CREATE TABLE.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	HiveStorageHandler.AcidSupportType

Field Summary

Fields

Modifier and Type	Field and Description
static List<AlterTableType>	DEFAULT_ALLOWED_ALTER_OPS

Method Summary

Modifier and Type	Method and Description
default List<org.apache.hadoop.hive.metastore.api.FieldSchema>	acidSelectColumns(Table table, Context.Operation operation) UpdateSemanticAnalyzer rewrites UPDATE and DeleteSemanticAnalyzer rewrites DELETE queries into INSERT queries.
default List<org.apache.hadoop.hive.metastore.api.FieldSchema>	acidSortColumns(Table table, Context.Operation operation) UpdateSemanticAnalyzer rewrites UPDATE and DeleteSemanticAnalyzer rewrites DELETE queries into INSERT queries.
default List<VirtualColumn>	acidVirtualColumns() Specifies which additional virtual columns should be added to the virtual column registry during compilation for tables that support ACID operations.
default boolean	addDynamicSplitPruningEdge(Table table, ExprNodeDesc syntheticFilterPredicate) Test if the storage handler allows the push-down of join filter predicate to prune further the splits.
default void	addResourcesForCreateTable(Map<String,String> tblProps, HiveConf hiveConf) Adds specific configurations to session for create table command.
default void	alterTableSnapshotRefOperation(Table table, AlterTableSnapshotRefSpec alterTableSnapshotRefSpec)
default boolean	alwaysUnpartitioned() Check if partition columns should be removed and added to the list of regular columns in HMS.
default void	appendFiles(org.apache.hadoop.hive.metastore.api.Table tbl, URI fromURI, boolean isOverwrite, Map<String,String> partitionSpec) Appends files to the table
default boolean	areSnapshotsSupported() Gets whether this storage handler supports snapshots.
default boolean	canComputeQueryUsingStats(Table tbl) Check if the storage handler answer a few queries like count(1) purely using stats.
default boolean	canPerformMetadataDelete(Table hmsTable, String branchName, SearchArgument searchArgument)
default boolean	canProvideBasicStatistics() Check if the storage handler can provide basic statistics.
default boolean	canProvideColStatistics(Table tbl) Check if the storage handler can provide col statistics.
default boolean	canSetColStatistics(Table tbl) Check if the storage handler can set col statistics.
default boolean	canUseTruncate(Table hmsTable, Map<String,String> partitionSpec)
default Table	checkAndSetTableMetaRef(Table hmsTable, String tableMetaRef)
default boolean	commitInMoveTask() Checks if we should keep the MoveTask and use the storageHandlerCommit(Properties, boolean) method for committing inserts instead of DefaultHiveMetaHook.commitInsertTable(Table, boolean).
void	configureInputJobCredentials(TableDesc tableDesc, Map<String,String> secrets) This method is called to allow the StorageHandlers the chance to populate secret keys into the job's credentials.
void	configureInputJobProperties(TableDesc tableDesc, Map<String,String> jobProperties) This method is called to allow the StorageHandlers the chance to populate the JobContext.getConfiguration() with properties that maybe be needed by the handler's bundled artifacts (ie InputFormat, SerDe, etc).
void	configureJobConf(TableDesc tableDesc, org.apache.hadoop.mapred.JobConf jobConf) Called just before submitting MapReduce job.
void	configureOutputJobProperties(TableDesc tableDesc, Map<String,String> jobProperties) This method is called to allow the StorageHandlers the chance to populate the JobContext.getConfiguration() with properties that maybe be needed by the handler's bundled artifacts (ie InputFormat, SerDe, etc).
void	configureTableJobProperties(TableDesc tableDesc, Map<String,String> jobProperties) Deprecated.
default DynamicPartitionCtx	createDPContext(HiveConf conf, Table table, Context.Operation writeOperation) Creates a DynamicPartitionCtx instance that will be set up by the storage handler itself.
default boolean	directInsert() Check if CTAS and CMV operations should behave in a direct-insert manner (i.e.
default void	executeOperation(Table table, AlterTableExecuteSpec executeSpec) Execute an operation on storage handler level
HiveAuthorizationProvider	getAuthorizationProvider() Returns the implementation specific authorization provider
default Map<String,String>	getBasicStatistics(Partish partish) Return some basic statistics (numRows, numFiles, totalSize) calculated by the underlying storage handler implementation.
default List<org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj>	getColStatistics(Table table) Return some col statistics (Lower bounds, Upper bounds, Null value counts, NaN, total counts) calculated by the underlying storage handler implementation.
default ColumnInfo	getColumnInfo(Table hmsTable, String colName)
default SnapshotContext	getCurrentSnapshotContext(Table table) Query the most recent unique snapshot's context of the passed table.
default String	getFileFormatPropertyKey() Get file format property key, if the file format is configured through a table property.
Class<? extends org.apache.hadoop.mapred.InputFormat>	getInputFormatClass()
default org.apache.hadoop.hive.metastore.api.LockType	getLockType(WriteEntity writeEntity)
HiveMetaHook	getMetaHook()
default Map<String,String>	getNativeProperties(Table table) Extract the native properties of the table which aren't stored in the HMS
default Map<String,String>	getOperatorDescProperties(OperatorDesc operatorDesc, Map<String,String> initialProps) Used to add additional operator specific information from storage handler during DESCRIBE EXTENDED statement.
Class<? extends org.apache.hadoop.mapred.OutputFormat>	getOutputFormatClass()
default List<org.apache.hadoop.hive.metastore.api.FieldSchema>	getPartitionKeys(Table hmsTable)
default List<String>	getPartitionNames(Table hmsTable, Map<String,String> partitionSpec)
default List<Partition>	getPartitionsByExpr(Table hmsTable, ExprNodeDesc desc)
default List<TransformSpec>	getPartitionTransformSpec(Table table) Return a list of partition transform specifications.
default org.apache.hadoop.hive.metastore.api.FieldSchema	getRowId()
Class<? extends AbstractSerDe>	getSerDeClass()
default StorageFormatDescriptor	getStorageFormatDescriptor(org.apache.hadoop.hive.metastore.api.Table table) Gets the storage format descriptor to be used for temp table for LOAD data.
default StorageHandlerInfo	getStorageHandlerInfo(org.apache.hadoop.hive.metastore.api.Table table) Used to fetch runtime information about storage handler during DESCRIBE EXTENDED statement
default StorageFormat.StorageHandlerTypes	getType()
default URI	getURIForAuth(org.apache.hadoop.hive.metastore.api.Table table) Constructs a URI for authorization purposes using the HMS table object
default Boolean	hasAppendsOnly(Table hmsTable, SnapshotContext since)
default boolean	isAllowedAlterOperation(AlterTableType opType) Checks whether a certain ALTER TABLE operation is supported by the storage handler implementation.
default boolean	isMetadataTableSupported() Deprecated. Use isTableMetaRefSupported()
default boolean	isTableMetaRefSupported() Check whether the table supports metadata references which mainly include branch, tag and metadata tables.
default boolean	isTimeTravelAllowed() Should return true if the StorageHandler is able to handle time travel.
default boolean	isValidMetadataTable(String metaTableName)
default void	prepareAlterTableEnvironmentContext(AbstractAlterTableDesc alterTableDesc, org.apache.hadoop.hive.metastore.api.EnvironmentContext environmentContext) Alter table operations can rely on this to customize the EnvironmentContext to be used during the alter table invocation (both on client and server side of HMS)
default boolean	setColStatistics(Table table, List<org.apache.hadoop.hive.metastore.api.ColumnStatistics> colStats) Set column stats for non-native tables
default void	setTableLocationForCTAS(CreateTableDesc desc, String location) Sets tables physical location at create table as select.
default void	setTableParametersForCTLT(Table tbl, CreateTableLikeDesc desc, Map<String,String> origParams) Retains storage handler specific properties during CTLT.
default boolean	shouldOverwrite(Table mTable, Context.Operation operation) Returns whether the data should be overwritten for the specific operation.
default List<String>	showPartitions(DDLOperationContext context, Table tbl) Checks if storage handler supports Show Partitions and returns a list of partitions
default List<org.apache.hadoop.hive.metastore.api.FieldSchema>	sortColumns(Table table) Collect the columns that are used to sort the content of the data files
default void	storageHandlerCommit(Properties commitProperties, boolean overwrite) Deprecated.
default void	storageHandlerCommit(Properties commitProperties, Context.Operation operation) Commits the inserts for the non-native tables.
default HiveStorageHandler.AcidSupportType	supportsAcidOperations() Specifies whether the table supports ACID operations or not (DELETE, UPDATE and MERGE statements).
default boolean	supportsAppendData(org.apache.hadoop.hive.metastore.api.Table table, boolean withPartClause) Checks whether the table supports appending data files to the table.
default boolean	supportsPartitionTransform() Check if the underlying storage handler implementation support partition transformations.
default boolean	supportsSortColumns() Check if the underlying storage handler implementation supports sort columns.
default boolean	supportsTruncateOnNonNativeTables() Check if the underlying storage handler implementation supports truncate operation for non native tables.
default void	validatePartSpec(Table hmsTable, Map<String,String> partitionSpec)
default void	validateSinkDesc(FileSinkDesc sinkDesc) Validates whether the sink operation is permitted for the specific storage handler, based on information contained in the sinkDesc.

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

DEFAULT_ALLOWED_ALTER_OPS

static final List<AlterTableType> DEFAULT_ALLOWED_ALTER_OPS

Method Detail

getInputFormatClass

Class<? extends org.apache.hadoop.mapred.InputFormat> getInputFormatClass()

Returns:

Class providing an implementation of InputFormat

getOutputFormatClass

Class<? extends org.apache.hadoop.mapred.OutputFormat> getOutputFormatClass()

Returns:

Class providing an implementation of OutputFormat

getSerDeClass

Class<? extends AbstractSerDe> getSerDeClass()

Returns:

Class providing an implementation of AbstractSerDe

getMetaHook

HiveMetaHook getMetaHook()

Returns:

metadata hook implementation, or null if this storage handler does not need any metadata notifications

getAuthorizationProvider

HiveAuthorizationProvider getAuthorizationProvider()
                                            throws HiveException

Returns the implementation specific authorization provider

Returns:

authorization provider

Throws:

HiveException

configureInputJobProperties

void configureInputJobProperties(TableDesc tableDesc,
                                 Map<String,String> jobProperties)

This method is called to allow the StorageHandlers the chance to populate the JobContext.getConfiguration() with properties that maybe be needed by the handler's bundled artifacts (ie InputFormat, SerDe, etc). Key value pairs passed into jobProperties are guaranteed to be set in the job's configuration object. User's can retrieve "context" information from tableDesc. User's should avoid mutating tableDesc and only make changes in jobProperties. This method is expected to be idempotent such that a job called with the same tableDesc values should return the same key-value pairs in jobProperties. Any external state set by this method should remain the same if this method is called again. It is up to the user to determine how best guarantee this invariant. This method in particular is to create a configuration for input.

Parameters:

tableDesc - descriptor for the table being accessed

jobProperties - receives properties copied or transformed from the table properties

configureInputJobCredentials

void configureInputJobCredentials(TableDesc tableDesc,
                                  Map<String,String> secrets)

This method is called to allow the StorageHandlers the chance to populate secret keys into the job's credentials.

configureOutputJobProperties

void configureOutputJobProperties(TableDesc tableDesc,
                                  Map<String,String> jobProperties)

This method is called to allow the StorageHandlers the chance to populate the JobContext.getConfiguration() with properties that maybe be needed by the handler's bundled artifacts (ie InputFormat, SerDe, etc). Key value pairs passed into jobProperties are guaranteed to be set in the job's configuration object. User's can retrieve "context" information from tableDesc. User's should avoid mutating tableDesc and only make changes in jobProperties. This method is expected to be idempotent such that a job called with the same tableDesc values should return the same key-value pairs in jobProperties. Any external state set by this method should remain the same if this method is called again. It is up to the user to determine how best guarantee this invariant. This method in particular is to create a configuration for output.

Parameters:

tableDesc - descriptor for the table being accessed

jobProperties - receives properties copied or transformed from the table properties

configureTableJobProperties

@Deprecated
void configureTableJobProperties(TableDesc tableDesc,
                                             Map<String,String> jobProperties)

Deprecated.

Deprecated use configureInputJobProperties/configureOutputJobProperties methods instead. Configures properties for a job based on the definition of the source or target table it accesses.

Parameters:

tableDesc - descriptor for the table being accessed

jobProperties - receives properties copied or transformed from the table properties

configureJobConf

void configureJobConf(TableDesc tableDesc,
                      org.apache.hadoop.mapred.JobConf jobConf)

Called just before submitting MapReduce job.

Parameters:

tableDesc - descriptor for the table being accessed

jobConf - jobConf for MapReduce job

getStorageHandlerInfo

default StorageHandlerInfo getStorageHandlerInfo(org.apache.hadoop.hive.metastore.api.Table table)
                                          throws org.apache.hadoop.hive.metastore.api.MetaException

Used to fetch runtime information about storage handler during DESCRIBE EXTENDED statement

Parameters:

table - table definition

Returns:

StorageHandlerInfo containing runtime information about storage handler OR `null` if the storage handler choose to not provide any runtime information.

Throws:

org.apache.hadoop.hive.metastore.api.MetaException

getType

default StorageFormat.StorageHandlerTypes getType()

getLockType

default org.apache.hadoop.hive.metastore.api.LockType getLockType(WriteEntity writeEntity)

addDynamicSplitPruningEdge

default boolean addDynamicSplitPruningEdge(Table table,
                                           ExprNodeDesc syntheticFilterPredicate)

Test if the storage handler allows the push-down of join filter predicate to prune further the splits.

Parameters:

table - The table to filter.

syntheticFilterPredicate - Join filter predicate.

Returns:

true if supports dynamic split pruning for the given predicate.

getOperatorDescProperties

default Map<String,String> getOperatorDescProperties(OperatorDesc operatorDesc,
                                                     Map<String,String> initialProps)

Used to add additional operator specific information from storage handler during DESCRIBE EXTENDED statement.

Parameters:

operatorDesc - operatorDesc

initialProps - Map containing initial operator properties

Returns:

Map<String, String> containing additional operator specific information from storage handler OR `initialProps` if the storage handler choose to not provide any such information.

getBasicStatistics

default Map<String,String> getBasicStatistics(Partish partish)

Return some basic statistics (numRows, numFiles, totalSize) calculated by the underlying storage handler implementation.

Parameters:

partish - a partish wrapper class

Returns:

map of basic statistics, can be null

canProvideBasicStatistics

default boolean canProvideBasicStatistics()

Check if the storage handler can provide basic statistics.

Returns:

true if the storage handler can supply the basic statistics

getColStatistics

default List<org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj> getColStatistics(Table table)

Return some col statistics (Lower bounds, Upper bounds, Null value counts, NaN, total counts) calculated by the underlying storage handler implementation.

Parameters:

table -

Returns:

A List of Column Statistics Objects, can be null

setColStatistics

default boolean setColStatistics(Table table,
                                 List<org.apache.hadoop.hive.metastore.api.ColumnStatistics> colStats)

Set column stats for non-native tables

Parameters:

table -

colStats -

Returns:

boolean

canProvideColStatistics

default boolean canProvideColStatistics(Table tbl)

Check if the storage handler can provide col statistics.

Parameters:

tbl -

Returns:

true if the storage handler can supply the col statistics

canSetColStatistics

default boolean canSetColStatistics(Table tbl)

Check if the storage handler can set col statistics.

Returns:

true if the storage handler can set the col statistics

canComputeQueryUsingStats

default boolean canComputeQueryUsingStats(Table tbl)

Check if the storage handler answer a few queries like count(1) purely using stats.

Returns:

true if the storage handler can answer query using statistics

getStorageFormatDescriptor

default StorageFormatDescriptor getStorageFormatDescriptor(org.apache.hadoop.hive.metastore.api.Table table)
                                                    throws SemanticException

Gets the storage format descriptor to be used for temp table for LOAD data.

Parameters:

table - table object

Returns:

StorageFormatDescriptor if the storage handler can support load data

Throws:

SemanticException

supportsAppendData

default boolean supportsAppendData(org.apache.hadoop.hive.metastore.api.Table table,
                                   boolean withPartClause)
                            throws SemanticException

Checks whether the table supports appending data files to the table.

Parameters:

table - the table

withPartClause - whether a partition is specified

Returns:

true if the table can append files directly to the table

Throws:

SemanticException - in case of any error.

appendFiles

default void appendFiles(org.apache.hadoop.hive.metastore.api.Table tbl,
                         URI fromURI,
                         boolean isOverwrite,
                         Map<String,String> partitionSpec)
                  throws SemanticException

Appends files to the table

Parameters:

tbl - the table object.

fromURI - the source of files.

isOverwrite - whether to overwrite the existing table data.

partitionSpec - the partition spec.

Throws:

SemanticException - in case of any error

directInsert

default boolean directInsert()

Check if CTAS and CMV operations should behave in a direct-insert manner (i.e. no move task).

Please note that the atomicity of the operation will suffer in this case, i.e. the created table might become exposed, depending on the implementation, before the CTAS or CMV operations finishes. Rollback (e.g. dropping the table) is also the responsibility of the storage handler in case of failures.

Returns:

whether direct insert CTAS or CMV is required

alwaysUnpartitioned

default boolean alwaysUnpartitioned()

Check if partition columns should be removed and added to the list of regular columns in HMS. This can be useful for non-native tables where the table format/layout differs from the standard Hive table layout, e.g. Iceberg tables. For these table formats, the partition column values are stored in the data files along with regular column values, therefore the object inspectors should include the partition columns as well. Any partitioning scheme provided via the standard HiveQL syntax will be honored but stored in someplace other than HMS, depending on the storage handler implementation.

Returns:

whether table should always be unpartitioned from the perspective of HMS

setTableParametersForCTLT

default void setTableParametersForCTLT(Table tbl,
                                       CreateTableLikeDesc desc,
                                       Map<String,String> origParams)

Retains storage handler specific properties during CTLT.

Parameters:

tbl - the table

desc - the table descriptor

origParams - the original table properties

setTableLocationForCTAS

default void setTableLocationForCTAS(CreateTableDesc desc,
                                     String location)

Sets tables physical location at create table as select. Some storage handlers requires specifying the location of tables others generates it internally.

getNativeProperties

default Map<String,String> getNativeProperties(Table table)

Extract the native properties of the table which aren't stored in the HMS

Parameters:

table - the table

Returns:

map with native table level properties

shouldOverwrite

default boolean shouldOverwrite(Table mTable,
                                Context.Operation operation)

Returns whether the data should be overwritten for the specific operation.

Parameters:

mTable - the table.

operation - operation type.

Returns:

if the data should be overwritten for the specified operation.

addResourcesForCreateTable

default void addResourcesForCreateTable(Map<String,String> tblProps,
                                        HiveConf hiveConf)

Adds specific configurations to session for create table command.

Parameters:

tblProps - table properties

hiveConf - configuration

supportsAcidOperations

default HiveStorageHandler.AcidSupportType supportsAcidOperations()

Specifies whether the table supports ACID operations or not (DELETE, UPDATE and MERGE statements). Possible return values:

• AcidSupportType.NONE - ACID operations are not supported
• AcidSupportType.WITH_TRANSACTIONS - ACID operations are supported, and must use a valid HiveTxnManager to wrap the operation in a transaction, like in the case of standard Hive ACID tables
• AcidSupportType.WITHOUT_TRANSACTIONS - ACID operations are supported, and there is no need for a HiveTxnManager to open/close transactions for the operation, i.e. org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager can be used

Returns:

the table's ACID support type

acidVirtualColumns

default List<VirtualColumn> acidVirtualColumns()

Specifies which additional virtual columns should be added to the virtual column registry during compilation for tables that support ACID operations. Should only return a non-empty list if supportsAcidOperations() returns something other NONE.

Returns:

the list of ACID virtual columns

acidSelectColumns

default List<org.apache.hadoop.hive.metastore.api.FieldSchema> acidSelectColumns(Table table,
                                                                                 Context.Operation operation)

UpdateSemanticAnalyzer rewrites UPDATE and DeleteSemanticAnalyzer rewrites DELETE queries into INSERT queries. - DELETE FROM T WHERE A = 32 is rewritten into INSERT INTO T SELECT <selectCols> FROM T WHERE A = 32 SORT BY <sortCols>. - UPDATE T SET B=12 WHERE A = 32 is rewritten into INSERT INTO T SELECT <selectCols>, <newValues> FROM T WHERE A = 32 SORT BY <sortCols>. This method specifies which columns should be injected into the <selectCols> part of the rewritten query. Should only return a non-empty list if supportsAcidOperations() returns something other NONE.

Parameters:

table - the table which is being deleted/updated/merged into

operation - the operation type we are executing

Returns:

the list of columns that should be projected in the rewritten ACID query

getRowId

default org.apache.hadoop.hive.metastore.api.FieldSchema getRowId()

acidSortColumns

default List<org.apache.hadoop.hive.metastore.api.FieldSchema> acidSortColumns(Table table,
                                                                               Context.Operation operation)

UpdateSemanticAnalyzer rewrites UPDATE and DeleteSemanticAnalyzer rewrites DELETE queries into INSERT queries. E.g. DELETE FROM T WHERE A = 32 is rewritten into INSERT INTO T SELECT <selectCols> FROM T WHERE A = 32 SORT BY <sortCols>. This method specifies which columns should be injected into the <sortCols> part of the rewritten query. Should only return a non-empty list if supportsAcidOperations() returns something other NONE.

Parameters:

table - the table which is being deleted/updated/merged into

operation - the operation type we are executing

Returns:

the list of columns that should be used as sort columns in the rewritten ACID query

supportsSortColumns

default boolean supportsSortColumns()

Check if the underlying storage handler implementation supports sort columns.

Returns:

true if the storage handler can support it

sortColumns

default List<org.apache.hadoop.hive.metastore.api.FieldSchema> sortColumns(Table table)

Collect the columns that are used to sort the content of the data files

Parameters:

table - the table which is being sorted

Returns:

the list of columns that are used during data sorting

supportsPartitionTransform

default boolean supportsPartitionTransform()

Check if the underlying storage handler implementation support partition transformations.

Returns:

true if the storage handler can support it

getPartitionTransformSpec

default List<TransformSpec> getPartitionTransformSpec(Table table)

Return a list of partition transform specifications. This method should be overwritten in case supportsPartitionTransform() returns true.

Parameters:

table - the HMS table, must be non-null

Returns:

partition transform specification, can be null.

createDPContext

default DynamicPartitionCtx createDPContext(HiveConf conf,
                                            Table table,
                                            Context.Operation writeOperation)
                                     throws SemanticException

Creates a DynamicPartitionCtx instance that will be set up by the storage handler itself. Useful for non-native tables where partitions are not handled by Hive, and sorting is required in a custom way before writing the table.

Parameters:

conf - job conf

table - the HMS table

Returns:

the created DP context object, null if DP context / sorting is not required

Throws:

SemanticException

getFileFormatPropertyKey

default String getFileFormatPropertyKey()

Get file format property key, if the file format is configured through a table property.

Returns:

table property key, can be null

commitInMoveTask

default boolean commitInMoveTask()

Checks if we should keep the MoveTask and use the storageHandlerCommit(Properties, boolean) method for committing inserts instead of DefaultHiveMetaHook.commitInsertTable(Table, boolean).

Returns:

Returns true if we should use the storageHandlerCommit(Properties, boolean) method

storageHandlerCommit

default void storageHandlerCommit(Properties commitProperties,
                                  Context.Operation operation)
                           throws HiveException

Commits the inserts for the non-native tables. Used in the MoveTask.

Parameters:

commitProperties - Commit properties which are needed for the handler based commit

operation - the operation type

Throws:

HiveException - If there is an error during commit

storageHandlerCommit

@Deprecated
default void storageHandlerCommit(Properties commitProperties,
                                              boolean overwrite)
                                       throws HiveException

Deprecated.

Throws:

HiveException

isAllowedAlterOperation

default boolean isAllowedAlterOperation(AlterTableType opType)

Checks whether a certain ALTER TABLE operation is supported by the storage handler implementation.

Parameters:

opType - The alter operation type (e.g. RENAME_COLUMNS)

Returns:

whether the operation is supported by the storage handler

supportsTruncateOnNonNativeTables

default boolean supportsTruncateOnNonNativeTables()

Check if the underlying storage handler implementation supports truncate operation for non native tables.

Returns:

true if the storage handler can support it

isTimeTravelAllowed

default boolean isTimeTravelAllowed()

Should return true if the StorageHandler is able to handle time travel.

Returns:

True if time travel is allowed

isMetadataTableSupported

@Deprecated
default boolean isMetadataTableSupported()

Deprecated. Use isTableMetaRefSupported()

Introduced by HIVE-25457 for iceberg to query metadata table.

Returns:

true if the storage handler can support it

isTableMetaRefSupported

default boolean isTableMetaRefSupported()

Check whether the table supports metadata references which mainly include branch, tag and metadata tables.

Returns:

true if the storage handler can support it

isValidMetadataTable

default boolean isValidMetadataTable(String metaTableName)

checkAndSetTableMetaRef

default Table checkAndSetTableMetaRef(Table hmsTable,
                                      String tableMetaRef)
                               throws SemanticException

Throws:

SemanticException

getURIForAuth

default URI getURIForAuth(org.apache.hadoop.hive.metastore.api.Table table)
                   throws URISyntaxException

Constructs a URI for authorization purposes using the HMS table object

Parameters:

table - The HMS table object

Returns:

the URI for authorization

Throws:

URISyntaxException

validateSinkDesc

default void validateSinkDesc(FileSinkDesc sinkDesc)
                       throws SemanticException

Validates whether the sink operation is permitted for the specific storage handler, based on information contained in the sinkDesc.

Parameters:

sinkDesc - The sink descriptor

Throws:

SemanticException - if the sink operation is not allowed

executeOperation

default void executeOperation(Table table,
                              AlterTableExecuteSpec executeSpec)

Execute an operation on storage handler level

Parameters:

executeSpec - operation specification

alterTableSnapshotRefOperation

default void alterTableSnapshotRefOperation(Table table,
                                            AlterTableSnapshotRefSpec alterTableSnapshotRefSpec)

areSnapshotsSupported

default boolean areSnapshotsSupported()

Gets whether this storage handler supports snapshots.

Returns:

true means snapshots are supported false otherwise

getCurrentSnapshotContext

default SnapshotContext getCurrentSnapshotContext(Table table)

Query the most recent unique snapshot's context of the passed table.

Parameters:

table - - Table which snapshot context should be returned.

Returns:

SnapshotContext wraps the snapshotId or null if no snapshot present.

prepareAlterTableEnvironmentContext

default void prepareAlterTableEnvironmentContext(AbstractAlterTableDesc alterTableDesc,
                                                 org.apache.hadoop.hive.metastore.api.EnvironmentContext environmentContext)

Alter table operations can rely on this to customize the EnvironmentContext to be used during the alter table invocation (both on client and server side of HMS)

Parameters:

alterTableDesc - the alter table desc (e.g.: AlterTableSetPropertiesDesc) containing the work to do

environmentContext - an existing EnvironmentContext created prior, now to be filled/amended

hasAppendsOnly

default Boolean hasAppendsOnly(Table hmsTable,
                               SnapshotContext since)

showPartitions

default List<String> showPartitions(DDLOperationContext context,
                                    Table tbl)
                             throws UnsupportedOperationException,
                                    HiveException

Checks if storage handler supports Show Partitions and returns a list of partitions

Returns:

List of partitions

Throws:

UnsupportedOperationException

HiveException

validatePartSpec

default void validatePartSpec(Table hmsTable,
                              Map<String,String> partitionSpec)
                       throws SemanticException

Throws:

SemanticException

canUseTruncate

default boolean canUseTruncate(Table hmsTable,
                               Map<String,String> partitionSpec)
                        throws SemanticException

Throws:

SemanticException

getPartitionNames

default List<String> getPartitionNames(Table hmsTable,
                                       Map<String,String> partitionSpec)
                                throws SemanticException

Throws:

SemanticException

getColumnInfo

default ColumnInfo getColumnInfo(Table hmsTable,
                                 String colName)
                          throws SemanticException

Throws:

SemanticException

canPerformMetadataDelete

default boolean canPerformMetadataDelete(Table hmsTable,
                                         String branchName,
                                         SearchArgument searchArgument)

getPartitionKeys

default List<org.apache.hadoop.hive.metastore.api.FieldSchema> getPartitionKeys(Table hmsTable)

getPartitionsByExpr

default List<Partition> getPartitionsByExpr(Table hmsTable,
                                            ExprNodeDesc desc)
                                     throws SemanticException

Throws:

SemanticException