org.apache.hadoop.hive.ql.parse

Class MergeSemanticAnalyzer

java.lang.Object

org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

org.apache.hadoop.hive.ql.parse.CalcitePlanner

org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer

org.apache.hadoop.hive.ql.parse.MergeSemanticAnalyzer

Direct Known Subclasses:

SplitMergeSemanticAnalyzer

public class MergeSemanticAnalyzer
extends RewriteSemanticAnalyzer

A subclass of the SemanticAnalyzer that just handles merge statements. It works by rewriting the updates and deletes into insert statements (since they are actually inserts) and then doing some patch up to make them work as merges instead.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer

RewriteSemanticAnalyzer.ColumnAppender, RewriteSemanticAnalyzer.NativeAcidColumnAppender, RewriteSemanticAnalyzer.NonNativeAcidColumnAppender, RewriteSemanticAnalyzer.ReparseResult

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.CalcitePlanner

CalcitePlanner.ASTSearcher, CalcitePlanner.CalcitePlannerAction, CalcitePlanner.InputContext, CalcitePlanner.PreCboCtx

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

SemanticAnalyzer.GenericUDAFInfo, SemanticAnalyzer.MaterializationRebuildMode

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

BaseSemanticAnalyzer.AnalyzeRewriteContext, BaseSemanticAnalyzer.TableSpec

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer

DELETE_PREFIX, INDENT, LOG, SUB_QUERY_ALIAS, useSuper

Fields inherited from class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

aliasToCTEs, analyzeRewrite, ast, autogenColAliasPrfxLbl, createVwDesc, defaultJoinMerge, disableJoinMerge, DUMMY_DATABASE, DUMMY_TABLE, forViewCreation, mvRebuildDbName, mvRebuildMode, mvRebuildName, opParseCtx, originalResultSchema, prunedPartitions, resultSchema, SUBQUERY_TAG_1, SUBQUERY_TAG_2, topOps, topOpsCopy, unparseTranslator, VALUES_TMP_TABLE_NAME_PREFIX, viewSelect

Fields inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

acidFileSinks, cacheUsage, cContext, columnAccessInfo, conf, console, ctx, db, fetchTask, idToTableNameMap, inputs, linfo, outputs, prepareQuery, queryProperties, queryState, rootTasks, STATIC_LOG, tableAccessInfo, transactionalInQuery, txnManager, updateColumnAccessInfo

Method Summary

Modifier and Type	Method and Description
protected int	addDestNamePrefixOfUpdate(int insClauseIdx, Context rewrittenCtx) This sets the destination name prefix for update clause.
protected String	addWhereClauseOfUpdate(StringBuilder rewrittenQueryStr, String onClauseAsString, ASTNode whenMatchedUpdateClause, String deleteExtraPredicate)
protected boolean	allowOutputMultipleTimes()
void	analyze(ASTNode tree, Table targetTable, ASTNode tableNameNode)
protected void	analyzeMerge(ASTNode tree, Table targetTable, ASTNode targetNameNode) Here we take a Merge statement AST and generate a semantically equivalent multi-insert statement to execute.
protected boolean	enableColumnStatsCollecting()
protected ASTNode	getTargetTableNode(ASTNode tree)
protected ASTNode	getWhenClauseOperation(ASTNode whenClause)
protected String	handleDelete(ASTNode whenMatchedDeleteClause, StringBuilder rewrittenQueryStr, String onClauseAsString, String updateExtraPredicate, String hintStr, RewriteSemanticAnalyzer.ColumnAppender columnAppender)
protected String	handleUpdate(ASTNode whenMatchedUpdateClause, StringBuilder rewrittenQueryStr, String onClauseAsString, String deleteExtraPredicate, String hintStr, RewriteSemanticAnalyzer.ColumnAppender columnAppender, String targetName, List<String> values)

Methods inherited from class org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer

addColsToSelect, addColsToSelect, addColsToSelect, addPartitionColsAsValues, analyzeInternal, analyzeRewrittenTree, appendInsertBranch, appendSortBy, appendTarget, checkValidSetClauseTarget, collectSetColumnsAndExpressions, createRewrittenQueryStrBuilder, findLHSofAssignment, getColumnAppender, getMatchedText, getSimpleTableName, getTable, isAliased, markReadEntityForUpdate, parseRewrittenQuery, patchProjectionForUpdate, setUpAccessControlInfoForUpdate, updateOutputs, validateTargetTable

Methods inherited from class org.apache.hadoop.hive.ql.parse.CalcitePlanner

createPlanner, createPlannerAction, fixUpAfterCbo, genLogicalPlan, getOptimizedSql, getTableObjectByName, resetCalciteConfiguration, warmup

Methods inherited from class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

addPartitionColsToInsert, addPartitionColsToInsert, analyzeCreateView, compilePlan, executeUnparseTranlations, forViewCreation, generateErrorMessage, genExprNodeDesc, genFileSinkPlan, getAcidAnalyzeTable, getAliasId, getAllInputs, getAllOutputs, getAllRootTasks, getColNameToDefaultValueMap, getColumnInternalName, getDefaultConstraints, getDummyTable, getFullTableNameForSQL, getGenericUDAFEvaluator, getGenericUDAFEvaluator2, getGenericUDAFInfo, getGenericUDAFInfo2, getInvalidAutomaticRewritingMaterializationReason, getInvalidResultCacheReason, getLoadFileWork, getLoadTableWork, getOpContext, getOriginalResultSchema, getParseContext, getQB, getResultSchema, getSinkOp, getTableObjectByName, getTargetTable, getTopOps, getViewAliasToInput, groupByDescModeToUDAFMode, hasGroupBySibling, init, isConstantParameterInAggregationParameters, isInsertInto, isSelectDistinct, isValidAutomaticRewritingMaterialization, isValidQueryCaching, parseSelect, processNoScanCommand, processTableColumnNames, reset, rewriteGroupingFunctionAST, saveViewDefinition, setAST, setBit, setInvalidAutomaticRewritingMaterializationReason, setInvalidResultCacheReason, setLoadFileWork, setLoadTableWork, startAnalysis, translateFieldDesc, updating, validate

Methods inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

addPropertyReadEntry, analyze, analyzeStoredAdDirs, charSetString, checkColumnName, constructListBucketingCtx, createFetchTask, createHiveDB, endAnalysis, escapeSQLString, findTabRefIdxs, getAcidDdlDesc, getAcidFileSinks, getAutoCommitValue, getCacheUsage, getCboInfo, getColumnAccessInfo, getColumnNames, getColumnNamesOrder, getColumns, getColumns, getColumns, getCtx, getDatabase, getDatabase, getDataConnector, getDataConnector, getDb, getDbTableNamePair, getFetchTask, getIdToTableNameMap, getInputs, getLineageInfo, getOutputs, getPartitionSpecs, getPartSpec, getProps, getQualifiedTableName, getQualifiedTableName, getQueryProperties, getQueryState, getRootTasks, getSimpleTableNameBase, getTable, getTable, getTable, getTable, getTable, getTable, getTableAccessInfo, getTableAlias, getTxnMgr, getTypeStringFromAST, getUnescapedName, getUnescapedName, getUnescapedUnqualifiedTableName, getUpdateColumnAccessInfo, getValidatedPartSpec, hasTransactionalInQuery, initCtx, isPrepareQuery, isValidPrefixSpec, raiseWrongType, readProps, setAutoCommitValue, setCacheUsage, setColumnAccessInfo, setFetchTask, setLineageInfo, setTableAccessInfo, setUpdateColumnAccessInfo, skipAuthorization, stripQuotes, toReadEntity, toReadEntity, toReadEntity, toWriteEntity, toWriteEntity, toWriteEntity, tryQualifyPath, unescapeIdentifier, unescapeSQLString, validatePartColumnType, validatePartSpec, validateUnsupportedPartitionClause

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getTargetTableNode

protected ASTNode getTargetTableNode(ASTNode tree)

Specified by:

getTargetTableNode in class RewriteSemanticAnalyzer

analyze

public void analyze(ASTNode tree,
                    Table targetTable,
                    ASTNode tableNameNode)
             throws SemanticException

Specified by:

analyze in class RewriteSemanticAnalyzer

Throws:

SemanticException

analyzeMerge

protected void analyzeMerge(ASTNode tree,
                            Table targetTable,
                            ASTNode targetNameNode)
                     throws SemanticException

Here we take a Merge statement AST and generate a semantically equivalent multi-insert statement to execute. Each Insert leg represents a single WHEN clause. As much as possible, the new SQL statement is made to look like the input SQL statement so that it's easier to map Query Compiler errors from generated SQL to original one this way. The generated SQL is a complete representation of the original input for the same reason. In many places SemanticAnalyzer throws exceptions that contain (line, position) coordinates. If generated SQL doesn't have everything and is patched up later, these coordinates point to the wrong place.

Throws:

SemanticException

addDestNamePrefixOfUpdate

protected int addDestNamePrefixOfUpdate(int insClauseIdx,
                                        Context rewrittenCtx)

This sets the destination name prefix for update clause.

Parameters:

insClauseIdx - index of insert clause in the rewritten multi-insert represents the merge update clause.

rewrittenCtx - the Context stores the prefixes

Returns:

the number of prefixes set.

handleUpdate

protected String handleUpdate(ASTNode whenMatchedUpdateClause,
                              StringBuilder rewrittenQueryStr,
                              String onClauseAsString,
                              String deleteExtraPredicate,
                              String hintStr,
                              RewriteSemanticAnalyzer.ColumnAppender columnAppender,
                              String targetName,
                              List<String> values)

addWhereClauseOfUpdate

protected String addWhereClauseOfUpdate(StringBuilder rewrittenQueryStr,
                                        String onClauseAsString,
                                        ASTNode whenMatchedUpdateClause,
                                        String deleteExtraPredicate)

handleDelete

protected String handleDelete(ASTNode whenMatchedDeleteClause,
                              StringBuilder rewrittenQueryStr,
                              String onClauseAsString,
                              String updateExtraPredicate,
                              String hintStr,
                              RewriteSemanticAnalyzer.ColumnAppender columnAppender)

Parameters:

onClauseAsString - - because there is no clone() and we need to use in multiple places

updateExtraPredicate - - see notes at caller

getWhenClauseOperation

protected ASTNode getWhenClauseOperation(ASTNode whenClause)

allowOutputMultipleTimes

protected boolean allowOutputMultipleTimes()

Overrides:

allowOutputMultipleTimes in class SemanticAnalyzer

enableColumnStatsCollecting

protected boolean enableColumnStatsCollecting()

Overrides:

enableColumnStatsCollecting in class SemanticAnalyzer