org.apache.hadoop.hive.ql.parse

Class RewriteSemanticAnalyzer<T>

java.lang.Object

org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

org.apache.hadoop.hive.ql.parse.CalcitePlanner

org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer<T>

Direct Known Subclasses:

AcidExportSemanticAnalyzer, DeleteSemanticAnalyzer, MergeSemanticAnalyzer, UpdateSemanticAnalyzer

public abstract class RewriteSemanticAnalyzer<T>
extends CalcitePlanner

A subclass of the SemanticAnalyzer that just handles update, delete and merge statements. It works by rewriting the updates and deletes into insert statements (since they are actually inserts) and then doing some patch up to make them work as updates and deletes instead.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.CalcitePlanner

CalcitePlanner.ASTSearcher, CalcitePlanner.CalcitePlannerAction, CalcitePlanner.InputContext, CalcitePlanner.PreCboCtx

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

SemanticAnalyzer.GenericUDAFInfo, SemanticAnalyzer.MaterializationRebuildMode

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

BaseSemanticAnalyzer.AnalyzeRewriteContext, BaseSemanticAnalyzer.TableSpec

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG
protected boolean	useSuper

Fields inherited from class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

aliasToCTEs, analyzeRewrite, ast, autogenColAliasPrfxLbl, createVwDesc, defaultJoinMerge, disableJoinMerge, DUMMY_DATABASE, DUMMY_TABLE, forViewCreation, mvRebuildMode, opParseCtx, originalResultSchema, prunedPartitions, resultSchema, SUBQUERY_TAG_1, SUBQUERY_TAG_2, topOps, topOpsCopy, unparseTranslator, VALUES_TMP_TABLE_NAME_PREFIX, viewSelect

Fields inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

acidFileSinks, cacheUsage, cContext, columnAccessInfo, conf, console, ctx, db, fetchTask, idToTableNameMap, inputs, linfo, outputs, prepareQuery, queryProperties, queryState, rootTasks, STATIC_LOG, tableAccessInfo, transactionalInQuery, txnManager, updateColumnAccessInfo, userSuppliedFunctions

Method Summary

Modifier and Type	Method and Description
protected abstract void	analyze(ASTNode tree, Table table, ASTNode tableName)
void	analyzeInternal(ASTNode tree)
protected void	analyzeRewrittenTree(ASTNode rewrittenTree, Context rewrittenCtx)
protected void	checkValidSetClauseTarget(ASTNode colName, Table targetTable) Assert that we are not asked to update a bucketing column or partition column.
protected Map<String,ASTNode>	collectSetColumnsAndExpressions(ASTNode setClause, Set<String> setRCols, Table targetTable)
protected ASTNode	findLHSofAssignment(ASTNode assignment)
protected static Table	getTable(ASTNode tabRef, Hive db, boolean throwException)
protected abstract ASTNode	getTargetTableNode(ASTNode tree)
protected void	markReadEntityForUpdate() Walk through all our inputs and set them to note that this read is part of an update or a delete.
protected void	rewriteAndAnalyze(T statementData, String subQueryAlias)
protected void	setUpAccessControlInfoForUpdate(Table mTable, Map<String,ASTNode> setCols) For updates, we need to set the column access info so that it contains information on the columns we are updating.
protected void	updateOutputs(Table targetTable) SemanticAnalyzer will generate a WriteEntity for the target table since it doesn't know/check if the read and write are of the same table in "insert ...
protected void	validateTargetTable(Table mTable) Assert it supports Acid write.

Methods inherited from class org.apache.hadoop.hive.ql.parse.CalcitePlanner

createPlanner, createPlannerAction, fixUpAfterCbo, genLogicalPlan, genRexNode, getOptimizedSql, getTableObjectByName, resetCalciteConfiguration, warmup

Methods inherited from class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

addPartitionColsToInsert, addPartitionColsToInsert, allowOutputMultipleTimes, analyzeCreateView, compilePlan, enableColumnStatsCollecting, executeUnParseTranslations, forViewCreation, generateErrorMessage, genExprNodeDesc, genFileSinkPlan, getAcidAnalyzeTable, getAliasId, getAllInputs, getAllOutputs, getAllRootTasks, getColNameToDefaultValueMap, getColumnInternalName, getDefaultConstraints, getDummyTable, getGenericUDAFEvaluator, getGenericUDAFEvaluator2, getGenericUDAFInfo, getGenericUDAFInfo2, getInvalidResultCacheReason, getLoadFileWork, getLoadTableWork, getMaterializationValidationResult, getOpContext, getOriginalResultSchema, getParseContext, getQB, getResultSchema, getSinkOp, getTableObjectByName, getTargetTable, getTopOps, getViewAliasToInput, groupByDescModeToUDAFMode, hasGroupBySibling, init, isConstantParameterInAggregationParameters, isInsertInto, isSelectDistinct, isValidQueryCaching, parseSelect, processNoScanCommand, processTableColumnNames, reset, rewriteGroupingFunctionAST, saveViewDefinition, setAST, setBit, setInvalidResultCacheReason, setLoadFileWork, setLoadTableWork, setMaterializationValidationResult, startAnalysis, translateFieldDesc, updating, validate

Methods inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

addPropertyReadEntry, analyze, analyzeStoredAdDirs, charSetString, checkColumnName, constructListBucketingCtx, createFetchTask, createHiveDB, endAnalysis, escapeSQLString, findTabRefIdxs, getAcidDdlDesc, getAcidFileSinks, getAutoCommitValue, getCacheUsage, getCboInfo, getColumnAccessInfo, getColumnNames, getColumnNamesOrder, getColumns, getColumns, getColumns, getCtx, getDatabase, getDatabase, getDataConnector, getDataConnector, getDb, getDbTableNamePair, getFetchTask, getFullTableNameForSQL, getIdToTableNameMap, getInputs, getLineageInfo, getOutputs, getPartitionSpecs, getPartSpec, getProps, getQualifiedTableName, getQualifiedTableName, getQueryProperties, getQueryState, getRootTasks, getSimpleTableNameBase, getTable, getTable, getTable, getTable, getTable, getTable, getTableAccessInfo, getTableAlias, getTxnMgr, getTypeStringFromAST, getUnescapedName, getUnescapedName, getUnescapedUnqualifiedTableName, getUpdateColumnAccessInfo, getUserSuppliedFunctions, getValidatedPartSpec, hasTransactionalInQuery, initCtx, isPrepareQuery, isValidPrefixSpec, processOrderSpec, raiseWrongType, readProps, setAutoCommitValue, setCacheUsage, setColumnAccessInfo, setFetchTask, setLineageInfo, setTableAccessInfo, setUpdateColumnAccessInfo, skipAuthorization, stripQuotes, toReadEntity, toReadEntity, toReadEntity, toWriteEntity, toWriteEntity, toWriteEntity, tryQualifyPath, unescapeIdentifier, unescapeSQLString, validatePartColumnType, validatePartSpec, validateUnsupportedPartitionClause

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

useSuper

protected boolean useSuper

Method Detail

analyzeInternal

public void analyzeInternal(ASTNode tree)
                     throws SemanticException

Overrides:

analyzeInternal in class CalcitePlanner

Throws:

SemanticException

getTargetTableNode

protected abstract ASTNode getTargetTableNode(ASTNode tree)

analyze

protected abstract void analyze(ASTNode tree,
                                Table table,
                                ASTNode tableName)
                         throws SemanticException

Throws:

SemanticException

rewriteAndAnalyze

protected void rewriteAndAnalyze(T statementData,
                                 String subQueryAlias)
                          throws SemanticException

Throws:

SemanticException

analyzeRewrittenTree

protected void analyzeRewrittenTree(ASTNode rewrittenTree,
                                    Context rewrittenCtx)
                             throws SemanticException

Throws:

SemanticException

checkValidSetClauseTarget

protected void checkValidSetClauseTarget(ASTNode colName,
                                         Table targetTable)
                                  throws SemanticException

Assert that we are not asked to update a bucketing column or partition column.

Parameters:

colName - it's the A in "SET A = B"

Throws:

SemanticException

findLHSofAssignment

protected ASTNode findLHSofAssignment(ASTNode assignment)

collectSetColumnsAndExpressions

protected Map<String,ASTNode> collectSetColumnsAndExpressions(ASTNode setClause,
                                                              Set<String> setRCols,
                                                              Table targetTable)
                                                       throws SemanticException

Throws:

SemanticException

getTable

protected static Table getTable(ASTNode tabRef,
                                Hive db,
                                boolean throwException)
                         throws SemanticException

Parameters:

throwException - if false, return null if table doesn't exist, else throw

Throws:

SemanticException

markReadEntityForUpdate

protected void markReadEntityForUpdate()

Walk through all our inputs and set them to note that this read is part of an update or a delete.

setUpAccessControlInfoForUpdate

protected void setUpAccessControlInfoForUpdate(Table mTable,
                                               Map<String,ASTNode> setCols)

For updates, we need to set the column access info so that it contains information on the columns we are updating. (But not all the columns of the target table even though the rewritten query writes all columns of target table since that is an implementation detail).

validateTargetTable

protected void validateTargetTable(Table mTable)
                            throws SemanticException

Assert it supports Acid write.

Throws:

SemanticException

updateOutputs

protected void updateOutputs(Table targetTable)

SemanticAnalyzer will generate a WriteEntity for the target table since it doesn't know/check if the read and write are of the same table in "insert ... select ....". Since DbTxnManager uses Read/WriteEntity objects to decide which locks to acquire, we get more concurrency if we have changed the table WriteEntity to a set of partition WriteEntity objects based on ReadEntity objects computed for this table.