org.apache.hadoop.hive.ql.parse

Class CacheTableHelper

java.lang.Object

org.apache.hadoop.hive.ql.parse.CacheTableHelper

public class CacheTableHelper
extends Object

Class to help populate the cache at the beginning of query analysis. We would like to minimize the number of calls to fetch validWriteIdLists from the metastore. HMS has an API to request this object for multiple tables within one call, and this class uses that API. The sole purpose of this class is to help populate the HMS query cache. Nothing is returned from the public methods. In this way, if another method attempts to fetch a validWriteIdList, the SessionHiveMetaStoreClient query cache will contain the information. Because this class is only responsible for cache population, it is not a requirement for a caller to supply all the tables necessary for the query. It is also not a requirement for the tables to be part of the query. Of course, the query qill benefit if those conditions were true, but if the table is not in the cache, a later call fetching the writeids will hit the HMS server and will not fail. One tricky aspect to this class is that if a view is passed in, we want to fetch the validWriteIdLists for the underlying tables. At the beginning of the query, it is impossible to know the underlying tables without contacting HMS. In order to handle the underlying tables to the views, we keep a cache that holds our best guess. If we see a view in any query, we set up a server-wide cache that tracks the underlying tables to the view. If the view doesn't change, then this information will be accurate and allow us to fetch the underlying tables on our next query. If the view does change and the underlying tables are different, our fetch won't retrieve the correct information. But that's ok...remember what was said earlier that it is not a requirement for the tables to be part of the query. Later on in the query, this class will be called on the view level via the populateCacheForView call. At that point, if something changed, it will populate the cache with the newly detected tables. It will also change the underying table information for the view to optimize the next query using the view.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
CacheTableHelper()

Method Summary

Modifier and Type	Method and Description
void	populateCache(List<org.apache.commons.lang3.tuple.Pair<String,String>> tables, HiveConf conf, HiveTxnManager txnMgr) Populates the cache for the given table pairs.
void	populateCacheForView(List<org.apache.commons.lang3.tuple.Pair<String,String>> tables, HiveConf conf, HiveTxnManager txnMgr, String dbName, String viewName) Populates the cache for the given table pairs associated with a viewName.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

CacheTableHelper

public CacheTableHelper()

Method Detail

populateCache

public void populateCache(List<org.apache.commons.lang3.tuple.Pair<String,String>> tables,
                          HiveConf conf,
                          HiveTxnManager txnMgr)

Populates the cache for the given table pairs. The tables passed in are a Pair containing the dbname (null if not given) and the table name.

populateCacheForView

public void populateCacheForView(List<org.apache.commons.lang3.tuple.Pair<String,String>> tables,
                                 HiveConf conf,
                                 HiveTxnManager txnMgr,
                                 String dbName,
                                 String viewName)

Populates the cache for the given table pairs associated with a viewName. If the table names provided match the table names that we think are associated with the view, we just return, because we presume that they have already been popuated via the "populateCache" method. If they are different, we populate the cache with the new associated tables and change our associated tables for the view.