org.apache.tephra.hbase.coprocessor

Class TransactionProcessor

java.lang.Object

org.apache.tephra.hbase.coprocessor.TransactionProcessor

All Implemented Interfaces:

org.apache.hadoop.hbase.Coprocessor, org.apache.hadoop.hbase.coprocessor.RegionCoprocessor, org.apache.hadoop.hbase.coprocessor.RegionObserver

public class TransactionProcessor
extends Object
implements org.apache.hadoop.hbase.coprocessor.RegionObserver, org.apache.hadoop.hbase.coprocessor.RegionCoprocessor

org.apache.hadoop.hbase.coprocessor.RegionObserver coprocessor that handles server-side processing for transactions:

• applies filtering to exclude data from invalid and in-progress transactions
• overrides the scanner returned for flush and compaction to drop data written by invalidated transactions, or expired due to TTL.

In order to use this coprocessor for transactions, configure the class on any table involved in transactions, or on all user tables by adding the following to hbase-site.xml: <property> <name>hbase.coprocessor.region.classes</name> <value>org.apache.tephra.hbase.coprocessor.TransactionProcessor</value> </property>

HBase Get and Scan operations should have the current transaction serialized on to the operation as an attribute: Transaction t = ...; Get get = new Get(...); TransactionCodec codec = new TransactionCodec(); codec.addToOperation(get, t);

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

org.apache.hadoop.hbase.coprocessor.RegionObserver.MutationType

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.Coprocessor

org.apache.hadoop.hbase.Coprocessor.State

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	allowEmptyValues
protected Boolean	pruneEnable
protected boolean	readNonTxnData
protected Map<byte[],Long>	ttlByFamily
protected Long	txMaxLifetimeMillis

Fields inherited from interface org.apache.hadoop.hbase.Coprocessor

PRIORITY_HIGHEST, PRIORITY_LOWEST, PRIORITY_SYSTEM, PRIORITY_USER, VERSION

Constructor Summary

Constructors

Constructor and Description
TransactionProcessor()

Method Summary

Modifier and Type	Method and Description
protected org.apache.hadoop.hbase.regionserver.InternalScanner	createStoreScanner(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env, String action, TransactionVisibilityState snapshot, org.apache.hadoop.hbase.regionserver.InternalScanner scanner, org.apache.hadoop.hbase.regionserver.ScanType type)
protected void	ensureValidTxLifetime(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env, org.apache.hadoop.hbase.client.OperationWithAttributes op, Transaction tx) Make sure that the transaction is within the max valid transaction lifetime.
protected org.apache.hadoop.conf.Configuration	getConfiguration(org.apache.hadoop.hbase.CoprocessorEnvironment env) Fetch the Configuration that contains the properties required by the coprocessor.
Optional	getRegionObserver()
protected org.apache.hadoop.hbase.filter.Filter	getTransactionFilter(Transaction tx, org.apache.hadoop.hbase.regionserver.ScanType type, org.apache.hadoop.hbase.filter.Filter filter) Derived classes can override this method to customize the filter used to return data visible for the current transaction.
protected CacheSupplier<TransactionStateCache>	getTransactionStateCacheSupplier(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env)
protected void	initializePruneState(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env) Refresh the properties related to transaction pruning.
void	postCompact(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.StoreFile resultFile, org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker tracker, org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
void	postFlush(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker tracker)
org.apache.hadoop.hbase.regionserver.InternalScanner	preCompact(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.InternalScanner scanner, org.apache.hadoop.hbase.regionserver.ScanType scanType, org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker tracker, org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
void	preCompactScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.ScanType scanType, org.apache.hadoop.hbase.regionserver.ScanOptions options, org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker tracker, org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
void	preDelete(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Delete delete, org.apache.hadoop.hbase.wal.WALEdit edit, org.apache.hadoop.hbase.client.Durability durability)
org.apache.hadoop.hbase.regionserver.InternalScanner	preFlush(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.InternalScanner scanner, org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker tracker)
void	preFlushScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.ScanOptions options, org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker tracker)
void	preGetOp(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Get get, List<org.apache.hadoop.hbase.Cell> results)
void	prePut(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Put put, org.apache.hadoop.hbase.wal.WALEdit edit, org.apache.hadoop.hbase.client.Durability durability)
void	preScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.client.Scan scan)
protected void	resetPruneState() Stop and clear state related to pruning.
void	start(org.apache.hadoop.hbase.CoprocessorEnvironment e)
void	stop(org.apache.hadoop.hbase.CoprocessorEnvironment e)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

postAppend, postBatchMutate, postBatchMutateIndispensably, postBulkLoadHFile, postCheckAndDelete, postCheckAndPut, postClose, postCloseRegionOperation, postCommitStoreFile, postCompactSelection, postDelete, postExists, postFlush, postGetOp, postIncrement, postInstantiateDeleteTracker, postMemStoreCompaction, postMutationBeforeWAL, postOpen, postPut, postReplayWALs, postScannerClose, postScannerFilterRow, postScannerNext, postScannerOpen, postStartRegionOperation, postStoreFileReaderOpen, postWALRestore, preAppend, preAppendAfterRowLock, preBatchMutate, preBulkLoadHFile, preCheckAndDelete, preCheckAndDeleteAfterRowLock, preCheckAndPut, preCheckAndPutAfterRowLock, preClose, preCommitStoreFile, preCompactSelection, preExists, preFlush, preIncrement, preIncrementAfterRowLock, preMemStoreCompaction, preMemStoreCompactionCompact, preMemStoreCompactionCompactScannerOpen, preOpen, prePrepareTimeStampForDeleteVersion, preReplayWALs, preScannerClose, preScannerNext, preStoreFileReaderOpen, preStoreScannerOpen, preWALRestore

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.RegionCoprocessor

getBulkLoadObserver, getEndpointObserver

Methods inherited from interface org.apache.hadoop.hbase.Coprocessor

getServices

Field Detail

pruneEnable

protected volatile Boolean pruneEnable

txMaxLifetimeMillis

protected volatile Long txMaxLifetimeMillis

ttlByFamily

protected Map<byte[],Long> ttlByFamily

allowEmptyValues

protected boolean allowEmptyValues

readNonTxnData

protected boolean readNonTxnData

Constructor Detail

TransactionProcessor

public TransactionProcessor()

Method Detail

getRegionObserver

public Optional getRegionObserver()

Specified by:

getRegionObserver in interface org.apache.hadoop.hbase.coprocessor.RegionCoprocessor

start

public void start(org.apache.hadoop.hbase.CoprocessorEnvironment e)
           throws IOException

Specified by:

start in interface org.apache.hadoop.hbase.Coprocessor

Throws:

IOException

getConfiguration

@Nullable
protected org.apache.hadoop.conf.Configuration getConfiguration(org.apache.hadoop.hbase.CoprocessorEnvironment env)

Fetch the Configuration that contains the properties required by the coprocessor. By default, the HBase configuration is returned. This method will never return null in Tephra but the derived classes might do so if Configuration is not available temporarily (for example, if it is being fetched from a HBase Table.

Parameters:

env - RegionCoprocessorEnvironment of the Region to which the coprocessor is associated

Returns:

Configuration, can be null if it is not available

getTransactionStateCacheSupplier

protected CacheSupplier<TransactionStateCache> getTransactionStateCacheSupplier(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env)

stop

public void stop(org.apache.hadoop.hbase.CoprocessorEnvironment e)
          throws IOException

Specified by:

stop in interface org.apache.hadoop.hbase.Coprocessor

Throws:

IOException

preGetOp

public void preGetOp(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                     org.apache.hadoop.hbase.client.Get get,
                     List<org.apache.hadoop.hbase.Cell> results)
              throws IOException

Specified by:

preGetOp in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

prePut

public void prePut(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                   org.apache.hadoop.hbase.client.Put put,
                   org.apache.hadoop.hbase.wal.WALEdit edit,
                   org.apache.hadoop.hbase.client.Durability durability)
            throws IOException

Specified by:

prePut in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

preDelete

public void preDelete(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                      org.apache.hadoop.hbase.client.Delete delete,
                      org.apache.hadoop.hbase.wal.WALEdit edit,
                      org.apache.hadoop.hbase.client.Durability durability)
               throws IOException

Specified by:

preDelete in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

preScannerOpen

public void preScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                           org.apache.hadoop.hbase.client.Scan scan)
                    throws IOException

Specified by:

preScannerOpen in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

preFlushScannerOpen

public void preFlushScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                                org.apache.hadoop.hbase.regionserver.Store store,
                                org.apache.hadoop.hbase.regionserver.ScanOptions options,
                                org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker tracker)
                         throws IOException

Specified by:

preFlushScannerOpen in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

preFlush

public org.apache.hadoop.hbase.regionserver.InternalScanner preFlush(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                                                                     org.apache.hadoop.hbase.regionserver.Store store,
                                                                     org.apache.hadoop.hbase.regionserver.InternalScanner scanner,
                                                                     org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker tracker)
                                                              throws IOException

Specified by:

preFlush in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

postFlush

public void postFlush(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                      org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker tracker)
               throws IOException

Specified by:

postFlush in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

preCompactScannerOpen

public void preCompactScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                                  org.apache.hadoop.hbase.regionserver.Store store,
                                  org.apache.hadoop.hbase.regionserver.ScanType scanType,
                                  org.apache.hadoop.hbase.regionserver.ScanOptions options,
                                  org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker tracker,
                                  org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
                           throws IOException

Specified by:

preCompactScannerOpen in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

preCompact

public org.apache.hadoop.hbase.regionserver.InternalScanner preCompact(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                                                                       org.apache.hadoop.hbase.regionserver.Store store,
                                                                       org.apache.hadoop.hbase.regionserver.InternalScanner scanner,
                                                                       org.apache.hadoop.hbase.regionserver.ScanType scanType,
                                                                       org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker tracker,
                                                                       org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
                                                                throws IOException

Specified by:

preCompact in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

postCompact

public void postCompact(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                        org.apache.hadoop.hbase.regionserver.Store store,
                        org.apache.hadoop.hbase.regionserver.StoreFile resultFile,
                        org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker tracker,
                        org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
                 throws IOException

Specified by:

postCompact in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Throws:

IOException

createStoreScanner

protected org.apache.hadoop.hbase.regionserver.InternalScanner createStoreScanner(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env,
                                                                                  String action,
                                                                                  TransactionVisibilityState snapshot,
                                                                                  org.apache.hadoop.hbase.regionserver.InternalScanner scanner,
                                                                                  org.apache.hadoop.hbase.regionserver.ScanType type)
                                                                           throws IOException

Throws:

IOException

ensureValidTxLifetime

protected void ensureValidTxLifetime(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env,
                                     org.apache.hadoop.hbase.client.OperationWithAttributes op,
                                     @Nullable
                                     Transaction tx)
                              throws IOException

Make sure that the transaction is within the max valid transaction lifetime.

Parameters:

env - RegionCoprocessorEnvironment of the Region to which the coprocessor is associated

op - OperationWithAttributes HBase operation to access its attributes if required

tx - Transaction supplied by the

Throws:

org.apache.hadoop.hbase.DoNotRetryIOException - thrown if the transaction is older than the max lifetime of a transaction IOException throw if the value of max lifetime of transaction is unavailable

IOException

getTransactionFilter

protected org.apache.hadoop.hbase.filter.Filter getTransactionFilter(Transaction tx,
                                                                     org.apache.hadoop.hbase.regionserver.ScanType type,
                                                                     org.apache.hadoop.hbase.filter.Filter filter)

Derived classes can override this method to customize the filter used to return data visible for the current transaction.

Parameters:

tx - the current transaction to apply

type - the type of scan being performed

initializePruneState

protected void initializePruneState(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env)

Refresh the properties related to transaction pruning. This method needs to be invoked if there is change in the prune related properties after clearing the state by calling resetPruneState().

Parameters:

env - RegionCoprocessorEnvironment of this region

resetPruneState

protected void resetPruneState()

Stop and clear state related to pruning.