org.apache.hadoop.hbase.master.snapshot

Class SnapshotManager

java.lang.Object

org.apache.hadoop.hbase.procedure.ProcedureManager

org.apache.hadoop.hbase.procedure.MasterProcedureManager

org.apache.hadoop.hbase.master.snapshot.SnapshotManager

All Implemented Interfaces:

Stoppable

@InterfaceAudience.LimitedPrivate(value="Configuration")
 @InterfaceStability.Unstable
public class SnapshotManager
extends MasterProcedureManager
implements Stoppable

This class manages the procedure of taking and restoring snapshots. There is only one SnapshotManager for the master.

The class provides methods for monitoring in-progress snapshot actions.

Note: Currently there can only be one snapshot being taken at a time over the cluster. This is a simplification in the current implementation.

Field Summary

Fields

Modifier and Type	Field and Description
static String	HBASE_SNAPSHOT_ENABLED Enable or disable snapshot support
static String	HBASE_SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLIS Wait time before removing a finished sentinel from the in-progress map NOTE: This is used as a safety auto cleanup.
static String	ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION Name of the operation to use in the controller
static String	SNAPSHOT_MAX_FILE_SIZE_PRESERVE Conf key for preserving original max file size configs
static int	SNAPSHOT_POOL_THREADS_DEFAULT number of current operations running on the master
static String	SNAPSHOT_POOL_THREADS_KEY Conf key for # of threads used by the SnapshotManager thread pool
static String	SNAPSHOT_PROCEDURE_ENABLED Enable or disable snapshot procedure
static boolean	SNAPSHOT_PROCEDURE_ENABLED_DEFAULT
static long	SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLS_DEFAULT

Constructor Summary

Constructors

Constructor and Description
SnapshotManager()

Method Summary

Modifier and Type	Method and Description
ServerName	acquireSnapshotVerifyWorker(SnapshotVerifyProcedure procedure)
void	checkPermissions(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc, AccessChecker accessChecker, User user) Check for required permissions before executing the procedure.
void	checkSnapshotSupport() Throws an exception if snapshot operations (take a snapshot, restore, clone) are not supported.
void	deleteSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Delete the specified snapshot
void	execProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc) Execute a distributed procedure on cluster
Integer	getAvailableWorker(ServerName serverName)
List<org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription>	getCompletedSnapshots() Gets the list of all completed snapshots.
String	getProcedureSignature() Return the unique signature of the procedure.
ReadWriteLock	getTakingSnapshotLock()
void	initialize(MasterServices master, MetricsMaster metricsMaster) Initialize a globally barriered procedure for master.
boolean	isProcedureDone(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc) Check if the procedure is finished successfully
boolean	isSnapshotDone(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription expected) Check if the specified snapshot is done
boolean	isStopped()
boolean	isTableTakingAnySnapshot(TableName tableName)
boolean	isTakingAnySnapshot() The snapshot operation processing as following: 1.
boolean	isTakingSnapshot(TableName tableName) Check to see if the specified table has a snapshot in progress.
void	prepareWorkingDirectory(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Check to make sure that we are OK to run the passed snapshot.
void	registerSnapshotProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, long procId)
void	releaseSnapshotVerifyWorker(SnapshotVerifyProcedure procedure, ServerName worker, MasterProcedureScheduler scheduler)
long	restoreOrCloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot, NonceKey nonceKey, boolean restoreAcl, String customSFT) Restore or Clone the specified snapshot
void	setSnapshotHandlerForTesting(TableName tableName, SnapshotSentinel handler) Set the handler for the current snapshot
boolean	snapshotProcedureEnabled()
void	stop(String why)
void	takeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Take a snapshot based on the enabled/disabled state of the table.
long	takeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, long nonceGroup, long nonce)
void	unregisterSnapshotProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, long procId)

Methods inherited from class org.apache.hadoop.hbase.procedure.MasterProcedureManager

execProcedureWithRet

Methods inherited from class org.apache.hadoop.hbase.procedure.ProcedureManager

equals, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

HBASE_SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLIS

public static final String HBASE_SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLIS

Wait time before removing a finished sentinel from the in-progress map NOTE: This is used as a safety auto cleanup. The snapshot and restore handlers map entries are removed when a user asks if a snapshot or restore is completed. This operation is part of the HBaseAdmin snapshot/restore API flow. In case something fails on the client side and the snapshot/restore state is not reclaimed after a default timeout, the entry is removed from the in-progress map. At this point, if the user asks for the snapshot/restore status, the result will be snapshot done if exists or failed if it doesn't exists.

See Also:

Constant Field Values

SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLS_DEFAULT

public static final long SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLS_DEFAULT

See Also:

Constant Field Values

HBASE_SNAPSHOT_ENABLED

public static final String HBASE_SNAPSHOT_ENABLED

Enable or disable snapshot support

See Also:

Constant Field Values

ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION

public static final String ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION

Name of the operation to use in the controller

See Also:

Constant Field Values

SNAPSHOT_POOL_THREADS_KEY

public static final String SNAPSHOT_POOL_THREADS_KEY

Conf key for # of threads used by the SnapshotManager thread pool

See Also:

Constant Field Values

SNAPSHOT_POOL_THREADS_DEFAULT

public static final int SNAPSHOT_POOL_THREADS_DEFAULT

number of current operations running on the master

See Also:

Constant Field Values

SNAPSHOT_MAX_FILE_SIZE_PRESERVE

public static final String SNAPSHOT_MAX_FILE_SIZE_PRESERVE

Conf key for preserving original max file size configs

See Also:

Constant Field Values

SNAPSHOT_PROCEDURE_ENABLED

public static final String SNAPSHOT_PROCEDURE_ENABLED

Enable or disable snapshot procedure

See Also:

Constant Field Values

SNAPSHOT_PROCEDURE_ENABLED_DEFAULT

public static final boolean SNAPSHOT_PROCEDURE_ENABLED_DEFAULT

See Also:

Constant Field Values

Constructor Detail

SnapshotManager

public SnapshotManager()

Method Detail

getCompletedSnapshots

public List<org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription> getCompletedSnapshots()
                                                                                                                 throws IOException

Gets the list of all completed snapshots.

Returns:

list of SnapshotDescriptions

Throws:

IOException - File system exception

deleteSnapshot

public void deleteSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                    throws IOException

Delete the specified snapshot

Throws:

SnapshotDoesNotExistException - If the specified snapshot does not exist.

IOException - For filesystem IOExceptions

isSnapshotDone

public boolean isSnapshotDone(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription expected)
                       throws IOException

Check if the specified snapshot is done

Returns:

true if snapshot is ready to be restored, false if it is still being taken.

Throws:

IOException - IOException if error from HDFS or RPC

UnknownSnapshotException - if snapshot is invalid or does not exist.

isTakingSnapshot

public boolean isTakingSnapshot(TableName tableName)

Check to see if the specified table has a snapshot in progress. Currently we have a limitation only allowing a single snapshot per table at a time.

Parameters:

tableName - name of the table being snapshotted.

Returns:

true if there is a snapshot in progress on the specified table.

isTableTakingAnySnapshot

public boolean isTableTakingAnySnapshot(TableName tableName)

prepareWorkingDirectory

public void prepareWorkingDirectory(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                             throws HBaseSnapshotException

Check to make sure that we are OK to run the passed snapshot. Checks to make sure that we aren't already running a snapshot or restore on the requested table.

Parameters:

snapshot - description of the snapshot we want to start

Throws:

HBaseSnapshotException - if the filesystem could not be prepared to start the snapshot

getTakingSnapshotLock

public ReadWriteLock getTakingSnapshotLock()

isTakingAnySnapshot

public boolean isTakingAnySnapshot()

The snapshot operation processing as following:
1. Create a Snapshot Handler, and do some initialization;
2. Put the handler into snapshotHandlers
So when we consider if any snapshot is taking, we should consider both the takingSnapshotLock and snapshotHandlers;

Returns:

true to indicate that there're some running snapshots.

takeSnapshot

public void takeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                  throws IOException

Take a snapshot based on the enabled/disabled state of the table.

Throws:

HBaseSnapshotException - when a snapshot specific exception occurs.

IOException - when some sort of generic IO exception occurs.

takeSnapshot

public long takeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                         long nonceGroup,
                         long nonce)
                  throws IOException

Throws:

IOException

setSnapshotHandlerForTesting

public void setSnapshotHandlerForTesting(TableName tableName,
                                         SnapshotSentinel handler)

Set the handler for the current snapshot

Exposed for TESTING

Parameters:

handler - handler the master should use TODO get rid of this if possible, repackaging, modify tests.

restoreOrCloneSnapshot

public long restoreOrCloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot,
                                   NonceKey nonceKey,
                                   boolean restoreAcl,
                                   String customSFT)
                            throws IOException

Restore or Clone the specified snapshot

Parameters:

nonceKey - unique identifier to prevent duplicated RPC

Throws:

IOException

stop

public void stop(String why)

Specified by:

stop in interface Stoppable

isStopped

public boolean isStopped()

Specified by:

isStopped in interface Stoppable

checkSnapshotSupport

public void checkSnapshotSupport()
                          throws UnsupportedOperationException

Throws an exception if snapshot operations (take a snapshot, restore, clone) are not supported. Called at the beginning of snapshot() and restoreSnapshot() methods.

Throws:

UnsupportedOperationException - if snapshot are not supported

initialize

public void initialize(MasterServices master,
                       MetricsMaster metricsMaster)
                throws org.apache.zookeeper.KeeperException,
                       IOException,
                       UnsupportedOperationException

Description copied from class: MasterProcedureManager

Initialize a globally barriered procedure for master.

Specified by:

initialize in class MasterProcedureManager

Parameters:

master - Master service interface

Throws:

org.apache.zookeeper.KeeperException

IOException

UnsupportedOperationException

getProcedureSignature

public String getProcedureSignature()

Description copied from class: ProcedureManager

Return the unique signature of the procedure. This signature uniquely identifies the procedure. By default, this signature is the string used in the procedure controller (i.e., the root ZK node name for the procedure)

Specified by:

getProcedureSignature in class ProcedureManager

execProcedure

public void execProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc)
                   throws IOException

Description copied from class: MasterProcedureManager

Execute a distributed procedure on cluster

Overrides:

execProcedure in class MasterProcedureManager

Parameters:

desc - Procedure description

Throws:

IOException

checkPermissions

public void checkPermissions(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc,
                             AccessChecker accessChecker,
                             User user)
                      throws IOException

Description copied from class: MasterProcedureManager

Check for required permissions before executing the procedure.

Specified by:

checkPermissions in class MasterProcedureManager

Throws:

IOException - if permissions requirements are not met.

isProcedureDone

public boolean isProcedureDone(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc)
                        throws IOException

Description copied from class: MasterProcedureManager

Check if the procedure is finished successfully

Specified by:

isProcedureDone in class MasterProcedureManager

Parameters:

desc - Procedure description

Returns:

true if the specified procedure is finished successfully

Throws:

IOException

registerSnapshotProcedure

public void registerSnapshotProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                      long procId)

unregisterSnapshotProcedure

public void unregisterSnapshotProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                        long procId)

snapshotProcedureEnabled

public boolean snapshotProcedureEnabled()

acquireSnapshotVerifyWorker

public ServerName acquireSnapshotVerifyWorker(SnapshotVerifyProcedure procedure)
                                       throws ProcedureSuspendedException

Throws:

ProcedureSuspendedException

releaseSnapshotVerifyWorker

public void releaseSnapshotVerifyWorker(SnapshotVerifyProcedure procedure,
                                        ServerName worker,
                                        MasterProcedureScheduler scheduler)

getAvailableWorker

public Integer getAvailableWorker(ServerName serverName)