org.apache.hadoop.hbase.master.procedure

Class ServerCrashProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>

org.apache.hadoop.hbase.procedure2.StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure

All Implemented Interfaces:

Comparable<Procedure>, ServerProcedureInterface

public class ServerCrashProcedure
extends StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>
implements ServerProcedureInterface

Handle crashed server. This is a port to ProcedureV2 of what used to be euphemistically called ServerShutdownHandler.

The procedure flow varies dependent on whether meta is assigned, if we are doing distributed log replay versus distributed log splitting, and if we are to split logs at all.

This procedure asks that all crashed servers get processed equally; we yield after the completion of each successful flow step. We do this so that we do not 'deadlock' waiting on a region assignment so we can replay edits which could happen if a region moved there are edits on two servers for replay.

TODO: ASSIGN and WAIT_ON_ASSIGN (at least) are not idempotent. Revisit when assign is pv2. TODO: We do not have special handling for system tables.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

StateMachineProcedure.Flow

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.ServerProcedureInterface

ServerProcedureInterface.ServerOperationType

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_RETRIES_ON_META
static int	DEFAULT_SHORT_WAIT_ON_META
static int	DEFAULT_WAIT_ON_RIT
static String	KEY_RETRIES_ON_META Configuration key to set how many retries to cycle before we give up on meta.
static String	KEY_SHORT_WAIT_ON_META Configuration key to set how long to wait in ms doing a quick check on meta state.
static String	KEY_WAIT_ON_RIT Configuration key to set how long to wait in ms on regions in transition.

Constructor Summary

Constructors

Constructor and Description
ServerCrashProcedure() Used when deserializing from a procedure store; we'll construct one of these then call deserializeStateData(InputStream).
ServerCrashProcedure(MasterProcedureEnv env, ServerName serverName, boolean shouldSplitWal, boolean carryingMeta) Call this constructor queuing up a Procedure.

Method Summary

Methods

Modifier and Type	Method and Description
protected boolean	abort(MasterProcedureEnv env)
protected boolean	acquireLock(MasterProcedureEnv env)
void	deserializeStateData(InputStream stream)
protected StateMachineProcedure.Flow	executeFromState(MasterProcedureEnv env, org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)
protected org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState	getInitialState()
ServerName	getServerName()
ServerProcedureInterface.ServerOperationType	getServerOperationType() Given an operation type we can take decisions about what to do with pending operations.
protected org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState	getState(int stateId)
protected int	getStateId(org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)
boolean	hasMetaTableRegion()
protected boolean	isYieldBeforeExecuteFromState(MasterProcedureEnv env, org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state) For this procedure, yield at end of each successful flow step so that all crashed servers can make progress rather than do the default which has each procedure running to completion before we move to the next.
protected void	releaseLock(MasterProcedureEnv env)
protected void	rollbackState(MasterProcedureEnv env, org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)
void	serializeStateData(OutputStream stream)
protected boolean	shouldWaitClientAck(MasterProcedureEnv env)
void	toStringClassDetails(StringBuilder sb)

Methods inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

addChildProcedure, execute, isYieldAfterExecutionStep, rollback, setNextState, toStringState

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, beforeReplay, childrenCountDown, compareTo, completionCleanup, convert, convert, createProcedureInfo, doExecute, doRollback, elapsedTime, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcId, getProcIdHashCode, getResult, getRootProcedureId, getStackIndexes, getStartTime, getState, getTimeout, getTimeRemaining, hasChildren, hasException, hasOwner, hasParent, hasTimeout, incChildrenLatch, isFailed, isFinished, isRunnable, isSuccess, isWaiting, newInstance, removeStackIndex, setAbortFailure, setChildrenLatch, setFailure, setFailure, setNonceKey, setOwner, setParentProcId, setProcId, setResult, setStackIndexes, setStartTime, setState, setTimeout, setTimeoutFailure, toString, toStringClass, toStringDetails, toStringSimpleSB, updateTimestamp, validateClass, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

KEY_SHORT_WAIT_ON_META

public static final String KEY_SHORT_WAIT_ON_META

Configuration key to set how long to wait in ms doing a quick check on meta state.

See Also:

Constant Field Values

DEFAULT_SHORT_WAIT_ON_META

public static final int DEFAULT_SHORT_WAIT_ON_META

See Also:

Constant Field Values

KEY_RETRIES_ON_META

public static final String KEY_RETRIES_ON_META

Configuration key to set how many retries to cycle before we give up on meta. Each attempt will wait at least KEY_SHORT_WAIT_ON_META milliseconds.

See Also:

Constant Field Values

DEFAULT_RETRIES_ON_META

public static final int DEFAULT_RETRIES_ON_META

See Also:

Constant Field Values

KEY_WAIT_ON_RIT

public static final String KEY_WAIT_ON_RIT

Configuration key to set how long to wait in ms on regions in transition.

See Also:

Constant Field Values

DEFAULT_WAIT_ON_RIT

public static final int DEFAULT_WAIT_ON_RIT

See Also:

Constant Field Values

Constructor Detail

ServerCrashProcedure

public ServerCrashProcedure(MasterProcedureEnv env,
                    ServerName serverName,
                    boolean shouldSplitWal,
                    boolean carryingMeta)

Call this constructor queuing up a Procedure.

Parameters:

serverName - Name of the crashed server.

shouldSplitWal - True if we should split WALs as part of crashed server processing.

carryingMeta - True if carrying hbase:meta table region.

ServerCrashProcedure

public ServerCrashProcedure()

Used when deserializing from a procedure store; we'll construct one of these then call deserializeStateData(InputStream). Do not use directly.

Method Detail

executeFromState

protected StateMachineProcedure.Flow executeFromState(MasterProcedureEnv env,
                                          org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)
                                               throws ProcedureYieldException

Specified by:

executeFromState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

Throws:

ProcedureYieldException

rollbackState

protected void rollbackState(MasterProcedureEnv env,
                 org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)
                      throws IOException

Specified by:

rollbackState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

Throws:

IOException

getState

protected org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState getState(int stateId)

Specified by:

getState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

getStateId

protected int getStateId(org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)

Specified by:

getStateId in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

getInitialState

protected org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState getInitialState()

Specified by:

getInitialState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

abort

protected boolean abort(MasterProcedureEnv env)

Specified by:

abort in class Procedure<MasterProcedureEnv>

acquireLock

protected boolean acquireLock(MasterProcedureEnv env)

Overrides:

acquireLock in class Procedure<MasterProcedureEnv>

releaseLock

protected void releaseLock(MasterProcedureEnv env)

Overrides:

releaseLock in class Procedure<MasterProcedureEnv>

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Overrides:

toStringClassDetails in class Procedure<MasterProcedureEnv>

serializeStateData

public void serializeStateData(OutputStream stream)
                        throws IOException

Overrides:

serializeStateData in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

Throws:

IOException

deserializeStateData

public void deserializeStateData(InputStream stream)
                          throws IOException

Overrides:

deserializeStateData in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

Throws:

IOException

getServerName

public ServerName getServerName()

Specified by:

getServerName in interface ServerProcedureInterface

Returns:

Name of this server instance.

hasMetaTableRegion

public boolean hasMetaTableRegion()

Specified by:

hasMetaTableRegion in interface ServerProcedureInterface

Returns:

True if this server has an hbase:meta table region.

getServerOperationType

public ServerProcedureInterface.ServerOperationType getServerOperationType()

Description copied from interface: ServerProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a crash handler and we have some assignment operation pending we can abort those operations.

Specified by:

getServerOperationType in interface ServerProcedureInterface

Returns:

the operation type that the procedure is executing.

isYieldBeforeExecuteFromState

protected boolean isYieldBeforeExecuteFromState(MasterProcedureEnv env,
                                    org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState state)

For this procedure, yield at end of each successful flow step so that all crashed servers can make progress rather than do the default which has each procedure running to completion before we move to the next. For crashed servers, especially if running with distributed log replay, we will want all servers to come along; we do not want the scenario where a server is stuck waiting for regions to online so it can replay edits.

Overrides:

isYieldBeforeExecuteFromState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.protobuf.generated.MasterProcedureProtos.ServerCrashState>

shouldWaitClientAck

protected boolean shouldWaitClientAck(MasterProcedureEnv env)

Overrides:

shouldWaitClientAck in class Procedure<MasterProcedureEnv>