com.sleepycat.je.rep.impl.node

Class Feeder

java.lang.Object

com.sleepycat.je.rep.impl.node.Feeder

public final class Feeder
extends Object

There is an instance of a Feeder for each client that needs a replication stream. Either a master, or replica (providing feeder services) may establish a feeder. A feeder is created in response to a request from a Replica, and is shutdown either upon loss of connectivity, or upon a change in mastership. The protocol used to validate and negotiate a connection is synchronous, but once this phase has been completed, the communication between the feeder and replica is asynchronous. To handle the async communications, the feeder has two threads associated with it: 1) An output thread whose sole purpose is to pump log records (and if necessary heart beat requests) down to the replica as fast as the network will allow it 2) An input thread that listens for responses to transaction commits and heart beat responses.

The feeder maintains several statistics that provide information about the replication rate for each replica. By comparing this information to information about master replication maintained by the FeederTxns class, it is also possible to estimate the lag between replicas and the master.

The statistics facilities do not expect the set of available statistics to change dynamically. To handle recording statistics about the changing set of replicas, the statistics are represented as maps that associated node names with statistics. Each feeder adds individual statistics in these maps at startup, and removes them at shutdown time to make sure that the statistics in the map only reflect up-to-date information.

Some notes about the specific statistics:

replicaDelay

The difference between the commit times of the latest transaction committed on the master and the transaction most recently processed by the replica. The master timestamp comes from the lastCommitTimestamp statistic maintained by FeederTxns. The feeder determines the commit timestamp of the replica's most recently processed transaction by obtaining timestamps from commit records being sent to the replica, and noting the last one prior to sending a heartbeat. When a heartbeat response is received, if the latest replica VLSN included in the response is equal or greater to the one recorded when the heartbeat request was sent, then the delay is computed by comparing the commit timestamp for that most recently sent transaction with the timestamp of the master's latest transaction. Replicas can send heartbeat responses on their own, so comparing the VLSNs is necessary to make sure that the response matches the request. Note that this arrangement depends on the fact that the replica processes transactions and heartbeats in order, and only sends a heartbeat response once all preceding transactions have been processed. If the master processes transactions at a fast enough rate that additional transactions are generated while waiting for a heartbeat response, then the value of this statistic will not reach zero, but will represent the total time for sending a commit operation to the replica and receiving the associated response, including the roundtrip latency of the network and any time spent due to buffering of replication data.

replicaLastCommitTimestamp

The commit timestamp of the last transaction committed before the most recent heartbeat for which a heartbeat response has been received. This statistic represents the commit time on the master of the most recent data known to have been processed on the replica. It provides the information used for the replica component of the replicaDelay statistic.

replicaLastCommitVLSN

The VLSN of the committed transaction described for replicaLastCommitTimestamp. This statistic provides the information used for the replica component of the replicaVLSNLag statistic.

replicaVLSNLag

The difference between the VLSN of the latest transaction committed on the master and the one most recently processed by the replica. The master VLSN comes from the lastCommitVLSN statistic maintained by FeederTxns. This statistic is similar to replicaDelay, but provides information about the VLSN lag rather than the time delay.

replicaVLSNRate

An exponential moving average of the rate of change of the replicaLastCommitVLSN statistic over time, averaged over a 10 second time period. This statistic provides information about how quickly the replica is processing replication data, which can be used, along with the vlsnRate statistic maintained by FeederTxns, to estimate the amount of time it will take for the replica to catch up with the master.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Feeder.ExitException A marker exception that wraps the real exception.

Constructor Summary

Constructors

Constructor and Description
Feeder()

Method Summary

Modifier and Type	Method and Description
String	dumpState() For debugging and exception messages.
ArbiterFeederSource	getArbiterFeederSource()
StreamAuthenticator	getAuthenticator()
Channel	getChannel() Return the channel associated with this feeder.
FeederFilter	getFeederFilter()
VLSN	getFeederVLSN() Returns the next VLSN that will be sent to the replica.
StatGroup	getProtocolStats(StatsConfig config)
JEVersion	getReplicaJEVersion() Returns the JE version supported by the replica, or null if the value is not yet known.
NameIdPair	getReplicaNameIdPair()
RepNodeImpl	getReplicaNode() Returns a RepNodeImpl that describes the replica, or null if the value is not yet known.
VLSN	getReplicaTxnEndVLSN() Returns the latest commit VLSN that was acked by the replica, or NULL_VLSN if no commit was acked since the time the feeder was established.
RepNode	getRepNode()
TestHook<BinaryProtocol.Message>	getWriteMessageHook() Get the test hook to be called before sending a message using the protocol's write method.
void	initMasterFeederSource(VLSN startVLSN) Creates the MasterFeederSource, which must be done while all files in the VLSNIndex range are protected by syncup.
boolean	isShutdown()
void	makeSecurityCheckResponse(String err) Sends a security check response to client and if failure, wait for a grace period before throwing an exception to caller
boolean	needSecurityChecks()
void	setFeederFilter(FeederFilter filter)
static void	setInitialWriteMessageHook(TestHook<BinaryProtocol.Message> initialWriteMessageHook) Set the value of the write message hook that will be used for newly created feeders.
void	setWriteMessageHook(TestHook<BinaryProtocol.Message> writeMessageHook) Set a test hook that will be called before sending a message using the protocol's write method, supplying the hook with the message as an argument.
void	shutdown(Exception shutdownException) Shutdown the feeder, closing its channel and releasing its threads.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Feeder

public Feeder()

Method Detail

initMasterFeederSource

public void initMasterFeederSource(VLSN startVLSN)
                            throws IOException

Creates the MasterFeederSource, which must be done while all files in the VLSNIndex range are protected by syncup.

Throws:

IOException

getProtocolStats

public StatGroup getProtocolStats(StatsConfig config)

getRepNode

public RepNode getRepNode()

getReplicaNameIdPair

public NameIdPair getReplicaNameIdPair()

setFeederFilter

public void setFeederFilter(FeederFilter filter)

getFeederFilter

public FeederFilter getFeederFilter()

getReplicaTxnEndVLSN

public VLSN getReplicaTxnEndVLSN()

Returns the latest commit VLSN that was acked by the replica, or NULL_VLSN if no commit was acked since the time the feeder was established.

getFeederVLSN

public VLSN getFeederVLSN()

Returns the next VLSN that will be sent to the replica. It will return VLSN.NULL if the Feeder is in the process of being created and FeederReplicaSyncup has not yet happened.

getChannel

public Channel getChannel()

Return the channel associated with this feeder.

getReplicaJEVersion

public JEVersion getReplicaJEVersion()

Returns the JE version supported by the replica, or null if the value is not yet known.

Returns:

the replica JE version or null

getReplicaNode

public RepNodeImpl getReplicaNode()

Returns a RepNodeImpl that describes the replica, or null if the value is not yet known. The value will be non-null if the feeder handshake has completed successfully.

Returns:

the replica node or null

shutdown

public void shutdown(Exception shutdownException)

Shutdown the feeder, closing its channel and releasing its threads. May be called internally upon noticing a problem, or externally when the RepNode is shutting down.

isShutdown

public boolean isShutdown()

getArbiterFeederSource

public ArbiterFeederSource getArbiterFeederSource()

getAuthenticator

public StreamAuthenticator getAuthenticator()

dumpState

public String dumpState()

For debugging and exception messages.

setWriteMessageHook

public void setWriteMessageHook(TestHook<BinaryProtocol.Message> writeMessageHook)

Set a test hook that will be called before sending a message using the protocol's write method, supplying the hook with the message as an argument.

getWriteMessageHook

public TestHook<BinaryProtocol.Message> getWriteMessageHook()

Get the test hook to be called before sending a message using the protocol's write method.

setInitialWriteMessageHook

public static void setInitialWriteMessageHook(TestHook<BinaryProtocol.Message> initialWriteMessageHook)

Set the value of the write message hook that will be used for newly created feeders.

needSecurityChecks

public boolean needSecurityChecks()

makeSecurityCheckResponse

public void makeSecurityCheckResponse(String err)
                               throws ReplicationSecurityException

Sends a security check response to client and if failure, wait for a grace period before throwing an exception to caller

Parameters:

err - error message sent to client

Throws:

ReplicationSecurityException