ai.eloquent.raft

Class EloquentRaftAlgorithm

java.lang.Object

ai.eloquent.raft.EloquentRaftAlgorithm

All Implemented Interfaces:

RaftAlgorithm

public class EloquentRaftAlgorithm
extends java.lang.Object
implements RaftAlgorithm

The implementation of Raft, filling in the expected behavior for all of the callbacks.

Field Summary

Fields

Modifier and Type	Field and Description
static long	MACHINE_DOWN_TIMEOUT A very conservative timeout that defines when we consider a machine to be down.
RaftTransport	transport The underlying transport for this raft implementation

Fields inherited from interface ai.eloquent.raft.RaftAlgorithm

DEFAULT_ELECTION_RANGE, summaryTiming

Constructor Summary

Constructors

Constructor and Description
EloquentRaftAlgorithm(RaftState state, RaftTransport transport, java.util.Optional<RaftLifecycle> lifecycle) A constructor for unit tests primarily, that explicitly sets the state for Raft.
EloquentRaftAlgorithm(java.lang.String serverName, RaftStateMachine stateMachine, RaftTransport transport, java.util.Collection<java.lang.String> quorum, java.util.concurrent.ExecutorService raftStateThreadPool, java.util.Optional<RaftLifecycle> lifecycle) Create a new fixed-size Raft algorithm.
EloquentRaftAlgorithm(java.lang.String serverName, RaftStateMachine stateMachine, RaftTransport transport, int targetClusterSize, java.util.concurrent.ExecutorService raftStateThreadPool, java.util.Optional<RaftLifecycle> lifecycle) Create a new auto-resizing Raft algorithm.

Method Summary

Modifier and Type	Method and Description
boolean	bootstrap(boolean force) Mark this node for bootstrapping.
void	broadcastAppendEntries(long now) Send out an append entries RPC (i.e., a heartbeat) to all listeners on the transport.
boolean	equals(java.lang.Object o)
java.util.List<java.lang.String>	errors() Print any errors we can find in the Raft node.
RaftTransport	getTransport() This gets the RaftTransport associated with this RaftAlgorithm.
int	hashCode()
void	heartbeat(long now) A method to be called on every heartbeat interval.
boolean	isRunning() Check if the algorithm has already been stopped via RaftAlgorithm.stop(boolean).
java.util.Optional<RaftLifecycle>	lifecycle() Get the RaftLifecycle object that this Algorithm is registered for.
RaftState	mutableState() DANGEROUS: USE ONLY IF YOU KNOW WHAT YOU ARE DOING get A REFERENCE TO the current Raft state.
RaftStateMachine	mutableStateMachine() DANGEROUS: USE ONLY IF YOU KNOW WHAT YOU ARE DOING get A REFERENCE TO the current Raft state machine.
java.util.concurrent.CompletableFuture<EloquentRaftProto.RaftMessage>	receiveAddServerRPC(EloquentRaftProto.AddServerRequest addServerRequest, long now) A request has been received to add a server to the cluster.
void	receiveAppendEntriesReply(EloquentRaftProto.AppendEntriesReply reply, long now) We received an asynchronous heartbeat from a server.
void	receiveAppendEntriesRPC(EloquentRaftProto.AppendEntriesRequest heartbeat, java.util.function.Consumer<EloquentRaftProto.RaftMessage> replyLeader, long now) Receive a request to apply a transition or multiple transitions.
java.util.concurrent.CompletableFuture<EloquentRaftProto.RaftMessage>	receiveApplyTransitionRPC(EloquentRaftProto.ApplyTransitionRequest transition, long now) Apply a transition to Raft.
void	receiveBadRequest(EloquentRaftProto.RaftMessage message) A bad RPC call was received.
void	receiveInstallSnapshotReply(EloquentRaftProto.InstallSnapshotReply reply, long now) We received an asynchronous snapshot reply from a server.
void	receiveInstallSnapshotRPC(EloquentRaftProto.InstallSnapshotRequest snapshot, java.util.function.Consumer<EloquentRaftProto.RaftMessage> replyLeader, long now) Receive a snapshot request RPC call.
java.util.concurrent.CompletableFuture<EloquentRaftProto.RaftMessage>	receiveRemoveServerRPC(EloquentRaftProto.RemoveServerRequest removeServerRequest, long now) A request has been received to remove a server to the cluster.
void	receiveRequestVoteRPC(EloquentRaftProto.RequestVoteRequest voteRequest, java.util.function.Consumer<EloquentRaftProto.RaftMessage> replyLeader, long now) Clearly an election has been initiated, and some candidate is requesting votes to become the new leader.
void	receiveRequestVotesReply(EloquentRaftProto.RequestVoteReply reply, long now) We received votes from another server.
void	sendAppendEntries(java.lang.String target, long nextIndex) Send an append entries request to a particular server.
java.lang.String	serverName() The name of this server.
RaftState	state() get A COPY OF the current Raft state.
void	stop(boolean kill) A method, called primarily from unit tests, to stop this algorithm and clean up the underlying transport, if appropriate.
long	term() The term this node sees.
void	triggerElection(long now) Signal to the cluster that we are a candidate for an election, and we are soliciting votes

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ai.eloquent.raft.RaftAlgorithm

electionTimeoutMillisRange, heartbeatMillis, receiveMessage, receiveRPC, shutdown

Field Detail

MACHINE_DOWN_TIMEOUT

public static final long MACHINE_DOWN_TIMEOUT

A very conservative timeout that defines when we consider a machine to be down. This is used in (at least) two places:

1. Timing out membership changes, in the case that it takes us longer than this amount to get to the point of being able to add a configuration entry.
2. Removing a server from the configuration that has not been responding to heartbeats.

See Also:

Constant Field Values

transport

public final RaftTransport transport

The underlying transport for this raft implementation

Constructor Detail

EloquentRaftAlgorithm

public EloquentRaftAlgorithm(java.lang.String serverName,
                             RaftStateMachine stateMachine,
                             RaftTransport transport,
                             int targetClusterSize,
                             java.util.concurrent.ExecutorService raftStateThreadPool,
                             java.util.Optional<RaftLifecycle> lifecycle)

Create a new auto-resizing Raft algorithm. Note that this node is initially silent -- it will not initiate elections, and is not a voting member of its own cluster.

Parameters:

serverName - The name of this server. This MUST be unique within a cluster.

stateMachine - The state machine we are running on this Raft cluster.

transport - The underlying transport. This should match the cluster name argument.

targetClusterSize - The target size for the Raft cluster.

raftStateThreadPool - A pool to pass to RaftState for it to use

EloquentRaftAlgorithm

public EloquentRaftAlgorithm(java.lang.String serverName,
                             RaftStateMachine stateMachine,
                             RaftTransport transport,
                             java.util.Collection<java.lang.String> quorum,
                             java.util.concurrent.ExecutorService raftStateThreadPool,
                             java.util.Optional<RaftLifecycle> lifecycle)

Create a new fixed-size Raft algorithm. That is, a node that won't automatically resize -- you can still add or remove nodes manually.

Parameters:

serverName - The name of this server. This MUST be unique within a cluster.

stateMachine - The state machine we are running on this Raft cluster.

transport - The underlying transport. This should match the cluster name argument.

quorum - The configuration of the voting members of the cluster. The cluster will not automatically resize.

raftStateThreadPool - A pool to pass to RaftState for it to use

EloquentRaftAlgorithm

public EloquentRaftAlgorithm(RaftState state,
                             RaftTransport transport,
                             java.util.Optional<RaftLifecycle> lifecycle)

A constructor for unit tests primarily, that explicitly sets the state for Raft.

Method Detail

state

public RaftState state()

get A COPY OF the current Raft state.

Specified by:

state in interface RaftAlgorithm

mutableState

public RaftState mutableState()

DANGEROUS: USE ONLY IF YOU KNOW WHAT YOU ARE DOING get A REFERENCE TO the current Raft state.

Specified by:

mutableState in interface RaftAlgorithm

mutableStateMachine

public RaftStateMachine mutableStateMachine()

Description copied from interface: RaftAlgorithm

DANGEROUS: USE ONLY IF YOU KNOW WHAT YOU ARE DOING get A REFERENCE TO the current Raft state machine.

Specified by:

mutableStateMachine in interface RaftAlgorithm

term

public long term()

The term this node sees. This should simply return RaftState.currentTerm

Specified by:

term in interface RaftAlgorithm

serverName

public java.lang.String serverName()

The name of this server. This should simply return RaftState.serverName

Specified by:

serverName in interface RaftAlgorithm

receiveAppendEntriesRPC

public void receiveAppendEntriesRPC(EloquentRaftProto.AppendEntriesRequest heartbeat,
                                    java.util.function.Consumer<EloquentRaftProto.RaftMessage> replyLeader,
                                    long now)

Receive a request to apply a transition or multiple transitions. This also doubles as the heartbeat for the server. There are three cases here:

1. If the node is an Oligarch: The node should respond to the heartbeat.
2. If the node is a Shadow and the Oligarchy is full: The node should apply the transition and not reply.
3. If the node is a Shadow and the Oligarchy is not full: The node should apply the transition and respond to the heartbeat.

Specified by:

receiveAppendEntriesRPC in interface RaftAlgorithm

Parameters:

heartbeat - The request body, doubling both as a heartbeat and a request to mutate the state machine.

replyLeader - The method for replying to the leader with an ACK of the request.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

receiveAppendEntriesReply

public void receiveAppendEntriesReply(EloquentRaftProto.AppendEntriesReply reply,
                                      long now)

We received an asynchronous heartbeat from a server. This function is called to handle that heartbeat reply.

Specified by:

receiveAppendEntriesReply in interface RaftAlgorithm

Parameters:

reply - The heartbeat ACK from the follower node.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

sendAppendEntries

public void sendAppendEntries(java.lang.String target,
                              long nextIndex)

Send an append entries request to a particular server.

Specified by:

sendAppendEntries in interface RaftAlgorithm

Parameters:

target - The server we're sending the append entries RPC to.

nextIndex - The nextIndex value on the target machine.

broadcastAppendEntries

public void broadcastAppendEntries(long now)

Send out an append entries RPC (i.e., a heartbeat) to all listeners on the transport.

Specified by:

broadcastAppendEntries in interface RaftAlgorithm

Parameters:

now - The current time. This is usually System.currentTimeMillis(), but can be mocked by unit tests.

receiveInstallSnapshotRPC

public void receiveInstallSnapshotRPC(EloquentRaftProto.InstallSnapshotRequest snapshot,
                                      java.util.function.Consumer<EloquentRaftProto.RaftMessage> replyLeader,
                                      long now)

Receive a snapshot request RPC call. This should install a snapshot, and does not have to issue a response.

Specified by:

receiveInstallSnapshotRPC in interface RaftAlgorithm

Parameters:

snapshot - The snapshot to install.

replyLeader - The method for replying to the leader with an ACK of the request.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

receiveInstallSnapshotReply

public void receiveInstallSnapshotReply(EloquentRaftProto.InstallSnapshotReply reply,
                                        long now)

We received an asynchronous snapshot reply from a server.

Specified by:

receiveInstallSnapshotReply in interface RaftAlgorithm

Parameters:

reply - The snapshot reply from the follower node.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

receiveRequestVoteRPC

public void receiveRequestVoteRPC(EloquentRaftProto.RequestVoteRequest voteRequest,
                                  java.util.function.Consumer<EloquentRaftProto.RaftMessage> replyLeader,
                                  long now)

Clearly an election has been initiated, and some candidate is requesting votes to become the new leader. This must be responded to both by Oligarch and Shadow nodes.

Specified by:

receiveRequestVoteRPC in interface RaftAlgorithm

Parameters:

voteRequest - The request for votes for a particular server.

replyLeader - The method for replying to the leader with an ACK of the request.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

receiveRequestVotesReply

public void receiveRequestVotesReply(EloquentRaftProto.RequestVoteReply reply,
                                     long now)

We received votes from another server.

Specified by:

receiveRequestVotesReply in interface RaftAlgorithm

Parameters:

reply - The vote from the server, which we should count to see if we have a majority.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

triggerElection

public void triggerElection(long now)

Signal to the cluster that we are a candidate for an election, and we are soliciting votes

Specified by:

triggerElection in interface RaftAlgorithm

receiveAddServerRPC

public java.util.concurrent.CompletableFuture<EloquentRaftProto.RaftMessage> receiveAddServerRPC(EloquentRaftProto.AddServerRequest addServerRequest,
                                                                                                 long now)

A request has been received to add a server to the cluster. This is only well-defined for the leader -- if the request was not received by the leader, then it should forward to the leader.

The algorithm for this is the following:

1. Reply NOT_LEADER if not leader (in practice, we forward it to the leader).
2. Catch up server for a fixed number of rounds. Reply TIMEOUT if a server does not make progress for an election timeout or if the last round takes longer than the election timeout.
3. Wait until previous configuration in log is committed.
4. Append a new configuration entry to the log, (old configuration plus new server), commit it using a majority of the new configuration.
5. Reply OK.

Specified by:

receiveAddServerRPC in interface RaftAlgorithm

Parameters:

addServerRequest - The request to add the server.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

Returns:

the RPC response, as a future.

receiveRemoveServerRPC

public java.util.concurrent.CompletableFuture<EloquentRaftProto.RaftMessage> receiveRemoveServerRPC(EloquentRaftProto.RemoveServerRequest removeServerRequest,
                                                                                                    long now)

A request has been received to remove a server to the cluster. This is only well-defined for the leader -- if the request was not received by the leader, then it should forward to the leader.

Specified by:

receiveRemoveServerRPC in interface RaftAlgorithm

Parameters:

removeServerRequest - The snapshot to install.

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

Returns:

the RPC response, as a future.

receiveApplyTransitionRPC

public java.util.concurrent.CompletableFuture<EloquentRaftProto.RaftMessage> receiveApplyTransitionRPC(EloquentRaftProto.ApplyTransitionRequest transition,
                                                                                                       long now)

Apply a transition to Raft.

Specified by:

receiveApplyTransitionRPC in interface RaftAlgorithm

Parameters:

transition - The transition to apply

now - The timestamp of the current time. In non-mock cases, this is always System.currentTimeMillis().

heartbeat

public void heartbeat(long now)

A method to be called on every heartbeat interval. If we're the leader, this sends out heartbeats. If we're a follower, this ensures that we can trigger the appropriate timeout events.

Specified by:

heartbeat in interface RaftAlgorithm

bootstrap

public boolean bootstrap(boolean force)

Mark this node for bootstrapping. This should just be an alias for RaftState.bootstrap(boolean), with some error checks beforehand. If this node cannot bootstrap, we return false.

Specified by:

bootstrap in interface RaftAlgorithm

Returns:

True if we could bootstrap this node.

stop

public void stop(boolean kill)

A method, called primarily from unit tests, to stop this algorithm and clean up the underlying transport, if appropriate. An empty implementation is OK if there's nothing to stop.

Specified by:

stop in interface RaftAlgorithm

Parameters:

kill - If true, kill the server unceremoniously, without waiting for a handoff.

isRunning

public boolean isRunning()

Check if the algorithm has already been stopped via RaftAlgorithm.stop(boolean).

Specified by:

isRunning in interface RaftAlgorithm

receiveBadRequest

public void receiveBadRequest(EloquentRaftProto.RaftMessage message)

A bad RPC call was received. Signal the error appropriately. This can happen for two reasons:

1. The RPC type is not supported.
2. The proto of the RPC was poorly formatted.

Specified by:

receiveBadRequest in interface RaftAlgorithm

Parameters:

message - The raw message received.

lifecycle

public java.util.Optional<RaftLifecycle> lifecycle()

Description copied from interface: RaftAlgorithm

Get the RaftLifecycle object that this Algorithm is registered for.

Specified by:

lifecycle in interface RaftAlgorithm

getTransport

public RaftTransport getTransport()

Description copied from interface: RaftAlgorithm

This gets the RaftTransport associated with this RaftAlgorithm.

Specified by:

getTransport in interface RaftAlgorithm

errors

public java.util.List<java.lang.String> errors()

Print any errors we can find in the Raft node.

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object