ai.eloquent.raft

Class RaftLog

java.lang.Object

ai.eloquent.raft.RaftLog

public class RaftLog
extends java.lang.Object

There turns out to be an annoying amount of book-keeping around the logging abstraction in Raft, since it can be compacted, truncated, rewritten, committed, and queried in all sorts of strange ways. That made it an excellent candidate to become a class and get tested in isolation.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RaftLog.Snapshot This is used in truncating the logs when they grow too long.

Field Summary

Fields

Modifier and Type	Field and Description
java.util.Set<java.lang.String>	committedQuorumMembers LOG COMPACTION + MEMBERSHIP CHANGES: This is used for log compaction, so we can track membership at the time of a snapshot of the state.
static int	COMPACTION_LIMIT The number of log entries to keep in the log before compacting into a snapshot.
java.util.Optional<RaftLog.Snapshot>	snapshot LOG COMPACTION: This is used to backstop the log when we compact committed entries.
RaftStateMachine	stateMachine CORE: This is the state machine reference that the logs refer to.

Constructor Summary

Constructors

Constructor and Description
RaftLog(RaftStateMachine stateMachine, java.util.Collection<java.lang.String> initialConfiguration, java.util.concurrent.ExecutorService pool) Create a log from a state machine and initial configuration.

Method Summary

Modifier and Type	Method and Description
boolean	appendEntries(long prevLogIndex, long prevLogTerm, java.util.List<EloquentRaftProto.LogEntry> entries) This gets called from handleAppendEntriesRPC() in EloquentRaftMember.
RaftLog	copy() Create a medium-deep copy of this log.
java.util.concurrent.CompletableFuture<java.lang.Boolean>	createCommitFuture(long index, long term)
java.util.concurrent.CompletableFuture<java.lang.Boolean>	createCommitFuture(RaftLogEntryLocation location)
boolean	equals(java.lang.Object o)
RaftLog.Snapshot	forceSnapshot() This forces a log compaction step, where all the committed log entries are compacted away, and we backstop with a single snapshot of the state machine.
java.util.List<EloquentRaftProto.LogEntry>	getAllUncompressedEntries()
long	getCommitIndex() This gets called when preparing an AppendEntriesRPC, in order to send other members the This returns the commit index that we've applied up to.
java.util.Optional<java.util.List<EloquentRaftProto.LogEntry>>	getEntriesSinceInclusive(long startIndex) This gets called when preparing to call an AppendEntriesRPC, in order to send other members of the cluster the current view of the log.
long	getLastEntryIndex() This gets called when preparing to call an RequestVoteRPC, in order to prevent other members from voting for an out-of-date log.
long	getLastEntryTerm() This gets called when preparing to call an RequestVoteRPC, in order to prevent other members from voting for an out-of-date log.
java.util.Optional<java.lang.Long>	getPreviousEntryTerm(long index) This gets called when preparing to call an AppendEntriesRPC, in order to send other members of the cluster the current view of the log.
java.util.Set<java.lang.String>	getQuorumMembers() This returns the most up-to-date view of the cluster membership that we have.
boolean	getSafeToChangeMembership() This should only be called by leaders while appending to their log! Raft only allows one membership change to be uncommitted at a time.
int	hashCode()
boolean	installSnapshot(RaftLog.Snapshot snapshot, long now) This gets called from handleInstallSnapshotRPC() in EloquentRaftMember.
java.util.Optional<RaftLogEntryLocation>	lastConfigurationEntryLocation() Get the location of the most recent log entry.
static int	level() Get the current log level.
void	setCommitIndex(long leaderCommit, long now) If leaderCommit > commitIndex, set commitIndex = min(leaderCommit, index of last new entry)
static void	setLevel(int level) Explicitly set the log level.
static void	setLevel(java.lang.String level) Explicitly set the log level.
void	unsafeSetLog(java.util.List<EloquentRaftProto.LogEntry> log) WARNING: UNSAFE!!! This function violates the correctness principles of Raft, and should never be called in production code.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

COMPACTION_LIMIT

public static final int COMPACTION_LIMIT

The number of log entries to keep in the log before compacting into a snapshot.

See Also:

Constant Field Values

stateMachine

public final RaftStateMachine stateMachine

CORE: This is the state machine reference that the logs refer to.

snapshot

public java.util.Optional<RaftLog.Snapshot> snapshot

LOG COMPACTION: This is used to backstop the log when we compact committed entries.

committedQuorumMembers

public final java.util.Set<java.lang.String> committedQuorumMembers

LOG COMPACTION + MEMBERSHIP CHANGES: This is used for log compaction, so we can track membership at the time of a snapshot of the state.

Constructor Detail

RaftLog

public RaftLog(RaftStateMachine stateMachine,
               java.util.Collection<java.lang.String> initialConfiguration,
               java.util.concurrent.ExecutorService pool)

Create a log from a state machine and initial configuration.

Method Detail

setLevel

public static void setLevel(int level)

Explicitly set the log level. These are:

• 0: trace
• 1: debug
• 2: info
• 3: warn
• 4: error

setLevel

public static void setLevel(java.lang.String level)

Explicitly set the log level. These are:

• trace
• debug
• info
• warn
• error
• off

On an invalid level, we default to 'info'

level

public static int level()

Get the current log level.

copy

public RaftLog copy()

Create a medium-deep copy of this log. The state machine is not copied, and the futures are not copied, but the cluster configuration is.

getCommitIndex

public long getCommitIndex()

This gets called when preparing an AppendEntriesRPC, in order to send other members the This returns the commit index that we've applied up to.

getLastEntryIndex

public long getLastEntryIndex()

This gets called when preparing to call an RequestVoteRPC, in order to prevent other members from voting for an out-of-date log. This returns index of the last log entry, regardless of whether it's been committed or not

getLastEntryTerm

public long getLastEntryTerm()

This gets called when preparing to call an RequestVoteRPC, in order to prevent other members from voting for an out-of-date log. This returns index of the last log entry, regardless of whether it's been committed or not

getQuorumMembers

public java.util.Set<java.lang.String> getQuorumMembers()

This returns the most up-to-date view of the cluster membership that we have.

getAllUncompressedEntries

public java.util.List<EloquentRaftProto.LogEntry> getAllUncompressedEntries()

Returns:

all entries that haven't been snapshotted yet.

getEntriesSinceInclusive

public java.util.Optional<java.util.List<EloquentRaftProto.LogEntry>> getEntriesSinceInclusive(long startIndex)

This gets called when preparing to call an AppendEntriesRPC, in order to send other members of the cluster the current view of the log.

Parameters:

startIndex - the index (inclusive) to get the logs from

Returns:

all the log entries starting at a given log index. If we've already compacted startIndex, then we return empty.

getPreviousEntryTerm

public java.util.Optional<java.lang.Long> getPreviousEntryTerm(long index)

This gets called when preparing to call an AppendEntriesRPC, in order to send other members of the cluster the current view of the log.

Parameters:

index - the index to find the term of (if possible)

Returns:

the term, if this entry is in our logs

getSafeToChangeMembership

public boolean getSafeToChangeMembership()

This should only be called by leaders while appending to their log! Raft only allows one membership change to be uncommitted at a time. If we already have a membership change in the log, and it's uncommitted, this returns false. Otherwise, this returns true.

Returns:

if it's safe to add an uncommitted membership change to the log.

lastConfigurationEntryLocation

public java.util.Optional<RaftLogEntryLocation> lastConfigurationEntryLocation()

Get the location of the most recent log entry. Note that this does NOT search into snapshots -- this just takes entries in the actual log.

Returns:

The index of the most recent log entry.

createCommitFuture

public java.util.concurrent.CompletableFuture<java.lang.Boolean> createCommitFuture(long index,
                                                                                    long term)

See Also:

createCommitFuture(long, long, boolean)

createCommitFuture

public java.util.concurrent.CompletableFuture<java.lang.Boolean> createCommitFuture(RaftLogEntryLocation location)

See Also:

createCommitFuture(long, long)

setCommitIndex

public void setCommitIndex(long leaderCommit,
                           long now)

If leaderCommit > commitIndex, set commitIndex = min(leaderCommit, index of last new entry)

Parameters:

leaderCommit - the commit from the leader

appendEntries

public boolean appendEntries(long prevLogIndex,
                             long prevLogTerm,
                             java.util.List<EloquentRaftProto.LogEntry> entries)

This gets called from handleAppendEntriesRPC() in EloquentRaftMember. It handles verifying that an append command is sound, by checking prevLogIndex and prevLogTerm, and then applying it. 1. Reply false if log doesn't contain an entry at prevLogIndex whose term matches prevLogTerm 2. If an existing entry conflicts with a new one (same index but different terms), delete the existing entry and all that follow it. 3. Append any new entries not already in the log

Parameters:

prevLogIndex - the log index for the entry immediately preceding the entries to be appended

prevLogTerm - the log term for the entry immediately preceding the entries to be appended

entries - the entries to be appended

Returns:

true if the append command is in the log.

installSnapshot

public boolean installSnapshot(RaftLog.Snapshot snapshot,
                               long now)

This gets called from handleInstallSnapshotRPC() in EloquentRaftMember. It handles installing the snapshot and updating our own logs appropriately. 1. If lastIndex is larger than the latest Snapshot's, then save the snapshot file and Raft state (lastIndex, lastTerm, lastConfig). Discard any existing or partial snapshots 2. If existing log entry has the same index and term as lastIndex, discard log up through lastIndex (but retain following entries), and return. 3. Otherwise, discard the entire log 4. Reset state machine using snapshot contents, and load cluster config from the snapshot.

Parameters:

snapshot - the snapshot to install

now - the current time, in case we want to mock it

Returns:

true if the snapshot got committed

forceSnapshot

public RaftLog.Snapshot forceSnapshot()

This forces a log compaction step, where all the committed log entries are compacted away, and we backstop with a single snapshot of the state machine. The resulting snapshot is returned.

unsafeSetLog

public void unsafeSetLog(java.util.List<EloquentRaftProto.LogEntry> log)

WARNING: UNSAFE!!! This function violates the correctness principles of Raft, and should never be called in production code. But, it's very useful for unit testing bizarre states Raft may accidentally end up in.

Parameters:

log - The new log to set. The old log is cleared in favor of these entries

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object