org.apache.ratis.server.raftlog

Class RaftLog

java.lang.Object

org.apache.ratis.server.raftlog.RaftLog

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

MemoryRaftLog, SegmentedRaftLog

public abstract class RaftLog
extends Object
implements Closeable

Base class of RaftLog. Currently we provide two types of RaftLog implementation: 1. MemoryRaftLog: all the log entries are stored in memory. This is only used for testing. 2. Segmented RaftLog: the log entries are persisted on disk, and are stored in segments.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	RaftLog.EntryWithData Holds proto entry along with future which contains read state machine data
static class	RaftLog.Metadata
static class	RaftLogSequentialOps.Runner

Field Summary

Fields

Modifier and Type	Field and Description
static long	INVALID_LOG_INDEX
static long	LEAST_VALID_LOG_INDEX The least valid log index, i.e.
static org.slf4j.Logger	LOG
static String	LOG_SYNC

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	RaftLog(RaftGroupMemberId memberId, long commitIndex, RaftProperties properties)

Method Summary

Modifier and Type	Method and Description
long	append(long term, RaftConfiguration configuration) Append asynchronously a log entry for the given term and configuration Used by the leader.
long	append(long term, TransactionContext transaction) Append asynchronously a log entry for the given term and transaction.
List<CompletableFuture<Long>>	append(org.apache.ratis.proto.RaftProtos.LogEntryProto... entries) Append asynchronously all the given log entries.
CompletableFuture<Long>	appendEntry(org.apache.ratis.proto.RaftProtos.LogEntryProto entry) Append asynchronously an entry.
protected abstract CompletableFuture<Long>	appendEntryImpl(org.apache.ratis.proto.RaftProtos.LogEntryProto entry)
protected abstract List<CompletableFuture<Long>>	appendImpl(org.apache.ratis.proto.RaftProtos.LogEntryProto... entries)
long	appendMetadata(long term, long newCommitIndex) Append asynchronously a log entry for the given term and commit index unless the given commit index is an index of a metadata entry Used by the leader.
void	checkLogState()
void	close()
boolean	contains(TermIndex ti) Does the log contains the given term and index? Used to check the consistency between the local log of a follower and the log entries sent by the leader.
abstract org.apache.ratis.proto.RaftProtos.LogEntryProto	get(long index) Get the log entry of the given index.
abstract TermIndex[]	getEntries(long startIndex, long endIndex)
abstract RaftLog.EntryWithData	getEntryWithData(long index) Get the log entry of the given index along with the state machine data.
abstract long	getFlushIndex()
long	getLastCommittedIndex()
abstract TermIndex	getLastEntryTermIndex()
String	getName()
long	getNextIndex()
RaftLogMetrics	getRaftLogMetrics()
long	getSnapshotIndex()
abstract long	getStartIndex()
abstract TermIndex	getTermIndex(long index) Get the TermIndex information of the given index.
boolean	hasReadLock()
boolean	hasWriteLock()
abstract boolean	isConfigEntry(TermIndex ti)
boolean	isOpened()
abstract RaftLog.Metadata	loadMetadata()
void	open(long lastIndexInSnapshot, Consumer<org.apache.ratis.proto.RaftProtos.LogEntryProto> consumer)
protected void	openImpl(long lastIndexInSnapshot, Consumer<org.apache.ratis.proto.RaftProtos.LogEntryProto> consumer)
CompletableFuture<Long>	purge(long suggestedIndex) Purge asynchronously the log transactions.
protected abstract CompletableFuture<Long>	purgeImpl(long index)
AutoCloseableLock	readLock()
abstract void	syncWithSnapshot(long lastSnapshotIndex)
String	toLogEntryString(org.apache.ratis.proto.RaftProtos.LogEntryProto logEntry)
String	toString()
CompletableFuture<Long>	truncate(long index) Truncate asynchronously the log entries till the given index (inclusively).
protected abstract CompletableFuture<Long>	truncateImpl(long index)
boolean	updateLastCommitted(long majorityIndex, long currentTerm) Update the last committed index.
void	updateSnapshotIndex(long newSnapshotIndex) Update the last committed index and snapshotIndex with the last index in the snapshot.
protected void	validateLogEntry(org.apache.ratis.proto.RaftProtos.LogEntryProto entry) Validate the term and index of entry w.r.t RaftLog
AutoCloseableLock	writeLock()
abstract void	writeMetadata(long term, RaftPeerId votedFor) Write and flush the metadata (votedFor and term) into the meta file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

LOG_SYNC

public static final String LOG_SYNC

LEAST_VALID_LOG_INDEX

public static final long LEAST_VALID_LOG_INDEX

The least valid log index, i.e. the index used when writing to an empty log.

See Also:

Constant Field Values

INVALID_LOG_INDEX

public static final long INVALID_LOG_INDEX

See Also:

Constant Field Values

Constructor Detail

RaftLog

protected RaftLog(RaftGroupMemberId memberId,
                  long commitIndex,
                  RaftProperties properties)

Method Detail

getRaftLogMetrics

public RaftLogMetrics getRaftLogMetrics()

getLastCommittedIndex

public long getLastCommittedIndex()

getSnapshotIndex

public long getSnapshotIndex()

checkLogState

public void checkLogState()

isOpened

public boolean isOpened()

updateLastCommitted

public boolean updateLastCommitted(long majorityIndex,
                                   long currentTerm)

Update the last committed index.

Parameters:

majorityIndex - the index that has achieved majority.

currentTerm - the current term.

Returns:

true if update is applied; otherwise, return false, i.e. no update required.

updateSnapshotIndex

public void updateSnapshotIndex(long newSnapshotIndex)

Update the last committed index and snapshotIndex with the last index in the snapshot.

Parameters:

newSnapshotIndex - the last index in the snapshot

contains

public boolean contains(TermIndex ti)

Does the log contains the given term and index? Used to check the consistency between the local log of a follower and the log entries sent by the leader.

getNextIndex

public long getNextIndex()

Returns:

the index of the next log entry to append.

append

public final long append(long term,
                         TransactionContext transaction)
                  throws StateMachineException

Append asynchronously a log entry for the given term and transaction. Used by the leader. Note that the underlying I/O operation is submitted but may not be completed when this method returns.

Returns:

the index of the new log entry.

Throws:

StateMachineException

appendMetadata

public final long appendMetadata(long term,
                                 long newCommitIndex)

Append asynchronously a log entry for the given term and commit index unless the given commit index is an index of a metadata entry Used by the leader. Note that the underlying I/O operation is submitted but may not be completed when this method returns.

Returns:

the index of the new log entry if it is appended; otherwise, return RaftServerConstants.INVALID_LOG_INDEX.

append

public final long append(long term,
                         RaftConfiguration configuration)

Append asynchronously a log entry for the given term and configuration Used by the leader. Note that the underlying I/O operation is submitted but may not be completed when this method returns.

Returns:

the index of the new log entry.

open

public final void open(long lastIndexInSnapshot,
                       Consumer<org.apache.ratis.proto.RaftProtos.LogEntryProto> consumer)
                throws IOException

Throws:

IOException

openImpl

protected void openImpl(long lastIndexInSnapshot,
                        Consumer<org.apache.ratis.proto.RaftProtos.LogEntryProto> consumer)
                 throws IOException

Throws:

IOException

getStartIndex

public abstract long getStartIndex()

get

public abstract org.apache.ratis.proto.RaftProtos.LogEntryProto get(long index)
                                                             throws RaftLogIOException

Get the log entry of the given index.

Parameters:

index - The given index.

Returns:

The log entry associated with the given index. Null if there is no log entry with the index.

Throws:

RaftLogIOException

getEntryWithData

public abstract RaftLog.EntryWithData getEntryWithData(long index)
                                                throws RaftLogIOException

Get the log entry of the given index along with the state machine data.

Parameters:

index - The given index.

Returns:

The log entry associated with the given index. Null if there is no log entry with the index.

Throws:

RaftLogIOException

getTermIndex

public abstract TermIndex getTermIndex(long index)

Get the TermIndex information of the given index.

Parameters:

index - The given index.

Returns:

The TermIndex of the log entry associated with the given index. Null if there is no log entry with the index.

getEntries

public abstract TermIndex[] getEntries(long startIndex,
                                       long endIndex)

Parameters:

startIndex - the starting log index (inclusive)

endIndex - the ending log index (exclusive)

Returns:

TermIndex of all log entries within the given index range. Null if startIndex is greater than the smallest available index.

getLastEntryTermIndex

public abstract TermIndex getLastEntryTermIndex()

Returns:

the last log entry's term and index.

validateLogEntry

protected void validateLogEntry(org.apache.ratis.proto.RaftProtos.LogEntryProto entry)

Validate the term and index of entry w.r.t RaftLog

truncate

public final CompletableFuture<Long> truncate(long index)

Truncate asynchronously the log entries till the given index (inclusively). Used by the leader and the followers.

truncateImpl

protected abstract CompletableFuture<Long> truncateImpl(long index)

purge

public final CompletableFuture<Long> purge(long suggestedIndex)

Purge asynchronously the log transactions. The implementation may choose to purge an index other than the suggested index.

Parameters:

suggestedIndex - the suggested index (inclusive) to be purged.

Returns:

the future of the actual purged log index.

purgeImpl

protected abstract CompletableFuture<Long> purgeImpl(long index)

appendEntry

public final CompletableFuture<Long> appendEntry(org.apache.ratis.proto.RaftProtos.LogEntryProto entry)

Append asynchronously an entry. Used by the leader and the followers.

appendEntryImpl

protected abstract CompletableFuture<Long> appendEntryImpl(org.apache.ratis.proto.RaftProtos.LogEntryProto entry)

append

public final List<CompletableFuture<Long>> append(org.apache.ratis.proto.RaftProtos.LogEntryProto... entries)

Append asynchronously all the given log entries. Used by the followers. If an existing entry conflicts with a new one (same index but different terms), delete the existing entry and all entries that follow it (§5.3).

appendImpl

protected abstract List<CompletableFuture<Long>> appendImpl(org.apache.ratis.proto.RaftProtos.LogEntryProto... entries)

getFlushIndex

public abstract long getFlushIndex()

Returns:

the index of the last entry that has been flushed to the local storage.

writeMetadata

public abstract void writeMetadata(long term,
                                   RaftPeerId votedFor)
                            throws IOException

Write and flush the metadata (votedFor and term) into the meta file. We need to guarantee that the order of writeMetadata calls is the same with that when we change the in-memory term/votedFor. Otherwise we may persist stale term/votedFor in file. Since the leader change is not frequent, currently we simply put this call in the RaftPeer's lock. Later we can use an IO task queue to enforce the order.

Throws:

IOException

loadMetadata

public abstract RaftLog.Metadata loadMetadata()
                                       throws IOException

Throws:

IOException

syncWithSnapshot

public abstract void syncWithSnapshot(long lastSnapshotIndex)

isConfigEntry

public abstract boolean isConfigEntry(TermIndex ti)

toString

public String toString()

Overrides:

toString in class Object

readLock

public AutoCloseableLock readLock()

writeLock

public AutoCloseableLock writeLock()

hasWriteLock

public boolean hasWriteLock()

hasReadLock

public boolean hasReadLock()

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

getName

public String getName()

toLogEntryString

public String toLogEntryString(org.apache.ratis.proto.RaftProtos.LogEntryProto logEntry)