org.apache.hadoop.hdfs.server.namenode
Class JournalSet

java.lang.Object
  org.apache.hadoop.hdfs.server.namenode.JournalSet

All Implemented Interfaces:

Closeable, org.apache.hadoop.hdfs.server.common.Storage.FormatConfirmable, org.apache.hadoop.hdfs.server.namenode.JournalManager

public class JournalSet
extends Object
implements org.apache.hadoop.hdfs.server.namenode.JournalManager

Manages a collection of Journals. None of the methods are synchronized, it is assumed that FSEditLog methods, that use this class, use proper synchronization.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.server.namenode.JournalManager

JournalManager.CorruptionException

Field Summary

static Comparator<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream>

EDIT_LOG_INPUT_STREAM_COMPARATOR

Method Summary

static void	chainAndMakeRedundantStreams(Collection<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> outStreams, PriorityQueue<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> allStreams, long fromTxId)
void	close() Close the journal manager, freeing any resources it may hold.
void	finalizeLogSegment(long firstTxId, long lastTxId) Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.
void	format(org.apache.hadoop.hdfs.server.protocol.NamespaceInfo nsInfo) Format the underlying storage, removing any previously stored data.
RemoteEditLogManifest	getEditLogManifest(long fromTxId, boolean forReading) Return a manifest of what finalized edit logs are available.
boolean	hasSomeData()
boolean	isEmpty() Returns true if there are no journals, all redundant journals are disabled, or any required journals are disabled.
void	purgeLogsOlderThan(long minTxIdToKeep) Remove all edit logs with transaction IDs lower than the given transaction ID.
void	recoverUnfinalizedSegments() Recover segments which have not been finalized.
void	selectInputStreams(Collection<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> streams, long fromTxId, boolean inProgressOk, boolean forReading) In this function, we get a bunch of streams from all of our JournalManager objects.
void	setOutputBufferCapacity(int size) Set the amount of memory that this stream should use to buffer edits
org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream	startLogSegment(long txId) Begin writing to a new segment of the log stream, which starts at the given transaction ID.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hdfs.server.common.Storage.FormatConfirmable

toString

Field Detail

EDIT_LOG_INPUT_STREAM_COMPARATOR

public static final Comparator<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> EDIT_LOG_INPUT_STREAM_COMPARATOR

Method Detail

format

public void format(org.apache.hadoop.hdfs.server.protocol.NamespaceInfo nsInfo)
            throws IOException

Description copied from interface: org.apache.hadoop.hdfs.server.namenode.JournalManager

Format the underlying storage, removing any previously stored data.

Specified by:

format in interface org.apache.hadoop.hdfs.server.namenode.JournalManager

Throws:

IOException

hasSomeData

public boolean hasSomeData()
                    throws IOException

Specified by:

hasSomeData in interface org.apache.hadoop.hdfs.server.common.Storage.FormatConfirmable

Returns:

true if the storage seems to have some valid data in it, and the user should be required to confirm the format. Otherwise, false.

Throws:

IOException - if the storage cannot be accessed at all.

startLogSegment

public org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream startLogSegment(long txId)
                                                                           throws IOException

Description copied from interface: org.apache.hadoop.hdfs.server.namenode.JournalManager

Begin writing to a new segment of the log stream, which starts at the given transaction ID.

Specified by:

startLogSegment in interface org.apache.hadoop.hdfs.server.namenode.JournalManager

Throws:

IOException

finalizeLogSegment

public void finalizeLogSegment(long firstTxId,
                               long lastTxId)
                        throws IOException

Description copied from interface: org.apache.hadoop.hdfs.server.namenode.JournalManager

Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.

Specified by:

finalizeLogSegment in interface org.apache.hadoop.hdfs.server.namenode.JournalManager

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: org.apache.hadoop.hdfs.server.namenode.JournalManager

Close the journal manager, freeing any resources it may hold.

Specified by:

close in interface Closeable

Specified by:

close in interface org.apache.hadoop.hdfs.server.namenode.JournalManager

Throws:

IOException

selectInputStreams

public void selectInputStreams(Collection<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> streams,
                               long fromTxId,
                               boolean inProgressOk,
                               boolean forReading)
                        throws IOException

In this function, we get a bunch of streams from all of our JournalManager objects. Then we add these to the collection one by one.

Parameters:

streams - The collection to add the streams to. It may or may not be sorted-- this is up to the caller.

fromTxId - The transaction ID to start looking for streams at

inProgressOk - Should we consider unfinalized streams?

forReading - Whether or not the caller intends to read from the returned streams.

Throws:

IOException - if the underlying storage has an error or is otherwise inaccessible

chainAndMakeRedundantStreams

public static void chainAndMakeRedundantStreams(Collection<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> outStreams,
                                                PriorityQueue<org.apache.hadoop.hdfs.server.namenode.EditLogInputStream> allStreams,
                                                long fromTxId)

isEmpty

public boolean isEmpty()

Returns true if there are no journals, all redundant journals are disabled, or any required journals are disabled.

Returns:

True if there no journals, all redundant journals are disabled, or any required journals are disabled.

setOutputBufferCapacity

public void setOutputBufferCapacity(int size)

Description copied from interface: org.apache.hadoop.hdfs.server.namenode.JournalManager

Set the amount of memory that this stream should use to buffer edits

Specified by:

setOutputBufferCapacity in interface org.apache.hadoop.hdfs.server.namenode.JournalManager

purgeLogsOlderThan

public void purgeLogsOlderThan(long minTxIdToKeep)
                        throws IOException

Remove all edit logs with transaction IDs lower than the given transaction ID.

Parameters:

minTxIdToKeep - the lowest transaction ID that should be retained

Throws:

IOException - in the event of error

recoverUnfinalizedSegments

public void recoverUnfinalizedSegments()
                                throws IOException

Description copied from interface: org.apache.hadoop.hdfs.server.namenode.JournalManager

Recover segments which have not been finalized.

Specified by:

recoverUnfinalizedSegments in interface org.apache.hadoop.hdfs.server.namenode.JournalManager

Throws:

IOException

getEditLogManifest

public RemoteEditLogManifest getEditLogManifest(long fromTxId,
                                                boolean forReading)

Return a manifest of what finalized edit logs are available. All available edit logs are returned starting from the transaction id passed.

Parameters:

fromTxId - Starting transaction id to read the logs.

Returns:

RemoteEditLogManifest object.