org.apache.hadoop.hbase.regionserver.wal

类 FSHLog

java.lang.Object

org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL<WALProvider.Writer>

org.apache.hadoop.hbase.regionserver.wal.FSHLog

所有已实现的接口:

Closeable, AutoCloseable, WALFileLengthProvider, WAL

@InterfaceAudience.Private
public class FSHLog
extends AbstractFSWAL<WALProvider.Writer>

The default implementation of FSWAL.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.hadoop.hbase.wal.WAL

WAL.Entry, WAL.Reader

字段概要

字段

限定符和类型	字段和说明
static long	FIXED_OVERHEAD

从类继承的字段 org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL

blocksize, closed, conf, coprocessorHost, DEFAULT_SLOW_SYNC_TIME_MS, filenum, fs, highestSyncedTxid, highestUnsyncedTxid, implClassName, listeners, logrollsize, maxLogs, numEntries, ourFiles, prefixPathStr, rollRequested, rollWriterLock, sequenceIdAccounting, shutdown, slowSyncNs, totalLogSize, useHsync, walArchiveDir, walDir, walFile2Props, walFilePrefix, walFileSuffix

构造器概要

构造器

构造器和说明
FSHLog(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path root, String logDir, org.apache.hadoop.conf.Configuration conf) Constructor.
FSHLog(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, String logDir, String archiveDir, org.apache.hadoop.conf.Configuration conf, List<WALActionsListener> listeners, boolean failIfWALExists, String prefix, String suffix) Create an edit log at the given dir location.

方法概要

限定符和类型	方法和说明
protected void	afterCreatingZigZagLatch() Used to manufacture race condition reliably.
protected long	append(RegionInfo hri, WALKeyImpl key, WALEdit edits, boolean inMemstore) Append a set of edits to the WAL.
protected void	beforeWaitOnSafePoint()
protected WALProvider.Writer	createWriterInstance(org.apache.hadoop.fs.Path path) This method allows subclasses to inject different writers without having to extend other methods like rollWriter().
protected void	doAppend(WALProvider.Writer writer, org.apache.hadoop.hbase.regionserver.wal.FSWALEntry entry)
protected boolean	doCheckLogLowReplication()
protected void	doReplaceWriter(org.apache.hadoop.fs.Path oldPath, org.apache.hadoop.fs.Path newPath, WALProvider.Writer nextWriter) Notice that you need to clear the AbstractFSWAL.rollRequested flag in this method, as the new writer will begin to work before returning from this method.
protected void	doShutdown()
protected long	getSequenceOnRingBuffer()
protected org.apache.hadoop.hbase.regionserver.wal.SyncFuture	publishSyncOnRingBuffer(long sequence, boolean forceSync)
void	sync() Sync what we have in the WAL.
void	sync(boolean forceSync)
void	sync(long txid) Sync the WAL if the txId was not already sync'd.
void	sync(long txid, boolean forceSync)

从类继承的方法 org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL

abortCacheFlush, appendData, appendEntry, appendMarker, atHeadOfRingBufferEventHandlerAppend, blockOnSync, checkLogLowReplication, close, completeCacheFlush, computeFilename, getCoprocessorHost, getCurrentFileName, getEarliestMemStoreSeqNum, getEarliestMemStoreSeqNum, getFilenum, getFileNumFromFileName, getLogFileSize, getLogFileSizeIfBeingWritten, getNumLogFiles, getNumRolledLogFiles, getPreallocatedEventCount, getSyncFuture, getWALArchivePath, init, isLogRollRequested, logRollAndSetupWalProps, main, postSync, registerWALActionsListener, requestLogRoll, requestLogRoll, rollWriter, rollWriter, shutdown, stampSequenceIdAndPublishToRingBuffer, startCacheFlush, startCacheFlush, toString, unregisterWALActionsListener, updateStore

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

字段详细资料

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

构造器详细资料

FSHLog

public FSHLog(org.apache.hadoop.fs.FileSystem fs,
              org.apache.hadoop.fs.Path root,
              String logDir,
              org.apache.hadoop.conf.Configuration conf)
       throws IOException

Constructor.

参数:

fs - filesystem handle

root - path for stored and archived wals

logDir - dir where wals are stored

conf - configuration to use

抛出:

IOException

FSHLog

public FSHLog(org.apache.hadoop.fs.FileSystem fs,
              org.apache.hadoop.fs.Path rootDir,
              String logDir,
              String archiveDir,
              org.apache.hadoop.conf.Configuration conf,
              List<WALActionsListener> listeners,
              boolean failIfWALExists,
              String prefix,
              String suffix)
       throws IOException

Create an edit log at the given dir location. You should never have to load an existing log. If there is a log at startup, it should have already been processed and deleted by the time the WAL object is started up.

参数:

fs - filesystem handle

rootDir - path to where logs and oldlogs

logDir - dir where wals are stored

archiveDir - dir where wals are archived

conf - configuration to use

listeners - Listeners on WAL events. Listeners passed here will be registered before we do anything else; e.g. the Constructor AbstractFSWAL.rollWriter().

failIfWALExists - If true IOException will be thrown if files related to this wal already exist.

prefix - should always be hostname and port in distributed env and it will be URL encoded before being used. If prefix is null, "wal" will be used

suffix - will be url encoded. null is treated as empty. non-empty must start with AbstractFSWALProvider.WAL_FILE_NAME_DELIMITER

抛出:

IOException

方法详细资料

createWriterInstance

protected WALProvider.Writer createWriterInstance(org.apache.hadoop.fs.Path path)
                                           throws IOException

This method allows subclasses to inject different writers without having to extend other methods like rollWriter().

指定者:

createWriterInstance 在类中 AbstractFSWAL<WALProvider.Writer>

返回:

Writer instance

抛出:

IOException

afterCreatingZigZagLatch

protected void afterCreatingZigZagLatch()

Used to manufacture race condition reliably. For testing only.

另请参阅:

beforeWaitOnSafePoint()

beforeWaitOnSafePoint

protected void beforeWaitOnSafePoint()

另请参阅:

afterCreatingZigZagLatch()

doAppend

protected void doAppend(WALProvider.Writer writer,
                        org.apache.hadoop.hbase.regionserver.wal.FSWALEntry entry)
                 throws IOException

指定者:

doAppend 在类中 AbstractFSWAL<WALProvider.Writer>

抛出:

IOException

doReplaceWriter

protected void doReplaceWriter(org.apache.hadoop.fs.Path oldPath,
                               org.apache.hadoop.fs.Path newPath,
                               WALProvider.Writer nextWriter)
                        throws IOException

从类复制的说明: AbstractFSWAL

Notice that you need to clear the AbstractFSWAL.rollRequested flag in this method, as the new writer will begin to work before returning from this method. If we clear the flag after returning from this call, we may miss a roll request. The implementation class should choose a proper place to clear the AbstractFSWAL.rollRequested flag so we do not miss a roll request, typically before you start writing to the new writer.

指定者:

doReplaceWriter 在类中 AbstractFSWAL<WALProvider.Writer>

抛出:

IOException

doShutdown

protected void doShutdown()
                   throws IOException

指定者:

doShutdown 在类中 AbstractFSWAL<WALProvider.Writer>

抛出:

IOException

append

protected long append(RegionInfo hri,
                      WALKeyImpl key,
                      WALEdit edits,
                      boolean inMemstore)
               throws IOException

从类复制的说明: AbstractFSWAL

Append a set of edits to the WAL.

The WAL is not flushed/sync'd after this transaction completes BUT on return this edit must have its region edit/sequence id assigned else it messes up our unification of mvcc and sequenceid. On return key will have the region edit/sequence id filled in.

NOTE: This append, at a time that is usually after this call returns, starts an mvcc transaction by calling 'begin' wherein which we assign this update a sequenceid. At assignment time, we stamp all the passed in Cells inside WALEdit with their sequenceId. You must 'complete' the transaction this mvcc transaction by calling MultiVersionConcurrencyControl#complete(...) or a variant otherwise mvcc will get stuck. Do it in the finally of a try/finally block within which this append lives and any subsequent operations like sync or update of memstore, etc. Get the WriteEntry to pass mvcc out of the passed in WALKey walKey parameter. Be warned that the WriteEntry is not immediately available on return from this method. It WILL be available subsequent to a sync of this append; otherwise, you will just have to wait on the WriteEntry to get filled in.

指定者:

append 在类中 AbstractFSWAL<WALProvider.Writer>

参数:

hri - the regioninfo associated with append

key - Modified by this call; we add to it this edits region edit/sequence id.

edits - Edits to append. MAY CONTAIN NO EDITS for case where we want to get an edit sequence id that is after all currently appended edits.

inMemstore - Always true except for case where we are writing a region event meta marker edit, for example, a compaction completion record into the WAL or noting a Region Open event. In these cases the entry is just so we can finish an unfinished compaction after a crash when the new Server reads the WAL on recovery, etc. These transition event 'Markers' do not go via the memstore. When memstore is false, we presume a Marker event edit.

返回:

Returns a 'transaction id' and key will have the region edit/sequence id in it.

抛出:

IOException

doCheckLogLowReplication

protected boolean doCheckLogLowReplication()

指定者:

doCheckLogLowReplication 在类中 AbstractFSWAL<WALProvider.Writer>

返回:

true if number of replicas for the WAL is lower than threshold

getSequenceOnRingBuffer

protected long getSequenceOnRingBuffer()

publishSyncOnRingBuffer

protected org.apache.hadoop.hbase.regionserver.wal.SyncFuture publishSyncOnRingBuffer(long sequence,
                                                                                      boolean forceSync)

sync

public void sync()
          throws IOException

从接口复制的说明: WAL

Sync what we have in the WAL.

抛出:

IOException

sync

public void sync(boolean forceSync)
          throws IOException

参数:

forceSync - Flag to force sync rather than flushing to the buffer. Example - Hadoop hflush vs hsync.

抛出:

IOException

sync

public void sync(long txid)
          throws IOException

从接口复制的说明: WAL

Sync the WAL if the txId was not already sync'd.

参数:

txid - Transaction id to sync to.

抛出:

IOException

sync

public void sync(long txid,
                 boolean forceSync)
          throws IOException

参数:

txid - Transaction id to sync to.

forceSync - Flag to force sync rather than flushing to the buffer. Example - Hadoop hflush vs hsync.

抛出:

IOException