org.apache.hadoop.hbase.wal

类 WALSplitter

java.lang.Object

org.apache.hadoop.hbase.wal.WALSplitter

@InterfaceAudience.Private
public class WALSplitter
extends Object

This class is responsible for splitting up a bunch of regionserver commit log files that are no longer being written to, into new files, one per region, for recovering data on startup. Delete the old log files when finished.

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	WALSplitter.PipelineController Contains some methods to control WAL-entries producer / consumer interactions
static class	WALSplitter.RegionEntryBuffer A buffer of some number of edits for a given region.
static class	WALSplitter.SinkWriter Class wraps the actual writer which writes data out and related statistics

字段概要

字段

限定符和类型	字段和说明
protected org.apache.hadoop.conf.Configuration	conf
protected Map<String,Long>	lastFlushedSequenceIds
protected Map<String,Map<byte[],Long>>	regionMaxSeqIdInStores
protected LastSequenceId	sequenceIdChecker
static boolean	SPLIT_SKIP_ERRORS_DEFAULT By default we retry errors in splitting, rather than skipping.
static String	SPLIT_WRITER_CREATION_BOUNDED
protected org.apache.hadoop.fs.Path	walDir
protected org.apache.hadoop.fs.FileSystem	walFS

方法概要

限定符和类型	方法和说明
protected WALProvider.Writer	createWriter(org.apache.hadoop.fs.Path logfile) Create a new WALProvider.Writer for writing log splits.
protected WAL.Reader	getReader(org.apache.hadoop.fs.FileStatus file, boolean skipErrors, CancelableProgressable reporter) Create a new WAL.Reader for reading logs to split.
protected WAL.Reader	getReader(org.apache.hadoop.fs.Path curLogFile, CancelableProgressable reporter) Create a new WAL.Reader for reading logs to split.
static List<org.apache.hadoop.fs.Path>	split(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path logDir, org.apache.hadoop.fs.Path oldLogDir, org.apache.hadoop.fs.FileSystem walFS, org.apache.hadoop.conf.Configuration conf, WALFactory factory)
static boolean	splitLogFile(org.apache.hadoop.fs.Path walDir, org.apache.hadoop.fs.FileStatus logfile, org.apache.hadoop.fs.FileSystem walFS, org.apache.hadoop.conf.Configuration conf, CancelableProgressable reporter, LastSequenceId idChecker, SplitLogWorkerCoordination splitLogWorkerCoordination, WALFactory factory) Splits a WAL file into region's recovered-edits directory.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

SPLIT_SKIP_ERRORS_DEFAULT

public static final boolean SPLIT_SKIP_ERRORS_DEFAULT

By default we retry errors in splitting, rather than skipping.

另请参阅:

常量字段值

walDir

protected final org.apache.hadoop.fs.Path walDir

walFS

protected final org.apache.hadoop.fs.FileSystem walFS

conf

protected final org.apache.hadoop.conf.Configuration conf

sequenceIdChecker

protected final LastSequenceId sequenceIdChecker

lastFlushedSequenceIds

protected Map<String,Long> lastFlushedSequenceIds

regionMaxSeqIdInStores

protected Map<String,Map<byte[],Long>> regionMaxSeqIdInStores

SPLIT_WRITER_CREATION_BOUNDED

public static final String SPLIT_WRITER_CREATION_BOUNDED

另请参阅:

常量字段值

方法详细资料

splitLogFile

public static boolean splitLogFile(org.apache.hadoop.fs.Path walDir,
                                   org.apache.hadoop.fs.FileStatus logfile,
                                   org.apache.hadoop.fs.FileSystem walFS,
                                   org.apache.hadoop.conf.Configuration conf,
                                   CancelableProgressable reporter,
                                   LastSequenceId idChecker,
                                   SplitLogWorkerCoordination splitLogWorkerCoordination,
                                   WALFactory factory)
                            throws IOException

Splits a WAL file into region's recovered-edits directory. This is the main entry point for distributed log splitting from SplitLogWorker.

If the log file has N regions then N recovered.edits files will be produced.

返回:

false if it is interrupted by the progress-able.

抛出:

IOException

split

public static List<org.apache.hadoop.fs.Path> split(org.apache.hadoop.fs.Path rootDir,
                                                    org.apache.hadoop.fs.Path logDir,
                                                    org.apache.hadoop.fs.Path oldLogDir,
                                                    org.apache.hadoop.fs.FileSystem walFS,
                                                    org.apache.hadoop.conf.Configuration conf,
                                                    WALFactory factory)
                                             throws IOException

抛出:

IOException

getReader

protected WAL.Reader getReader(org.apache.hadoop.fs.FileStatus file,
                               boolean skipErrors,
                               CancelableProgressable reporter)
                        throws IOException,
                               org.apache.hadoop.hbase.wal.WALSplitter.CorruptedLogFileException

Create a new WAL.Reader for reading logs to split.

抛出:

IOException

org.apache.hadoop.hbase.wal.WALSplitter.CorruptedLogFileException

createWriter

protected WALProvider.Writer createWriter(org.apache.hadoop.fs.Path logfile)
                                   throws IOException

Create a new WALProvider.Writer for writing log splits.

返回:

a new Writer instance, caller should close

抛出:

IOException

getReader

protected WAL.Reader getReader(org.apache.hadoop.fs.Path curLogFile,
                               CancelableProgressable reporter)
                        throws IOException

Create a new WAL.Reader for reading logs to split.

返回:

new Reader instance, caller should close

抛出:

IOException