org.apache.hadoop.hbase.regionserver.compactions

类 Compactor<T extends Compactor.CellSink>

java.lang.Object

org.apache.hadoop.hbase.regionserver.compactions.Compactor<T>

直接已知子类:

AbstractMultiOutputCompactor, DefaultCompactor

@InterfaceAudience.Private
public abstract class Compactor<T extends Compactor.CellSink>
extends Object

A compactor is a compaction algorithm associated a given policy. Base class also contains reusable parts for implementing compactors (what is common and what isn't is evolving).

嵌套类概要

嵌套类

限定符和类型	类和说明
static interface	Compactor.CellSink
protected static interface	Compactor.CellSinkFactory<S>
protected static class	Compactor.FileDetails The sole reason this class exists is that java has no ref/out/pointer parameters.
protected static interface	Compactor.InternalScannerFactory

字段概要

字段

限定符和类型	字段和说明
protected Compression.Algorithm	compactionCompression
protected int	compactionKVMax
protected org.apache.hadoop.conf.Configuration	conf
protected Compactor.InternalScannerFactory	defaultScannerFactory
protected int	keepSeqIdPeriod specify how many days to keep MVCC values during major compaction
protected static String	MAJOR_COMPACTION_DROP_CACHE
protected static String	MINOR_COMPACTION_DROP_CACHE
protected CompactionProgress	progress
protected Store	store

方法概要

方法

限定符和类型	方法和说明
protected abstract void	abortWriter(T writer)
protected abstract List<org.apache.hadoop.fs.Path>	commitWriter(T writer, Compactor.FileDetails fd, CompactionRequest request)
protected List<org.apache.hadoop.fs.Path>	compact(CompactionRequest request, Compactor.InternalScannerFactory scannerFactory, Compactor.CellSinkFactory<T> sinkFactory, ThroughputController throughputController, User user)
protected List<StoreFileScanner>	createFileScanners(Collection<StoreFile> filesToCompact, long smallestReadPoint, boolean useDropBehind) Creates file scanners for compaction.
protected InternalScanner	createScanner(Store store, List<StoreFileScanner> scanners, long smallestReadPoint, long earliestPutTs, byte[] dropDeletesFromRow, byte[] dropDeletesToRow)
protected InternalScanner	createScanner(Store store, List<StoreFileScanner> scanners, ScanType scanType, long smallestReadPoint, long earliestPutTs)
protected StoreFile.Writer	createTmpWriter(Compactor.FileDetails fd, boolean shouldDropBehind) Creates a writer for a new file in a temporary directory.
protected Compactor.FileDetails	getFileDetails(Collection<StoreFile> filesToCompact, boolean allFiles) Extracts some details about the files to compact that are commonly needed by compactors.
CompactionProgress	getProgress()
protected long	getSmallestReadPoint()
protected boolean	performCompaction(InternalScanner scanner, Compactor.CellSink writer, long smallestReadPoint, boolean cleanSeqId, ThroughputController throughputController) Performs the compaction.
protected InternalScanner	postCreateCoprocScanner(CompactionRequest request, ScanType scanType, InternalScanner scanner, User user) Calls coprocessor, if any, to create scanners - after normal scanner creation.
protected InternalScanner	preCreateCoprocScanner(CompactionRequest request, ScanType scanType, long earliestPutTs, List<StoreFileScanner> scanners, User user, long readPoint) Calls coprocessor, if any, to create compaction scanner - before normal scanner creation.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

progress

protected volatile CompactionProgress progress

conf

protected final org.apache.hadoop.conf.Configuration conf

store

protected final Store store

compactionKVMax

protected final int compactionKVMax

compactionCompression

protected final Compression.Algorithm compactionCompression

keepSeqIdPeriod

protected int keepSeqIdPeriod

specify how many days to keep MVCC values during major compaction

MAJOR_COMPACTION_DROP_CACHE

protected static final String MAJOR_COMPACTION_DROP_CACHE

另请参阅:

常量字段值

MINOR_COMPACTION_DROP_CACHE

protected static final String MINOR_COMPACTION_DROP_CACHE

另请参阅:

常量字段值

defaultScannerFactory

protected final Compactor.InternalScannerFactory defaultScannerFactory

方法详细资料

getProgress

public CompactionProgress getProgress()

getFileDetails

protected Compactor.FileDetails getFileDetails(Collection<StoreFile> filesToCompact,
                                   boolean allFiles)
                                        throws IOException

Extracts some details about the files to compact that are commonly needed by compactors.

参数:

filesToCompact - Files.

allFiles - Whether all files are included for compaction

返回:

The result.

抛出:

IOException

createFileScanners

protected List<StoreFileScanner> createFileScanners(Collection<StoreFile> filesToCompact,
                                        long smallestReadPoint,
                                        boolean useDropBehind)
                                             throws IOException

Creates file scanners for compaction.

参数:

filesToCompact - Files.

返回:

Scanners.

抛出:

IOException

getSmallestReadPoint

protected long getSmallestReadPoint()

createTmpWriter

protected StoreFile.Writer createTmpWriter(Compactor.FileDetails fd,
                               boolean shouldDropBehind)
                                    throws IOException

Creates a writer for a new file in a temporary directory.

参数:

fd - The file details.

返回:

Writer for a new StoreFile in the tmp dir.

抛出:

IOException - if creation failed

compact

protected List<org.apache.hadoop.fs.Path> compact(CompactionRequest request,
                                      Compactor.InternalScannerFactory scannerFactory,
                                      Compactor.CellSinkFactory<T> sinkFactory,
                                      ThroughputController throughputController,
                                      User user)
                                           throws IOException

抛出:

IOException

commitWriter

protected abstract List<org.apache.hadoop.fs.Path> commitWriter(T writer,
                                           Compactor.FileDetails fd,
                                           CompactionRequest request)
                                                         throws IOException

抛出:

IOException

abortWriter

protected abstract void abortWriter(T writer)
                             throws IOException

抛出:

IOException

preCreateCoprocScanner

protected InternalScanner preCreateCoprocScanner(CompactionRequest request,
                                     ScanType scanType,
                                     long earliestPutTs,
                                     List<StoreFileScanner> scanners,
                                     User user,
                                     long readPoint)
                                          throws IOException

Calls coprocessor, if any, to create compaction scanner - before normal scanner creation.

参数:

request - Compaction request.

scanType - Scan type.

earliestPutTs - Earliest put ts.

scanners - File scanners for compaction files.

user - the User

readPoint - the read point to help create scanner by Coprocessor if required.

返回:

Scanner override by coprocessor; null if not overriding.

抛出:

IOException

postCreateCoprocScanner

protected InternalScanner postCreateCoprocScanner(CompactionRequest request,
                                      ScanType scanType,
                                      InternalScanner scanner,
                                      User user)
                                           throws IOException

Calls coprocessor, if any, to create scanners - after normal scanner creation.

参数:

request - Compaction request.

scanType - Scan type.

scanner - The default scanner created for compaction.

返回:

Scanner scanner to use (usually the default); null if compaction should not proceed.

抛出:

IOException

performCompaction

protected boolean performCompaction(InternalScanner scanner,
                        Compactor.CellSink writer,
                        long smallestReadPoint,
                        boolean cleanSeqId,
                        ThroughputController throughputController)
                             throws IOException

Performs the compaction.

参数:

scanner - Where to read from.

writer - Where to write to.

smallestReadPoint - Smallest read point.

cleanSeqId - When true, remove seqId(used to be mvcc) value which is <= smallestReadPoint

返回:

Whether compaction ended; false if it was interrupted for some reason.

抛出:

IOException

createScanner

protected InternalScanner createScanner(Store store,
                            List<StoreFileScanner> scanners,
                            ScanType scanType,
                            long smallestReadPoint,
                            long earliestPutTs)
                                 throws IOException

参数:

store - store

scanners - Store file scanners.

scanType - Scan type.

smallestReadPoint - Smallest MVCC read point.

earliestPutTs - Earliest put across all files.

返回:

A compaction scanner.

抛出:

IOException

createScanner

protected InternalScanner createScanner(Store store,
                            List<StoreFileScanner> scanners,
                            long smallestReadPoint,
                            long earliestPutTs,
                            byte[] dropDeletesFromRow,
                            byte[] dropDeletesToRow)
                                 throws IOException

参数:

store - The store.

scanners - Store file scanners.

smallestReadPoint - Smallest MVCC read point.

earliestPutTs - Earliest put across all files.

dropDeletesFromRow - Drop deletes starting with this row, inclusive. Can be null.

dropDeletesToRow - Drop deletes ending with this row, exclusive. Can be null.

返回:

A compaction scanner.

抛出:

IOException