org.apache.hadoop.hbase.master.cleaner

Class CleanerChore<T extends FileCleanerDelegate>

java.lang.Object

org.apache.hadoop.hbase.ScheduledChore

org.apache.hadoop.hbase.master.cleaner.CleanerChore<T>

Type Parameters:

T - Cleaner delegate class that is dynamically loaded from configuration

All Implemented Interfaces:

Runnable

Direct Known Subclasses:

HFileCleaner, LogCleaner

@InterfaceAudience.Private
public abstract class CleanerChore<T extends FileCleanerDelegate>
extends ScheduledChore

Abstract Cleaner that uses a chain of delegates to clean a directory of files

Field Summary

Fields

Modifier and Type	Field and Description
static String	CHORE_POOL_SIZE Configures the threadpool used for scanning the archive directory for the HFileCleaner If it is an integer and >= 1, it would be the size; if 0.0 < size <= 1.0, size would be available processors * size.
protected List<T>	cleanersChain
protected List<String>	excludeDirs
protected org.apache.hadoop.fs.FileSystem	fs
static String	LOG_CLEANER_CHORE_DIRECTORY_SORTING Enable the CleanerChore to sort the subdirectories by consumed space and start the cleaning with the largest subdirectory.
static String	LOG_CLEANER_CHORE_SIZE Configures the threadpool used for scanning the Old logs directory for the LogCleaner Follows the same configuration mechanism as CHORE_POOL_SIZE, but has a default of 1 thread.
protected Map<String,Object>	params

Constructor Summary

Constructors

Constructor and Description
CleanerChore(String name, int sleepPeriod, Stoppable s, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path oldFileDir, String confKey, DirScanPool pool)
CleanerChore(String name, int sleepPeriod, Stoppable s, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path oldFileDir, String confKey, DirScanPool pool, Map<String,Object> params, List<org.apache.hadoop.fs.Path> excludePaths)

Method Summary

Modifier and Type	Method and Description
protected void	chore()
void	cleanup()
protected int	deleteFiles(Iterable<org.apache.hadoop.fs.FileStatus> filesToDelete) Delete the given files
boolean	getEnabled()
boolean	runCleaner()
boolean	setEnabled(boolean enabled)
protected abstract boolean	validate(org.apache.hadoop.fs.Path file) Validate the file to see if it even belongs in the directory.

Methods inherited from class org.apache.hadoop.hbase.ScheduledChore

cancel, cancel, choreForTesting, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, initialChore, isInitialChoreComplete, isScheduled, run, shutdown, shutdown, toString, triggerNow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CHORE_POOL_SIZE

public static final String CHORE_POOL_SIZE

Configures the threadpool used for scanning the archive directory for the HFileCleaner If it is an integer and >= 1, it would be the size; if 0.0 < size <= 1.0, size would be available processors * size. Pay attention that 1.0 is different from 1, former indicates it will use 100% of cores, while latter will use only 1 thread for chore to scan dir.

See Also:

Constant Field Values

LOG_CLEANER_CHORE_SIZE

public static final String LOG_CLEANER_CHORE_SIZE

Configures the threadpool used for scanning the Old logs directory for the LogCleaner Follows the same configuration mechanism as CHORE_POOL_SIZE, but has a default of 1 thread.

See Also:

Constant Field Values

LOG_CLEANER_CHORE_DIRECTORY_SORTING

public static final String LOG_CLEANER_CHORE_DIRECTORY_SORTING

Enable the CleanerChore to sort the subdirectories by consumed space and start the cleaning with the largest subdirectory. Enabled by default.

See Also:

Constant Field Values

fs

protected final org.apache.hadoop.fs.FileSystem fs

params

protected final Map<String,Object> params

cleanersChain

protected List<T extends FileCleanerDelegate> cleanersChain

excludeDirs

protected List<String> excludeDirs

Constructor Detail

CleanerChore

public CleanerChore(String name,
                    int sleepPeriod,
                    Stoppable s,
                    org.apache.hadoop.conf.Configuration conf,
                    org.apache.hadoop.fs.FileSystem fs,
                    org.apache.hadoop.fs.Path oldFileDir,
                    String confKey,
                    DirScanPool pool)

CleanerChore

public CleanerChore(String name,
                    int sleepPeriod,
                    Stoppable s,
                    org.apache.hadoop.conf.Configuration conf,
                    org.apache.hadoop.fs.FileSystem fs,
                    org.apache.hadoop.fs.Path oldFileDir,
                    String confKey,
                    DirScanPool pool,
                    Map<String,Object> params,
                    List<org.apache.hadoop.fs.Path> excludePaths)

Parameters:

name - name of the chore being run

sleepPeriod - the period of time to sleep between each run

s - the stopper

conf - configuration to use

fs - handle to the FS

oldFileDir - the path to the archived files

confKey - configuration key for the classes to instantiate

pool - the thread pool used to scan directories

params - members could be used in cleaner

Method Detail

validate

protected abstract boolean validate(org.apache.hadoop.fs.Path file)

Validate the file to see if it even belongs in the directory. If it is valid, then the file will go through the cleaner delegates, but otherwise the file is just deleted.

Parameters:

file - full Path of the file to be checked

Returns:

true if the file is valid, false otherwise

chore

protected void chore()

Specified by:

chore in class ScheduledChore

runCleaner

public boolean runCleaner()

deleteFiles

protected int deleteFiles(Iterable<org.apache.hadoop.fs.FileStatus> filesToDelete)

Delete the given files

Parameters:

filesToDelete - files to delete

Returns:

number of deleted files

cleanup

public void cleanup()

Overrides:

cleanup in class ScheduledChore

setEnabled

public boolean setEnabled(boolean enabled)

getEnabled

public boolean getEnabled()