com.sleepycat.je.cleaner

Class Cleaner

java.lang.Object

com.sleepycat.je.cleaner.Cleaner

All Implemented Interfaces:

EnvConfigObserver, DaemonRunner

public class Cleaner
extends Object
implements DaemonRunner, EnvConfigObserver

The Cleaner is responsible for effectively garbage collecting the JE log. It selects the least utilized log file for cleaning (see FileSelector), reads through the log file (FileProcessor) and determines whether each entry is obsolete (no longer relevant) or active (referenced by the Btree). Entries that are active are migrated (copied) to the end of the log, and finally the cleaned file is deleted. The migration of active entries is a multi-step process that can be configured to operate in different ways. Eviction and checkpointing, as well as the cleaner threads (FileProcessor instances) are participants in this process. Migration may be immediate or lazy. Active INs are always migrated lazily, which means that they are marked dirty by the FileProcessor, and then logged later by an eviction or checkpoint. Active LNs are always migrated immediately by the FileProcessor by logging them. When the FileProcessor is finished with a file, all lazy migration for that file is normally completed by the end of the next checkpoint, if not sooner via eviction. The checkpoint/recovery mechanism will ensure that obsolete entries will not be referenced by the Btree. At the end of the checkpoint, it is therefore safe to delete the log file. There is one exception to the above paragraph. When attempting to migrate an LN, if the LN cannot be locked then we must retry the migration at a later time. Also, if a database removal is in progress, we consider all entries in the database obsolete but cannot delete the log file until database removal is complete. Such "pending" LNs and databases are queued and processed periodically during file processing and at the start of a checkpoint; see processPending(). In this case, we may have to wait for more than one checkpoint to occur before the log file can be deleted. See FileSelector and the use of the pendingLNs and pendingDBs collections.

Constructor Summary

Constructors

Constructor and Description
Cleaner(EnvironmentImpl env, String name)

Method Summary

Modifier and Type	Method and Description
void	close() Release resources and update memory budget.
FileProcessor	createProcessor()
int	doClean(boolean cleanMultipleFiles, boolean forceCleaning) Cleans selected files and returns the number of files cleaned.
void	enableFileDeletion(boolean enable) Enables or disabling processing of safe-to-delete files, including the truncation of the VLSN index.
void	envConfigUpdate(DbConfigManager cm, EnvironmentMutableConfig ignore) Process notifications of mutable property changes.
void	freshenLogSizeStats() Updates the cached set of log size stats, including maxDiskOverage, freeDiskShortage, diskUsageMessage and diskUsageViolationMessage.
String	getDiskLimitMessage() Returns a message describing disk space limits and usage, regardless of whether the limit is violated or not.
String	getDiskLimitViolation() Uses cached disk usage info to determine whether disk space limits are currently violated.
ExpirationProfile	getExpirationProfile()
boolean	getFetchObsoleteSize(DatabaseImpl db)
FileProtector	getFileProtector()
FileSelector.CheckpointStartCleanerState	getFilesAtCheckpointStart() Returns a copy of the cleaned files at the time a checkpoint starts.
FileSelector	getFileSelector()
int	getNWakeupRequests()
UtilizationProfile	getUtilizationProfile()
UtilizationTracker	getUtilizationTracker()
boolean	isRMWFixEnabled()
StatGroup	loadStats(StatsConfig config) Load stats.
void	manageDiskUsage() Updates log size stats and deletes unprotected reserved files in order to stay within disk limits.
boolean	repairReservedFile(DatabaseImpl dbImpl, CursorImpl cursorImpl, byte[] key) If the LSN at the cursor position refers to an LN in a reserved file, and the LN may be fetched (is not embedded, etc) then repair the problem by reactivating the file.
void	requestShutdown()
void	runOrPause(boolean run)
void	shutdown()
void	updateFilesAtCheckpointEnd(FileSelector.CheckpointStartCleanerState info) When a checkpoint is complete, update the files that were returned at the beginning of the checkpoint.
void	wakeupActivate() Wakeup the file processor threads in activate mode, meaning that FileProcessor.doClean will be called.
void	wakeupAfterWrite(int writeSize) If the number of bytes written since the last activation exceeds the cleaner's byte interval, wakeup the file processor threads in activate mode.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Cleaner

public Cleaner(EnvironmentImpl env,
               String name)

Method Detail

envConfigUpdate

public void envConfigUpdate(DbConfigManager cm,
                            EnvironmentMutableConfig ignore)

Process notifications of mutable property changes.

Specified by:

envConfigUpdate in interface EnvConfigObserver

Throws:

IllegalArgumentException - via Environment ctor and setMutableConfig.

getFileProtector

public FileProtector getFileProtector()

getUtilizationTracker

public UtilizationTracker getUtilizationTracker()

getUtilizationProfile

public UtilizationProfile getUtilizationProfile()

getExpirationProfile

public ExpirationProfile getExpirationProfile()

getFileSelector

public FileSelector getFileSelector()

getFetchObsoleteSize

public boolean getFetchObsoleteSize(DatabaseImpl db)

isRMWFixEnabled

public boolean isRMWFixEnabled()

See Also:

EnvironmentParams.CLEANER_RMW_FIX, FileSummaryLN.postFetchInit(com.sleepycat.je.dbi.DatabaseImpl, long)

runOrPause

public void runOrPause(boolean run)

Specified by:

runOrPause in interface DaemonRunner

wakeupAfterWrite

public void wakeupAfterWrite(int writeSize)

If the number of bytes written since the last activation exceeds the cleaner's byte interval, wakeup the file processor threads in activate mode.

wakeupActivate

public void wakeupActivate()

Wakeup the file processor threads in activate mode, meaning that FileProcessor.doClean will be called.

See Also:

FileProcessor.onWakeup()

requestShutdown

public void requestShutdown()

Specified by:

requestShutdown in interface DaemonRunner

shutdown

public void shutdown()

Specified by:

shutdown in interface DaemonRunner

getNWakeupRequests

public int getNWakeupRequests()

Specified by:

getNWakeupRequests in interface DaemonRunner

doClean

public int doClean(boolean cleanMultipleFiles,
                   boolean forceCleaning)

Cleans selected files and returns the number of files cleaned. This method is not invoked by a deamon thread, it is programatically.

Parameters:

cleanMultipleFiles - is true to clean until we're under budget, or false to clean at most one file.

forceCleaning - is true to clean even if we're not under the utilization threshold.

Returns:

the number of files cleaned, not including files cleaned unsuccessfully.

createProcessor

public FileProcessor createProcessor()

loadStats

public StatGroup loadStats(StatsConfig config)

Load stats.

enableFileDeletion

public void enableFileDeletion(boolean enable)

Enables or disabling processing of safe-to-delete files, including the truncation of the VLSN index. Disabling this is needed for tests, when VLSNIndex changes should be prevented.

manageDiskUsage

public void manageDiskUsage()

Updates log size stats and deletes unprotected reserved files in order to stay within disk limits. This method must be called frequently enough to maintain disk usage safely below the limits. For an HA env this is particularly important since we retain all reserved files until we approach the disk limits. For this, calling this method at least every CLEANER_BYTES_INTERVAL should suffice. It is also important to call this method based on a time interval when writing stops, to retry deletions when files are protected or the env is locked by read-only processes. For this, calling this method at least every CLEANER_WAKEUP_INTERVAL should suffice.

freshenLogSizeStats

public void freshenLogSizeStats()

Updates the cached set of log size stats, including maxDiskOverage, freeDiskShortage, diskUsageMessage and diskUsageViolationMessage. Normally this should only be called by manageDiskUsage while holding the manageDiskUsageLock. However, it may be called directly during (single threaded) recovery.

getDiskLimitMessage

public String getDiskLimitMessage()

Returns a message describing disk space limits and usage, regardless of whether the limit is violated or not. If there is a limit violation, returns the same value as getDiskLimitViolation(). Does not return null.

getDiskLimitViolation

public String getDiskLimitViolation()

Uses cached disk usage info to determine whether disk space limits are currently violated. This method simply returns a volatile field. The cached information is updated frequently enough to prevent violating the limits by a large amount.

Returns:

a non-null message (appropriate for an exception) if a disk limit is currently violated, else null.

getFilesAtCheckpointStart

public FileSelector.CheckpointStartCleanerState getFilesAtCheckpointStart()

Returns a copy of the cleaned files at the time a checkpoint starts.

If FileSelector.CheckpointStartCleanerState.isEmpty() returns false, the checkpoint should flush an extra level, and updateFilesAtCheckpointEnd(com.sleepycat.je.cleaner.FileSelector.CheckpointStartCleanerState) should be called when the checkpoint is complete.

updateFilesAtCheckpointEnd

public void updateFilesAtCheckpointEnd(FileSelector.CheckpointStartCleanerState info)

When a checkpoint is complete, update the files that were returned at the beginning of the checkpoint.

repairReservedFile

public boolean repairReservedFile(DatabaseImpl dbImpl,
                                  CursorImpl cursorImpl,
                                  byte[] key)

If the LSN at the cursor position refers to an LN in a reserved file, and the LN may be fetched (is not embedded, etc) then repair the problem by reactivating the file.

To guarantee that an active LN at the cursor position is detected, a read lock on the record at the cursor position should be held when this method is called. If a read lock is not held, a write lock may be held by another thread and a different LSN may be restored to the BIN slot after an abort.

This method may release the record lock for the cursor's current LSN. The lock is always held while determining the active LSN, but released to performing more costly operations. If no costly operations are necessary, the lock is not released.

Returns:

true if the LSN could be checked, or false if it could not be checked because the extinction filter returned MAYBE_EXTINCT.

close

public void close()

Release resources and update memory budget. Should only be called when this environment is closed and will never be accessed again.