com.sleepycat.je.cleaner

Class ExtinctionScanner

java.lang.Object

com.sleepycat.je.cleaner.ExtinctionScanner

All Implemented Interfaces:

EnvConfigObserver

public class ExtinctionScanner
extends Object
implements EnvConfigObserver

Performs async processing for Record Extinction and Database Extinction, via the RecordExtinction and DatabaseExtinction tasks.

When the application calls Environment.discardExtinctRecords(com.sleepycat.je.Transaction, java.util.Set<java.lang.String>, com.sleepycat.je.DatabaseEntry, com.sleepycat.je.DatabaseEntry, com.sleepycat.je.ScanFilter, java.lang.String) (record extinction) or Environment.removeDatabase(com.sleepycat.je.Transaction, java.lang.String) or Environment.truncateDatabase(com.sleepycat.je.Transaction, java.lang.String, boolean) (DB extinction), we store a scan record in an internal scan DB and queue the scan task.

The scan DB is a replicated DB, but the scan records themselves may be replicated or non-replicated as described below. (This mixing is only possible for internal DBs, and is also used for the DbTree naming DB.)

• For record extinction, the scan record is replicated and is inserted in the user transaction passed to discardExtinctRecords. The durability of this record is therefore the same as the durability of the record extinction, and if this txn aborts, the scan record insertion will also be aborted. When this record is replicated, the replica replay will initiate the scan on the replica.
• For DB extinction, the scan record is inserted after the commit of the txn passed to removeDatabase or truncateDatabase. This insertion is not replicated, and instead the replay of NameLN causes insertion of the scan record on each replica. If there is a crash between the user txn commit and insertion of the scan record, recovery redo of the NameLN will cause insertion of the scan record.

The scan task is always performed independently on each node in a rep group, and therefore update and deletion of the scan record are non-replicated operations. Recovery will re-queue the scan task for all scan records not yet deleted.

For now, although a thread executor is used, it is limited to just use one thread. So for now, all extinction tasks will execute serially. If using multiple threads becomes necessary, some synchronization changes will be needed.

Field Summary

Fields

Modifier and Type	Field and Description
TestHook<?>	dbBeforeWriteTaskHook
static JEVersion	TEST_MIN_JE_VERSION

Constructor Summary

Constructors

Constructor and Description
ExtinctionScanner(@NonNull EnvironmentImpl envImpl)

Method Summary

Modifier and Type	Method and Description
void	deleteCompletedRecordScans(Set<Long> scanIds) After a checkpoint, delete the scans previously returned by getCompletedRecordScans().
long	discardExtinctRecords(@NonNull Locker locker, @NonNull Set<String> dbNames, @Nullable DatabaseEntry beginKey, @Nullable DatabaseEntry endKey, @Nullable ScanFilter filter, @NonNull String label) Inserts the scan record in the DB and queues the scan for execution.
void	ensureDbExtinction(DatabaseImpl dbImpl) Called during recovery when a NameLN delete/truncate is not followed by the appropriate MapLN deletion.
void	envConfigUpdate(DbConfigManager configManager, EnvironmentMutableConfig ignore) Notifies the observer that one or more mutable properties have been changed.
void	executeRecoveredTasks() After recovery, queue any incomplete scans for execution.
Set<Long>	getCompletedRecordScans() Get a copy of the completed scan IDs before starting a checkpoint.
long	getLastLocalId()
long	getLastReplicatedId()
static JEVersion	getMinJEVersion()
boolean	isEmpty()
boolean	isEnabled()
boolean	isScanTaskActive(long id)
static long	materializeKey(DatabaseEntry keyEntry)
void	prepareForDbExtinction(ReplicationContext repContext) Must be called before writing a NameLN, when startDbExtinction(com.sleepycat.je.dbi.DatabaseImpl) will be called, i.e., for a DB remove/truncate operation.
void	recoverIncompleteTasks() During recovery, save incomplete scan tasks after reading them from the scans DB.
void	replay(byte[] scanKey, byte[] scanData) Queues a task to process a scan LN that was replayed on this replica.
void	requestShutdown()
void	setLastIds(long lastRepScanID, long lastNonRepScanID)
void	shutdown()
void	startDbExtinction(DatabaseImpl dbImpl) Used after transaction commit or non-transactional operation end in these cases: purge the deleted database after a commit of Environment.removeDatabase purge the deleted database after a commit of Environment.truncateDatabase purge the newly created database after an abort of Environment.truncateDatabase

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TEST_MIN_JE_VERSION

public static JEVersion TEST_MIN_JE_VERSION

dbBeforeWriteTaskHook

public TestHook<?> dbBeforeWriteTaskHook

Constructor Detail

ExtinctionScanner

public ExtinctionScanner(@NonNull EnvironmentImpl envImpl)

Method Detail

getMinJEVersion

public static JEVersion getMinJEVersion()

shutdown

public void shutdown()

requestShutdown

public void requestShutdown()

envConfigUpdate

public void envConfigUpdate(DbConfigManager configManager,
                            EnvironmentMutableConfig ignore)

Description copied from interface: EnvConfigObserver

Notifies the observer that one or more mutable properties have been changed.

Specified by:

envConfigUpdate in interface EnvConfigObserver

isEnabled

public boolean isEnabled()

isEmpty

public boolean isEmpty()

getLastLocalId

public long getLastLocalId()

getLastReplicatedId

public long getLastReplicatedId()

setLastIds

public void setLastIds(long lastRepScanID,
                       long lastNonRepScanID)

isScanTaskActive

public boolean isScanTaskActive(long id)

Returns:

whether the given ID is still present in the extinction scan DB, meaning that that scan is not complete.

recoverIncompleteTasks

public void recoverIncompleteTasks()

During recovery, save incomplete scan tasks after reading them from the scans DB. Incomplete scans are held until after recovery is finished, and then executed by executeRecoveredTasks(). If an extinction is requested at the end of recovery (e.g., for temp DB deletion), it is queued rather than being executed immediately.

executeRecoveredTasks

public void executeRecoveredTasks()

After recovery, queue any incomplete scans for execution.

replay

public void replay(byte[] scanKey,
                   byte[] scanData)

Queues a task to process a scan LN that was replayed on this replica. The LN replay must be done before this method is called, since RecordExtinction#isTaskCommitted() assumes that the record is in the Btree.

materializeKey

public static long materializeKey(DatabaseEntry keyEntry)

discardExtinctRecords

public long discardExtinctRecords(@NonNull Locker locker,
                                  @NonNull Set<String> dbNames,
                                  @Nullable DatabaseEntry beginKey,
                                  @Nullable DatabaseEntry endKey,
                                  @Nullable ScanFilter filter,
                                  @NonNull String label)

Inserts the scan record in the DB and queues the scan for execution. Caller must check for a read-write env and isEnabled().

Throws:

DatabaseNotFoundException - if a DB name does not exist.

getCompletedRecordScans

public Set<Long> getCompletedRecordScans()

Get a copy of the completed scan IDs before starting a checkpoint.

deleteCompletedRecordScans

public void deleteCompletedRecordScans(Set<Long> scanIds)

After a checkpoint, delete the scans previously returned by getCompletedRecordScans().

prepareForDbExtinction

public void prepareForDbExtinction(ReplicationContext repContext)

Must be called before writing a NameLN, when startDbExtinction(com.sleepycat.je.dbi.DatabaseImpl) will be called, i.e., for a DB remove/truncate operation. This method ensures that the scan DB can be opened on the replica when replaying the NameLN.

Returns without taking any action if this is a replica and the NameLN for the scan DB has not yet been replayed (the DB does not exist), and the truncate/remove operation itself is not replicated. See startDbExtinction for handling of this special case.

Throws:

ReplicaWriteException - if this is a replica node and the scan DB NameLN has not yet been replayed.

startDbExtinction

public void startDbExtinction(DatabaseImpl dbImpl)
                       throws DatabaseException

Used after transaction commit or non-transactional operation end in these cases:

• purge the deleted database after a commit of Environment.removeDatabase
• purge the deleted database after a commit of Environment.truncateDatabase
• purge the newly created database after an abort of Environment.truncateDatabase

Note that the processing of the naming tree means the MapLN is never actually accessible from the current tree, but making the DB extinct is necessary to reclaim the memory and disk space it occupies.

This method initiates an async task to perform the following:

• Release the INs for the deleted database,
• count all log entries for this database as obsolete,
• delete the MapLN,
• set the DatabaseImpl state to DELETE_FINISHED,
• and call DbTree.releaseDb.

In one special case the task is executed synchronously (see below).

Throws:

DatabaseException

ensureDbExtinction

public void ensureDbExtinction(DatabaseImpl dbImpl)

Called during recovery when a NameLN delete/truncate is not followed by the appropriate MapLN deletion. Add an async task for the DB extinction if one does not already exist. Ensures that DbTree.releaseDb is called.