com.sleepycat.je.dbi

Class TTL

java.lang.Object

com.sleepycat.je.dbi.TTL

public class TTL
extends Object

Internal documentation and utility functions for the TTL feature. Repeatable-read ----------------- As described in WriteOptions.setTTL(int), repeatable-read is supported in simple cases by treating a record that expires after being locked as if it were not expired. This is implemented and documented in CursorImpl.lockLN(com.sleepycat.je.txn.LockType). Unfortunately, we must check for whether a lock is already owned or shared by the locker before we attempt to lock the record. To optimize and avoid this extra overhead when it is unnecessary, we only do this when a record might expire during the transaction, according to the EnvironmentParams.ENV_TTL_MAX_TXN_TIME threshold. When a slot contains an expired record, CursorImpl.lockLN(com.sleepycat.je.txn.LockType) returns true in the LockStanding.defunct field, just as it does for deleted records. That way deleted records and expired records are filtered out of queries in the same way. Locking (read or write locks) also protects a record from being purged. The cleaner only considers an LN expired if its lock is uncontended, meaning that it could write-lock it. It places locked LNs on the pending LN queue. The compressor also only removes an expired slot if its lock is uncontended. However, if the clock was changed, purging may have occurred. Therefore, when an LN being fetched is in a cleaned file (LOG_FILE_NOT_FOUND), we treat it as a deleted record if it expires within EnvironmentParams.ENV_TTL_CLOCK_TOLERANCE. Records for which IN.fetchLN(int, com.sleepycat.je.CacheMode) returns null must also be filtered out of queries. This can happen even after locking the record and determining that the slot is not expired. To prevent an LN from being purged while an operation is attempting to lock it, due to thread scheduling, we purge LNs only if they are already expired by at least EnvironmentParams.ENV_TTL_MAX_TXN_TIME. This is done to compensate for the fact that the BIN is latched by the cleaner when locking an expired LN, while all other LN locking does latch the BIN. This also means that, when calculating utilization of a .jdb file, we don't consider LNs expired until the ENV_TTL_MAX_TXN_TIME after their expiration time. There are several special cases involving LNs discovered to be purged after locking the record. In the cases where the operation fails, the situation is documented in WriteOptions.setTTL(int). + For a read operation with a non-null 'data' param, if the LN was previously locked but the data was not requested, and the LN is found to be purged during the read, the operation fails (returns null). + For an update operation with a partial 'data' param, if the LN was previously locked (but the data was not requested), and the LN is found to be purged during the update, the operation fails (returns null). + For an update of a primary record with secondary keys, if the record is locked and then we find the LN has been purged, we simply don't delete any pre-existing secondary keys. This is OK because those secondary records are also expired and will be purged naturally. Note that when the expiration time is reduced, including setting it to zero, no special handling is needed. The update operation itself will ensure that the expiration time in the BIN and LN are in sync, in the case of a single record, and that a primary record and its associated and secondary records have expiration times that are in sync. Since expiration checking always occurs after locking, the updated expiration time will always be used. Secondaries ----------- Locking also supports repeatable-read for secondaries, as long as the records being accessed were locked. To make this work when reading via a secondary, we must lock the secondary if it expires within EnvironmentParams.ENV_TTL_MAX_TXN_TIME. Normally we don't lock the secondary at all in this case, and rely only on the primary record lock. This extra lock is taken after the primary lock, so locking order it not violated, i.e., this does not increase the potential for deadlocks. When reading via a secondary, if the secondary exists but the primary record expired (within EnvironmentParams.ENV_TTL_CLOCK_TOLERANCE), then we we treat the record as deleted. When updating or deleting a primary record and its associated secondary records, we ignore integrity problems if the secondary record has expired (within EnvironmentParams.ENV_TTL_CLOCK_TOLERANCE). Specifically we ignore the integrity error when: 1. we are deleting the secondary record and it does not exist; 2. we are updating secondary record and it does not exist -- in this case we insert it. Maximum TTL value ----------------- TTL maximum values are limited by the largest expiration time that can be stored in an 'int'. We use 'int' values for the expiration base in the BIN class and for the expiration in the LN class. Perhaps 'long' values could be used in these cases, but for now the implementation is limited to 'int'. The largest expiration time in hours is Integer.MAX_VALUE. We use hours because the BIN class sometimes must store expiration times in hours, even if they were originally specified in days. This is roughly 245,000 years from the system base time (1970). The max TTL value is the delta between the current time and the max expiration time. For simplicity we divide the max expiration time in half and limit TTL values to roughly 120,000 years. This is how the values of WriteOptions.TTL_MAX_HOURS and WriteOptions.TTL_MAX_DAYS are derived.

Field Summary

Fields

Modifier and Type	Field and Description
static long	MILLIS_PER_DAY
static long	MILLIS_PER_HOUR
static JEVersion	TEST_MIN_JE_VERSION

Constructor Summary

Constructors

Constructor and Description
TTL()

Method Summary

Modifier and Type	Method and Description
static long	currentSystemTime()
static long	expirationToSystemTime(int expiration, boolean hours) Translates from expiration days or hours to a Java time in ms.
static boolean	expiresWithin(int expiration, boolean hours, long withinMs) Returns whether the given expiration time is LT the current system time plus withinMs.
static boolean	expiresWithin(long expirationTime, long withinMs) Same as expiresWithin(int, boolean, long) but with a single expirationTime param.
static String	formatExpiration(int expiration, boolean hours) For logging and debugging output.
static String	formatExpirationTime(long time) For logging and debugging output.
static JEVersion	getMinJEVersion()
static boolean	isExpired(int expiration, boolean hours) Returns whether a given expiration time precedes the current system time, i.e., the expiration time has passed.
static boolean	isExpired(long expirationTime) Returns whether a given expiration time precedes the current system time, i.e., the expiration time has passed.
static boolean	isSystemTimeInHours(long systemMs) Returns whether the given time in millis, when converted to hours, rounding up, is not an even multiple of 24.
static void	setTimeTestHook(TestHook<Long> hook) Sets a hook for simulating changes in the clock time that is used in TTL processing.
static int	systemTimeToExpiration(long systemMs, boolean hours) Converts the user-supplied expirationTime parameter to an internal expiration time in days or hours.
static int	ttlToExpiration(int ttl, TimeUnit ttlUnits) Translates from the user-supplied ttl parameters to the expiration value that we store internally.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MILLIS_PER_HOUR

public static final long MILLIS_PER_HOUR

See Also:

Constant Field Values

MILLIS_PER_DAY

public static final long MILLIS_PER_DAY

See Also:

Constant Field Values

TEST_MIN_JE_VERSION

public static JEVersion TEST_MIN_JE_VERSION

Constructor Detail

TTL

public TTL()

Method Detail

getMinJEVersion

public static JEVersion getMinJEVersion()

setTimeTestHook

public static void setTimeTestHook(TestHook<Long> hook)

Sets a hook for simulating changes in the clock time that is used in TTL processing. If the hook is non-null, TestHook.getHookValue() returns the value used as the system clock time for all TTL processing. Other methods in the hook interface are not used.

For unit testing, this might return a fixed time. For stress testing, this might return a time that advances more quickly than the real clock.

currentSystemTime

public static long currentSystemTime()

expirationToSystemTime

public static long expirationToSystemTime(int expiration,
                                          boolean hours)

Translates from expiration days or hours to a Java time in ms.

ttlToExpiration

public static int ttlToExpiration(int ttl,
                                  TimeUnit ttlUnits)

Translates from the user-supplied ttl parameters to the expiration value that we store internally. Validates the ttl parameters as a side effect.

isSystemTimeInHours

public static boolean isSystemTimeInHours(long systemMs)

Returns whether the given time in millis, when converted to hours, rounding up, is not an even multiple of 24.

systemTimeToExpiration

public static int systemTimeToExpiration(long systemMs,
                                         boolean hours)

Converts the user-supplied expirationTime parameter to an internal expiration time in days or hours. Assumes that the user parameter is evenly divisible by days or hours (call isSystemTimeInHours first).

formatExpiration

public static String formatExpiration(int expiration,
                                      boolean hours)

For logging and debugging output.

formatExpirationTime

public static String formatExpirationTime(long time)

For logging and debugging output.

isExpired

public static boolean isExpired(int expiration,
                                boolean hours)

Returns whether a given expiration time precedes the current system time, i.e., the expiration time has passed.

isExpired

public static boolean isExpired(long expirationTime)

Returns whether a given expiration time precedes the current system time, i.e., the expiration time has passed.

expiresWithin

public static boolean expiresWithin(int expiration,
                                    boolean hours,
                                    long withinMs)

Returns whether the given expiration time is LT the current system time plus withinMs. withinMs may be negative to check whether the expiration time is LT the current system time minus abs(withinMs).

expiresWithin

public static boolean expiresWithin(long expirationTime,
                                    long withinMs)

Same as expiresWithin(int, boolean, long) but with a single expirationTime param.