com.sleepycat.je.tree

Class Tree

java.lang.Object

com.sleepycat.je.tree.Tree

All Implemented Interfaces:

Loggable

public final class Tree
extends Object
implements Loggable

Tree implements the JE B+Tree. A note on tree search patterns: There's a set of Tree.search* methods. Some clients of the tree use those search methods directly, whereas other clients of the tree tend to use methods built on top of search. The semantics of search* are they leave you pointing at a BIN or IN they don't tell you where the reference of interest is. The semantics of the get* methods are: they leave you pointing at a BIN or IN they return the index of the slot of interest they traverse down to whatever level is needed they are built on top of search* methods. For the future: Over time, we need to clarify which methods are to be used by clients of the tree. Preferably clients that call the tree use get*, although their are cases where they need visibility into the tree structure. Also, search* should return the location of the slot to save us a second binary search. Search Method Call Hierarchy ---------------------------- getFirst/LastNode search CALLED BY: CursorImpl.getFirstOrLast getNext/PrevBin getParentINForChildIN searchSubTree CALLED BY: DupConvert CursorImpl.getNext getParentINForChildIN IN.findParent does not use shared latching CALLED BY: Checkpointer.flushIN (doFetch=false, targetLevel=-1) FileProcessor.processIN (doFetch=true, targetLevel=LEVEL) Evictor.evictIN (doFetch=true, targetLevel=-1) RecoveryManager.replaceOrInsertChild (doFetch=true, targetLevel=-1) getNext/PrevBin (doFetch=true, targetLevel=-1) search searchSubTree CALLED BY: CursorImpl.searchAndPosition INCompressor to find BIN searchSubTree uses shared grandparent latching getParentBINForChildLN searchSplitsAllowed CALLED BY: RecoveryManager.redo RecoveryManager.recoveryUndo search CALLED BY: RecoveryManager.abortUndo RecoveryManager.rollbackUndo FileProcessor.processLN Cleaner.processPendingLN UtilizationProfile.verifyLsnIsObsolete (utility) findBinForInsert searchSplitsAllowed CALLED BY: CursorImpl.putInternal searchSplitsAllowed uses shared non-grandparent latching CALLED BY: DupConvert (instead of findBinForInsert, which needs a cursor) Possible Shared Latching Improvements ------------------------------------- By implementing shared latching for BINs we would get better concurrency in these cases: Reads when LN is in cache, or LN is not needed (key-only op, e.g., dups)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Tree.SearchType Embodies an enum for the type of search being performed.

Constructor Summary

Constructors

Constructor and Description
Tree() Create a tree that's being read in from the log.
Tree(DatabaseImpl database) Create a new tree.

Method Summary

Modifier and Type	Method and Description
void	delete(byte[] idKey) Deletes a BIN specified by key from the tree.
void	dump()
void	dumpLog(StringBuilder sb, boolean verbose) Write the object into the string buffer for log dumping.
String	dumpString(int nSpaces)
BIN	findBinForInsert(byte[] key, CacheMode cacheMode) Find the BIN that is relevant to the insert.
DatabaseImpl	getDatabase()
BIN	getFirstNode(CacheMode cacheMode) Find the leftmost node (IN or BIN) in the tree.
BIN	getLastNode(CacheMode cacheMode) Find the rightmost node (IN or BIN) in the tree.
int	getLogSize()
long	getMaxLNs() Cheaply calculates and returns the maximum possible number of LNs in the btree.
BIN	getNextBin(BIN bin, CacheMode cacheMode) Return a reference to the adjacent BIN.
IN	getNextIN(IN prevIn, boolean forward, boolean latchShared, CacheMode cacheMode) Returns the next IN in the tree before/after the given IN, and at the same level.
boolean	getParentBINForChildLN(TreeLocation location, byte[] key, boolean splitsAllowed, boolean blindDeltaOps, CacheMode cacheMode) Return a reference to the parent of this LN.
SearchResult	getParentINForChildIN(IN child, boolean useTargetLevel, boolean doFetch, CacheMode cacheMode) Search for the parent P of a given IN C (where C is viewed as a logical node; not as a java obj).
SearchResult	getParentINForChildIN(IN child, boolean useTargetLevel, boolean doFetch, CacheMode cacheMode, List<TrackingInfo> trackingList) This version of getParentINForChildIN does the same thing as the version above, but also adds a "trackingList" param.
SearchResult	getParentINForChildIN(long targetNodeId, byte[] targetKey, int targetLevel, int exclusiveLevel, boolean requireExactMatch, boolean doFetch, CacheMode cacheMode, List<TrackingInfo> trackingList) This version of getParentINForChildIN() is the actual implementation of the previous 2 versions (read the comments there), but it also implements one additional use cases via the extra "requireExactMatch" param.
BIN	getPrevBin(BIN bin, CacheMode cacheMode) Return a reference to the previous BIN.
IN	getResidentRootIN(boolean latched)
IN	getRootIN(CacheMode cacheMode) Helper to obtain the root IN with shared root latching.
IN	getRootINLatchedExclusive(CacheMode cacheMode) Helper to obtain the root IN with exclusive root latching.
IN	getRootINRootAlreadyLatched(CacheMode cacheMode, boolean exclusive) Helper to obtain the root IN, when the root latch is already held.
long	getRootLsn() Get LSN of the rootIN.
long	getTransactionId()
boolean	isRootResident() Perform a fast check to see if the root IN is resident.
void	latchRootLatchExclusive()
boolean	logicalEquals(Loggable other)
ChildReference	makeRootChildReference(Node target, byte[] key, long lsn)
void	readFromLog(ByteBuffer itemBuffer, int entryVersion) Initialize this object from the data in itemBuf.
void	rebuildINList() rebuildINList is used by recovery to add all the resident nodes to the IN list.
void	releaseRootLatch()
boolean	rootExists()
BIN	search(byte[] key, CacheMode cacheMode)
BIN	search(byte[] key, Tree.SearchType searchType, BINBoundary binBoundary, CacheMode cacheMode, Comparator<byte[]> comparator) Search the tree, starting at the root.
BIN	searchSplitsAllowed(byte[] key, CacheMode cacheMode) Do a key based search, permitting pre-emptive splits.
void	setCkptHook(TestHook hook)
void	setDatabase(DatabaseImpl database) Set the database for this tree.
void	setFetchINHook(TestHook hook)
void	setGetParentINHook(TestHook hook)
void	setRoot(ChildReference newRoot, boolean notLatched) Set the root for the tree.
void	setSearchHook(TestHook hook)
void	setTreeStatsAccumulator(TreeWalkerStatsAccumulator tSA)
void	setWaitHook(TestHook hook)
void	validateINList(IN parent) Debugging check that all resident nodes are on the INList and no stray nodes are present in the unused portion of the IN arrays.
IN	withRootLatchedExclusive(WithRootLatched wrl)
IN	withRootLatchedShared(WithRootLatched wrl)
void	writeToLog(ByteBuffer logBuffer) Serialize this object into the buffer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tree

public Tree(DatabaseImpl database)

Create a new tree.

Tree

public Tree()

Create a tree that's being read in from the log.

Method Detail

setDatabase

public void setDatabase(DatabaseImpl database)

Set the database for this tree. Used by recovery when recreating an existing tree.

getDatabase

public DatabaseImpl getDatabase()

Returns:

the database for this Tree.

latchRootLatchExclusive

public void latchRootLatchExclusive()
                             throws DatabaseException

Throws:

DatabaseException

releaseRootLatch

public void releaseRootLatch()
                      throws DatabaseException

Throws:

DatabaseException

setRoot

public void setRoot(ChildReference newRoot,
                    boolean notLatched)

Set the root for the tree. Should only be called within the root latch.

makeRootChildReference

public ChildReference makeRootChildReference(Node target,
                                             byte[] key,
                                             long lsn)

rootExists

public boolean rootExists()

isRootResident

public boolean isRootResident()

Perform a fast check to see if the root IN is resident. No latching is performed. To ensure that the root IN is not loaded by another thread, this method should be called while holding a write lock on the MapLN. That will prevent opening the DB in another thread, and potentially loading the root IN. [#13415]

getRootIN

public IN getRootIN(CacheMode cacheMode)
             throws DatabaseException

Helper to obtain the root IN with shared root latching. Optionally updates the generation of the root when latching it.

Throws:

DatabaseException

getRootINLatchedExclusive

public IN getRootINLatchedExclusive(CacheMode cacheMode)
                             throws DatabaseException

Helper to obtain the root IN with exclusive root latching. Optionally updates the generation of the root when latching it.

Throws:

DatabaseException

getRootINRootAlreadyLatched

public IN getRootINRootAlreadyLatched(CacheMode cacheMode,
                                      boolean exclusive)

Helper to obtain the root IN, when the root latch is already held.

getResidentRootIN

public IN getResidentRootIN(boolean latched)
                     throws DatabaseException

Throws:

DatabaseException

withRootLatchedExclusive

public IN withRootLatchedExclusive(WithRootLatched wrl)
                            throws DatabaseException

Throws:

DatabaseException

withRootLatchedShared

public IN withRootLatchedShared(WithRootLatched wrl)
                         throws DatabaseException

Throws:

DatabaseException

getRootLsn

public long getRootLsn()

Get LSN of the rootIN. Obtained without latching, should only be accessed while quiescent.

getMaxLNs

public long getMaxLNs()

Cheaply calculates and returns the maximum possible number of LNs in the btree.

delete

public void delete(byte[] idKey)
            throws NodeNotEmptyException,
                   CursorsExistException

Deletes a BIN specified by key from the tree. If the BIN resides in a subtree that can be pruned away, prune as much as possible, so we don't leave a branch that has no BINs. It's possible that the targeted BIN will now have entries, or will have resident cursors. Either will prevent deletion (see exceptions). Unlike splits, IN deletion does not immediately log the subtree parent or its ancestors. It is sufficient to simply dirty the subtree parent. Logging is not necessary for correctness, and if a checkpoint does not flush the subtree parent then recovery will add the BINs to the compressor queue when redoing the LN deletions.

Parameters:

idKey - - the identifier key of the node to delete.

Throws:

NodeNotEmptyException - if the BIN is not empty. The deletion is no longer possible.

CursorsExistException - is the BIN has cursors. The deletion should be retried later by the INCompressor.

getFirstNode

public BIN getFirstNode(CacheMode cacheMode)
                 throws DatabaseException

Find the leftmost node (IN or BIN) in the tree.

Returns:

the leftmost node in the tree, null if the tree is empty. The returned node is latched and the caller must release it.

Throws:

DatabaseException

getLastNode

public BIN getLastNode(CacheMode cacheMode)
                throws DatabaseException

Find the rightmost node (IN or BIN) in the tree.

Returns:

the rightmost node in the tree, null if the tree is empty. The returned node is latched and the caller must release it.

Throws:

DatabaseException

getNextBin

public BIN getNextBin(BIN bin,
                      CacheMode cacheMode)
               throws DatabaseException

Return a reference to the adjacent BIN.

Parameters:

bin - The BIN to find the next BIN for. This BIN is latched.

Returns:

The next BIN, or null if there are no more. The returned node is latched and the caller must release it. If null is returned, the argument BIN remains latched.

Throws:

DatabaseException

getPrevBin

public BIN getPrevBin(BIN bin,
                      CacheMode cacheMode)
               throws DatabaseException

Return a reference to the previous BIN.

Parameters:

bin - The BIN to find the next BIN for. This BIN is latched.

Returns:

The previous BIN, or null if there are no more. The returned node is latched and the caller must release it. If null is returned, the argument bin remains latched.

Throws:

DatabaseException

getNextIN

public IN getNextIN(IN prevIn,
                    boolean forward,
                    boolean latchShared,
                    CacheMode cacheMode)

Returns the next IN in the tree before/after the given IN, and at the same level. For example, if a BIN is passed in the prevIn parameter, the next BIN will be returned. TODO: A possible problem with this method is that we don't know for certain whether it works properly in the face of splits. There are comments below indicating it does. But the Cursor.checkForInsertion method was apparently added because getNextBin/getPrevBin didn't work properly, and may skip a BIN. So at least it didn't work properly in the distant past. Archeology and possibly testing are needed to find the truth. Hopefully it does now work, and Cursor.checkForInsertion can be removed. TODO: To eliminate EX latches on upper INs, a new getParentINForChildIN is needed, which will return with both the parent and the grandparent SH-latched. If we do this, then in Tree.getNextIN() the call to searchSubtree() will be able to do grandparent latching, and the call to parent.fetchIN(index) will also be replace with a local version of grandparent latching.

getParentINForChildIN

public SearchResult getParentINForChildIN(IN child,
                                          boolean useTargetLevel,
                                          boolean doFetch,
                                          CacheMode cacheMode)
                                   throws DatabaseException

Search for the parent P of a given IN C (where C is viewed as a logical node; not as a java obj). If found, P is returned latched exclusively. The method is used when C has been accessed "directly", i.e., not via a tree search, and we need to perform an operation on C that requires an update to its parent. Such situations arise during eviction (C has been accessed via the LRU list), checkpointing, and recovery (C has been read from the log and is not attached to the in-memory tree). The method uses C's identifierKey to search down the tree until: (a) doFetch is false and we need to access a node that is not cached. In this case, we are actually looking for the cached copies of both C and its parent, so a cache miss on the path to C is considered a failure. This search mode is used by the evictor: to evict C (which has been retrieved from the LRU), its parent must be found and EX-latched; however, if the C has been evicted already by another thread, there is nothing to do (C will GC-ed). or (b) We reach a node whose node id equals the C's node id. In this case, we know for sure that C still belongs to the BTree and its parent has been found. or (c) useTargetLevel is true and we reach a node P that is at one level above C's level. We know that P contains a slot S whose corresponding key range includes C's identifierKey. Since we haven't read the child node under S to check its node id, we cannot know for sure that C is still in the tree. Nevertheless, we consider this situation a success, i.e., P is the parent node we are looking for. In this search mode, after this method returns, the caller is expected to take further action based on the info in slot S. For example, if C was created by reading a log entry at LSN L, and the LSN at slot S is also L, then we know P is the real parent (and we have thus saved a possible extra I/O to refetch the C node from the log to check its node id). This search mode is used by the cleaner. or (d) None of the above conditions occur and the bottom of the BTree is reached. In this case, no parent exists (the child node is an old version of a node that has been removed from the BTree).

Parameters:

child - The child node for which to find the parent. This node is latched by the caller and is unlatched by this function before returning to the caller.

useTargetLevel - If true, the search is considered successful if a node P is reached at one level above C's level. P is the parent to return to the caller.

doFetch - if false, stop the search if we run into a non-resident child and assume that no parent exists.

cacheMode - The CacheMode for affecting the hotness of the nodes visited during the search.

Returns:

a SearchResult object. If the parent has been found, result.foundExactMatch is true, result.parent refers to that node, and result.index is the slot for the child IN inside the parent IN. Otherwise, result.foundExactMatch is false and result.parent is null.

Throws:

DatabaseException

getParentINForChildIN

public SearchResult getParentINForChildIN(IN child,
                                          boolean useTargetLevel,
                                          boolean doFetch,
                                          CacheMode cacheMode,
                                          List<TrackingInfo> trackingList)
                                   throws DatabaseException

This version of getParentINForChildIN does the same thing as the version above, but also adds a "trackingList" param. If trackingList is not null, the LSNs of the parents visited along the way are added to the list, as a debug tracing mechanism. This is meant to stay in production, to add information to the log.

Throws:

DatabaseException

getParentINForChildIN

public SearchResult getParentINForChildIN(long targetNodeId,
                                          byte[] targetKey,
                                          int targetLevel,
                                          int exclusiveLevel,
                                          boolean requireExactMatch,
                                          boolean doFetch,
                                          CacheMode cacheMode,
                                          List<TrackingInfo> trackingList)
                                   throws DatabaseException

This version of getParentINForChildIN() is the actual implementation of the previous 2 versions (read the comments there), but it also implements one additional use cases via the extra "requireExactMatch" param. requireExactMatch == false && doFetch == false In this case we are actually looking for the lowest cached ancestor of the C node. The method will always return a node (considered as the "parent") unless the BTree is empty (has no nodes at all). The returned node must be latched, but not necessarily in EX mode. This search mode is used by the checkpointer. The exclusiveLevel param: In general, if exclusiveLevel == L, nodes above L will be SH latched and nodes at or below L will be EX-latched. In all current use cases, L is set to 1 + C.level. Note that if doFetch == false, the normalized exclusiveLevel must be >= 2 so that loadIN can be called.

Throws:

DatabaseException

getParentBINForChildLN

public boolean getParentBINForChildLN(TreeLocation location,
                                      byte[] key,
                                      boolean splitsAllowed,
                                      boolean blindDeltaOps,
                                      CacheMode cacheMode)
                               throws DatabaseException

Return a reference to the parent of this LN. This searches through the tree and allows splits, if the splitsAllowed param is true. Set the tree location to the proper BIN parent whether or not the LN child is found. That's because if the LN is not found, recovery or abort will need to place it within the tree, and so we must point at the appropriate position.

When this method returns with location.bin non-null, the BIN is latched and must be unlatched by the caller. Note that location.bin may be non-null even if this method returns false.

Parameters:

location - a holder class to hold state about the location of our search. Sort of an internal cursor.

key - key to navigate through main key

splitsAllowed - true if this method is allowed to cause tree splits as a side effect. In practice, recovery can cause splits, but abort can't.

blindDeltaOps - Normally, if this method lands on a BIN-delta and the search key is not in that delta, it will mutate the delta to a full BIN to make sure whether the search key exists in the tree or not. However, by passing true for blindDeltaOps, the caller indicates that it doesn't really care whether the key is in the tree or not: it is going to insert the key in the BIN-delta, if not already there, essentially overwritting the slot that may exist in the full BIN. So, if blindDeltaOps is true, the method will not mutate a BIN-delta parent (unless the BIN-delta has no space for a slot insertion).

cacheMode - The CacheMode for affecting the hotness of the tree.

Returns:

true if node found in tree. If false is returned and there is the possibility that we can insert the record into a plausible parent we must also set - location.bin (may be null if no possible parent found) - location.lnKey (don't need to set if no possible parent).

Throws:

DatabaseException

findBinForInsert

public BIN findBinForInsert(byte[] key,
                            CacheMode cacheMode)

Find the BIN that is relevant to the insert. If the tree doesn't exist yet, then create the first IN and BIN. On return, the cursor is set to the BIN that is found or created, and the BIN is latched.

searchSplitsAllowed

public BIN searchSplitsAllowed(byte[] key,
                               CacheMode cacheMode)

Do a key based search, permitting pre-emptive splits. Returns the target node's parent.

search

public BIN search(byte[] key,
                  CacheMode cacheMode)

search

public BIN search(byte[] key,
                  Tree.SearchType searchType,
                  BINBoundary binBoundary,
                  CacheMode cacheMode,
                  Comparator<byte[]> comparator)

Search the tree, starting at the root. Depending on search type either (a) search for the BIN that *should* contain a given key, or (b) return the right-most or left-most BIN in the tree. Preemptive splitting is not done during the search.

Parameters:

key - - the key to search for, or null if searchType is LEFT or RIGHT.

searchType - - The type of tree search to perform. NORMAL means we're searching for key in the tree. LEFT/RIGHT means we're descending down the left or right side, resp.

binBoundary - - If non-null, information is returned about whether the BIN found is the first or last BIN in the database.

Returns:

- the BIN that matches the criteria, if any. Returns null if the root is null. BIN is latched (unless it's null) and must be unlatched by the caller. In a NORMAL search, it is the caller's responsibility to do the findEntry() call on the key and BIN to locate the entry (if any) that matches key.

rebuildINList

public void rebuildINList()
                   throws DatabaseException

rebuildINList is used by recovery to add all the resident nodes to the IN list.

Throws:

DatabaseException

validateINList

public void validateINList(IN parent)
                    throws DatabaseException

Debugging check that all resident nodes are on the INList and no stray nodes are present in the unused portion of the IN arrays.

Throws:

DatabaseException

getLogSize

public int getLogSize()

Specified by:

getLogSize in interface Loggable

Returns:

number of bytes used to store this object.

See Also:

Loggable.getLogSize()

writeToLog

public void writeToLog(ByteBuffer logBuffer)

Description copied from interface: Loggable

Serialize this object into the buffer.

Specified by:

writeToLog in interface Loggable

Parameters:

logBuffer - is the destination buffer

See Also:

Loggable.writeToLog(java.nio.ByteBuffer)

readFromLog

public void readFromLog(ByteBuffer itemBuffer,
                        int entryVersion)

Description copied from interface: Loggable

Initialize this object from the data in itemBuf.

Specified by:

readFromLog in interface Loggable

Parameters:

itemBuffer - the source buffer

entryVersion - the log version of the data

See Also:

Loggable.readFromLog(java.nio.ByteBuffer, int)

dumpLog

public void dumpLog(StringBuilder sb,
                    boolean verbose)

Description copied from interface: Loggable

Write the object into the string buffer for log dumping. Each object should be dumped without indentation or new lines and should be valid XML.

Specified by:

dumpLog in interface Loggable

Parameters:

sb - destination string buffer

verbose - if true, dump the full, verbose version

See Also:

Loggable.dumpLog(java.lang.StringBuilder, boolean)

getTransactionId

public long getTransactionId()

Specified by:

getTransactionId in interface Loggable

Returns:

the transaction id embedded within this loggable object. Objects that have no transaction id should return 0.

See Also:

Loggable.getTransactionId()

logicalEquals

public boolean logicalEquals(Loggable other)

Specified by:

logicalEquals in interface Loggable

Returns:

true if these two loggable items are logically the same. Used for replication testing.

See Also:

Always return false, this item should never be compared.

setTreeStatsAccumulator

public void setTreeStatsAccumulator(TreeWalkerStatsAccumulator tSA)

dump

public void dump()

dumpString

public String dumpString(int nSpaces)

setWaitHook

public void setWaitHook(TestHook hook)

setSearchHook

public void setSearchHook(TestHook hook)

setCkptHook

public void setCkptHook(TestHook hook)

setGetParentINHook

public void setGetParentINHook(TestHook hook)

setFetchINHook

public void setFetchINHook(TestHook hook)