UtilizationProfile (starrocks-bdb-je 18.3.17 API)

java.lang.Object
- com.sleepycat.je.cleaner.UtilizationProfile

```
public class UtilizationProfile
extends Object
```
The UP tracks utilization summary information for all log files.
Unlike the UtilizationTracker, the UP is not accessed under the log write latch and is instead synchronized on itself for protecting the cache. It is not accessed during the primary data access path, except for when flushing (writing) file summary LNs. This occurs in the following cases:
1. The summary information is flushed at the end of a checkpoint. This allows tracking to occur in memory in between checkpoints, and replayed during recovery.
2. When committing the truncateDatabase and removeDatabase operations, the summary information is flushed because detail tracking for those operations is not replayed during recovery
3. The evictor will ask the UtilizationTracker to flush the largest summary if the memory taken by the tracker exeeds its budget.
The cache is populated by the RecoveryManager just before performing the initial checkpoint. The UP must be open and populated in order to respond to requests to flush summaries and to evict tracked detail, even if the cleaner is disabled.

WARNING: While synchronized on this object, eviction is not permitted. If it were, this could cause deadlocks because the order of locking would be the UP object and then the evictor. During normal eviction the order is to first lock the evictor and then the UP, when evicting tracked detail.

The methods in this class synchronize to protect the cached summary information. Some methods also access the UP database. However, because eviction must not occur while synchronized, UP database access is not performed while synchronized except in one case: when inserting a new summary record. In that case we disallow eviction during the database operation.

Constructor Summary

Constructors
Constructor and Description

UtilizationProfile(EnvironmentImpl env, UtilizationTracker tracker)
Creates an empty UP.

Constructors
Constructor and Description
`UtilizationProfile(EnvironmentImpl env, UtilizationTracker tracker)` Creates an empty UP.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`flushFileUtilization(Collection<TrackedFileSummary> activeFiles)` Flush a FileSummaryLN node for each given TrackedFileSummary.
`void`	`flushLocalTracker(LocalUtilizationTracker localTracker)` Count the given locally tracked info as obsolete and then log the file info.
`int`	`getFileSize(Long file)` Gets the size of the file.
`SortedMap<Long,Integer>`	`getFileSizeSummaryMap()` Returns a simplified copy of the current file summary map, i.e.
`FileSummary`	`getFileSummary(Long file)` Gets the base summary from the cached map.
`SortedMap<Long,FileSummary>`	`getFileSummaryMap(boolean includeTrackedFiles)` Returns a copy of the current file summary map, optionally including tracked summary information, for use by the DbSpace utility and by unit tests.
`long[]`	`getObsoleteDetailSorted(Long fileNum)` Returns the sorted obsolete offsets for the given file.
`boolean`	`hasFileSummaryLN(long fileNum)` Returns whether an LN exists in the file summary DB for the given file.
`boolean`	`hasReservedFileLN(long fileNum)` Returns whether an LN exists in the reserved file DB for the given file.
`void`	`populateCache(StartupTracker.Counter counter, RecoveryInfo recoveryInfo)` Populate the profile for file selection.
`static void`	`setIncompleteReactivateHook(TestHook hook)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - UtilizationProfile
```
public UtilizationProfile(EnvironmentImpl env,
                          UtilizationTracker tracker)
```
    Creates an empty UP.
- Method Detail
  - getFileSummary
```
public FileSummary getFileSummary(Long file)
```
    Gets the base summary from the cached map. Add the tracked summary, if one exists, to the base summary. Sets all entries obsolete, if the file is in the migrateFiles set.
  - flushLocalTracker
```
public void flushLocalTracker(LocalUtilizationTracker localTracker)
                       throws DatabaseException
```
    Count the given locally tracked info as obsolete and then log the file info.
    
    Throws:
    
    DatabaseException
  - flushFileUtilization
```
public void flushFileUtilization(Collection<TrackedFileSummary> activeFiles)
                          throws DatabaseException
```
    Flush a FileSummaryLN node for each given TrackedFileSummary.
    
    Throws:
    
    DatabaseException
  - getFileSummaryMap
```
public SortedMap<Long,FileSummary> getFileSummaryMap(boolean includeTrackedFiles)
```
    Returns a copy of the current file summary map, optionally including tracked summary information, for use by the DbSpace utility and by unit tests. The returned map's key is a Long file number and its value is a FileSummary.
  - getFileSize
```
public int getFileSize(Long file)
```
    Gets the size of the file. If the file does not exist in fileSummaryMap, then return -1.
  - getFileSizeSummaryMap
```
public SortedMap<Long,Integer> getFileSizeSummaryMap()
```
    Returns a simplified copy of the current file summary map, i.e. the value is only the total size of the file. Besides, we are not sure whether the FileSummary for the current last file is complete, so we remove the entry for the last file.
  - getObsoleteDetailSorted
```
public long[] getObsoleteDetailSorted(Long fileNum)
                               throws DatabaseException
```
    Returns the sorted obsolete offsets for the given file.
    
    Throws:
    
    DatabaseException
  - populateCache
```
public void populateCache(StartupTracker.Counter counter,
                          RecoveryInfo recoveryInfo)
                   throws DatabaseException
```
    Populate the profile for file selection. This method performs eviction and is not synchronized. It must be called before recovery is complete so that synchronization is unnecessary. It should be called before the recovery checkpoint so that the checkpoint can flush dirty metadata. After a file is cleaned, at checkpoint end it is moved to reserved status and these steps are taken to delete its metadata: 1. The file is cleaned and a checkpoint flushes the updated INs. 2. Utilization info in MapLNs referencing the file are updated in cache and the MapLNs are dirtied. At some point later, the dirty MapLNs are flushed by a normally scheduled checkpoint. This could happen after any of the steps below. 3. A record is inserted for the file in the reserved file db. 4. All file summary db records for the file are deleted. When the reserved file is deleted, this is done in these steps. 5. The file itself is deleted. Note that because we do not flush the log before deleting the file, step 3 and 4 may not be durable. 6. The files's record in the reserved file db is deleted. Data file deletion and file metadata deletion cannot be performed atomically. In fact none of the steps are grouped atomically into transactions, and no-sync logging is used when writing to the file summary db and the reserved file db. A crash can occur at any point, and we must handle that in some way, either in recovery or here (in the populateCache method). - A crash prior to durable completion of step 3 will be handled naturally by cleaning the file again, since it is present in the file summary db but not in the reserved file db. - If step 3 completes durably but a crash occurs before the dirty MapLNs are flushed, this is indicated by the presence of a reserved file record in the recovery interval, and step 2 will be repeated here during recovery. - If step 3 completes durably but not step 4 or 5, then step 4 is completed here. - If step 3 and 5 complete durably but not step 4 and 6, then steps 4 and 6 are completed here. - If step 4 and 5 complete durably but not step 6, then step 6 is completed here. Prior to log version 15, the reserved file db did not exist and reserved files were re-cleaned if they were not deleted before a crash. The old steps were: A. The file is cleaned and a checkpoint flushes the updated INs. B. If the file is reserved, this is tracked in memory but nothing is logged. When deleting a file: C. The file itself is deleted. D. All file summary db records for the file are deleted. This is handled now as follows, which is the same as the old approach. - A crash prior to step C results in re-cleaning the file. - If step C completes but step D does not, step D is performed here.
    
    Throws:
    
    DatabaseException
  - hasReservedFileLN
```
public boolean hasReservedFileLN(long fileNum)
```
    Returns whether an LN exists in the reserved file DB for the given file. Intended for testing.
  - hasFileSummaryLN
```
public boolean hasFileSummaryLN(long fileNum)
```
    Returns whether an LN exists in the file summary DB for the given file. Intended for testing.
  - setIncompleteReactivateHook
```
public static void setIncompleteReactivateHook(TestHook hook)
```

Class UtilizationProfile

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

UtilizationProfile

Method Detail

getFileSummary

flushLocalTracker

flushFileUtilization

getFileSummaryMap

getFileSize

getFileSizeSummaryMap

getObsoleteDetailSorted

populateCache

hasReservedFileLN

hasFileSummaryLN

setIncompleteReactivateHook