Class JournalImpl
- java.lang.Object
-
- org.apache.activemq.artemis.core.journal.impl.JournalImpl
-
- All Implemented Interfaces:
JournalRecordProvider,Journal,TestableJournal,ActiveMQComponent
public class JournalImpl extends Object implements TestableJournal, JournalRecordProvider
A circular log implementation.
Look at
load(LoaderCallback)for the file layout
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.apache.activemq.artemis.core.journal.Journal
Journal.JournalState
-
-
Field Summary
Fields Modifier and Type Field Description static byteADD_RECORDstatic byteADD_RECORD_TXstatic StringBKPstatic byteCOMMIT_RECORDstatic byteDELETE_RECORDstatic byteDELETE_RECORD_TXstatic byteEVENT_RECORDprotected intfileSizeprotected static byteFILL_CHARACTERstatic intFORMAT_VERSIONprotected ExecutorFactoryioExecutorFactorystatic intMIN_FILE_SIZEstatic bytePREPARE_RECORDprotected booleanremoveExtraFilesOnLoadstatic byteROLLBACK_RECORDstatic intSIZE_ADD_RECORDstatic intSIZE_ADD_RECORD_TXstatic intSIZE_COMMIT_RECORDstatic intSIZE_COMPLETE_TRANSACTION_RECORDstatic intSIZE_DELETE_RECORDstatic intSIZE_DELETE_RECORD_TXstatic intSIZE_HEADERstatic intSIZE_PREPARE_RECORDstatic intSIZE_ROLLBACK_RECORDstatic doubleUPDATE_FACTORthis is a factor where when you have more than UPDATE_FACTOR updates for every ADD.static byteUPDATE_RECORDstatic byteUPDATE_RECORD_TX
-
Constructor Summary
Constructors Constructor Description JournalImpl(int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, int journalFileOpenTimeout, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)JournalImpl(int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO)JournalImpl(int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)JournalImpl(ExecutorFactory ioExecutors, int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, int journalFileOpenTimeout, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)JournalImpl(ExecutorFactory ioExecutors, int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, int journalFileOpenTimeout, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion, IOCriticalErrorListener criticalErrorListener, int maxAtticFiles)JournalImpl(ExecutorFactory ioExecutors, int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidappendAddEvent(long id, byte recordType, Persister persister, Object record, boolean sync, IOCompletion callback)An event is data recorded on the journal, but it won't have any weight or deletes.voidappendAddRecord(long id, byte recordType, byte[] record, boolean sync)voidappendAddRecord(long id, byte recordType, Persister persister, Object record, boolean sync)voidappendAddRecord(long id, byte recordType, Persister persister, Object record, boolean sync, IOCompletion callback)voidappendAddRecordTransactional(long txID, long id, byte recordType, byte[] record)voidappendAddRecordTransactional(long txID, long id, byte recordType, Persister persister, Object record)voidappendCommitRecord(long txID, boolean sync)voidappendCommitRecord(long txID, boolean sync, IOCompletion callback)voidappendCommitRecord(long txID, boolean sync, IOCompletion callback, boolean lineUpContext)Regarding the number of operations in a given file seeJournalCompleteRecordTX.voidappendDeleteRecord(long id, boolean sync)voidappendDeleteRecord(long id, boolean sync, IOCompletion callback)voidappendDeleteRecordTransactional(long txID, long id)voidappendDeleteRecordTransactional(long txID, long id, byte[] record)voidappendDeleteRecordTransactional(long txID, long id, EncodingSupport record)voidappendPrepareRecord(long txID, byte[] transactionData, boolean sync)voidappendPrepareRecord(long txID, EncodingSupport transactionData, boolean sync)If the system crashed after a prepare was called, it should store information that is required to bring the transaction back to a state it could be committed.voidappendPrepareRecord(long txID, EncodingSupport transactionData, boolean sync, IOCompletion callback)If the system crashed after a prepare was called, it should store information that is required to bring the transaction back to a state it could be committed.voidappendRollbackRecord(long txID, boolean sync)voidappendRollbackRecord(long txID, boolean sync, IOCompletion callback)voidappendUpdateRecord(long id, byte recordType, byte[] record, boolean sync)voidappendUpdateRecord(long id, byte recordType, Persister persister, Object record, boolean sync)voidappendUpdateRecord(long id, byte recordType, Persister persister, Object record, boolean sync, IOCompletion callback)voidappendUpdateRecordTransactional(long txID, long id, byte recordType, byte[] record)voidappendUpdateRecordTransactional(long txID, long id, byte recordType, Persister persister, Object record)booleancheckReclaimStatus()This method is called automatically when a new file is opened.voidcompact()Note: This method can't be called from the main executor, as it will invoke other methods depending on it.protected SequentialFilecreateControlFile(List<JournalFile> files, List<JournalFile> newFiles, Pair<String,String> cleanupRename)Map<Long,JournalFile>createFilesForBackupSync(long[] fileIds)Returns Map with aJournalFilefor all existing files.Stringdebug()voiddebugWait()Method for use on testcases.protected voiddeleteControlFile(SequentialFile controlFile)voidflush()It will make sure there are no more pending operations on the Executors.booleanflushAppendExecutor(long timeout, TimeUnit unit)voidforceBackup(int timeout, TimeUnit unit)voidforceMoveNextFile()Force the usage of a newJournalFile.intgetAlignment()intgetCompactCount()For tests onlyintgetCompactMinFiles()JournalCompactorgetCompactor()floatgetCompactPercentage()IOCriticalErrorListenergetCriticalErrorListener()JournalFilegetCurrentFile()JournalFile[]getDataFiles()Returns theJournalFiles in use.intgetDataFilesCount()StringgetDatePortion(String name)longgetDatePortionMillis(String name)StringgetFileExtension()SequentialFileFactorygetFileFactory()StringgetFilePrefix()intgetFileSize()JournalFilesRepositorygetFilesRepository()intgetFreeFilesCount()StringgetHistoryFileName(long sequence, Calendar calendar)intgetIDMapSize()intgetMaxAIO()longgetMaxRecordSize()The max size record that can be stored in the journalintgetMinFiles()intgetNumberOfRecords()intgetOpenedFilesCount()ConcurrentLongHashMap<JournalRecord>getRecords()protected SyncIOCompletiongetSyncCallback(boolean sync)intgetUserVersion()static intinitFileHeader(SequentialFileFactory fileFactory, SequentialFile sequentialFile, int userVersion, long fileID)booleanisAutoReclaim()booleanisHistory()booleanisRemoveExtraFilesOnLoad()booleanisStarted()voidlineUpContext(IOCompletion callback)JournalLoadInformationload(List<RecordInfo> committedRecords, List<PreparedTransactionInfo> preparedTransactions, TransactionFailureCallback transactionFailure, boolean fixBadTx)JournalLoadInformationload(LoaderCallback loadManager)Load data accordingly to the record layoutsJournalLoadInformationload(SparseArrayLinkedList<RecordInfo> committedRecords, List<PreparedTransactionInfo> preparedTransactions, TransactionFailureCallback failureCallback, boolean fixBadTX)JournalLoadInformationloadInternalOnly()Load internal data structures and not expose any data.JournalLoadInformationloadSyncOnly(Journal.JournalState syncState)Load internal data structures, and remain waiting for synchronization to complete.protected voidmoveNextFile(boolean scheduleReclaim, boolean blockOnClose)You need to guarantee lock.acquire() before calling this method!protected voidonCompactDone()This is an interception point for testcases, when the compacted files are written, before replacing the data structuresprotected voidonCompactLockingTheJournal()This is an interception point for testcases, when the compacted files are written, to be called as soon as the compactor gets a writeLockprotected voidonCompactStart()This is an interception point for testcases, when the compacted files are written, before replacing the data structuresList<JournalFile>orderFiles()this method is used internally only however tools may use it to maintenance.voidprocessBackup()With the exception of initialization, this has to be always called within the compactorExecutorvoidprocessBackupCleanup()It will check max files and max days on files and remove extra files.JournalFileImplreadFileHeader(SequentialFile file)static intreadJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader)this method is used internally only however tools may use it to maintenance.static intreadJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader, AtomicReference<ByteBuffer> wholeFileBufferReference)static intreadJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader, AtomicReference<ByteBuffer> wholeFileBufferReference, boolean reclaimed)static intreadJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader, AtomicReference<ByteBuffer> wholeFileBufferReference, boolean reclaimed, io.netty.util.collection.ByteObjectHashMap<Boolean> replaceableRecords)StringremoveBackupExtension(String name)protected static StringrenameExtensionFile(String name, String extension)protected voidrenameFiles(List<JournalFile> oldFiles, List<JournalFile> newFiles)being protected as testcases can override this methodvoidreplaceableRecord(byte recordType)This will declare a record type as being replaceable on updates.voidreplicationSyncFinished()Restarts file reclaim and compacting on the journal.voidreplicationSyncPreserveOldFiles()Stops any operation that may delete or modify old (stale) data.voidscheduleCompactAndBlock(int timeout)This method will start compact using the compactorExecutor and block up to timeout secondsvoidsetAutoReclaim(boolean autoReclaim)JournalImplsetCriticalErrorListener(IOCriticalErrorListener criticalErrorListener)JournalImplsetHistoryFolder(File historyFolder, long maxBytes, long period)voidsetRemoveExtraFilesOnLoad(boolean setting)protected JournalFilesetUpCurrentFile(int lastDataPos)voidstart()voidstop()protected JournalFileswitchFileIfNecessary(int size)voidsynchronizationLock()Write lock the Journal and write lock the compacting process.voidsynchronizationUnlock()Unlock the Journal and the compacting process.voidtestCompact()StringtoString()voidtryAppendDeleteRecord(long id, boolean sync, JournalUpdateCallback updateCallback, IOCompletion callback)voidtryAppendDeleteRecord(long id, JournalUpdateCallback updateCallback, boolean sync)voidtryAppendUpdateRecord(long id, byte recordType, byte[] record, JournalUpdateCallback updateCallback, boolean sync, boolean replaceableRecord)voidtryAppendUpdateRecord(long id, byte recordType, Persister persister, Object record, boolean sync, boolean replaceableUpdate, JournalUpdateCallback updateCallback, IOCompletion callback)voidtryAppendUpdateRecord(long id, byte recordType, Persister persister, Object record, JournalUpdateCallback updateCallback, boolean sync, boolean replaceableUpdate)protected SequentialFilewriteControlFile(SequentialFileFactory fileFactory, List<JournalFile> files, List<JournalFile> newFiles, List<Pair<String,String>> renames)static voidwriteHeader(ActiveMQBuffer buffer, int userVersion, long fileID)-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.activemq.artemis.core.server.ActiveMQComponent
asyncStop
-
Methods inherited from interface org.apache.activemq.artemis.core.journal.Journal
appendAddRecord, appendAddRecord, appendAddRecord, appendAddRecord, appendAddRecordTransactional, appendAddRecordTransactional, appendCommitRecord, appendCommitRecord, appendDeleteRecord, appendDeleteRecordTransactional, appendDeleteRecordTransactional, appendPrepareRecord, appendPrepareRecord, appendRollbackRecord, appendUpdateRecord, appendUpdateRecord, appendUpdateRecord, appendUpdateRecord, appendUpdateRecordTransactional, appendUpdateRecordTransactional, isRemoveExtraFilesOnLoad, load, load, setRemoveExtraFilesOnLoad, tryAppendDeleteRecord, tryAppendUpdateRecord, tryAppendUpdateRecord, tryAppendUpdateRecord, tryAppendUpdateRecord
-
-
-
-
Field Detail
-
UPDATE_FACTOR
public static final double UPDATE_FACTOR
this is a factor where when you have more than UPDATE_FACTOR updates for every ADD. When this happens we should issue a compacting event. I don't foresee users needing to configure this value. However if this ever happens we would have a system property aligned for this. With that being said, if you needed this, please raise an issue on why you needed to use this, so we may eventually add it to broker.xml when a real use case would determine the configuration exposed in there. To update this value, define a System Property org.apache.activemq.artemis.core.journal.impl.JournalImpl.UPDATE_FACTOR=YOUR VALUE We only calculate this against replaceable updates, on this case for redelivery counts and rescheduled redelivery in artemis server
-
BKP
public static final String BKP
- See Also:
- Constant Field Values
-
FORMAT_VERSION
public static final int FORMAT_VERSION
- See Also:
- Constant Field Values
-
MIN_FILE_SIZE
public static final int MIN_FILE_SIZE
- See Also:
- Constant Field Values
-
SIZE_HEADER
public static final int SIZE_HEADER
- See Also:
- Constant Field Values
-
SIZE_ADD_RECORD
public static final int SIZE_ADD_RECORD
- See Also:
- Constant Field Values
-
EVENT_RECORD
public static final byte EVENT_RECORD
- See Also:
- Constant Field Values
-
ADD_RECORD
public static final byte ADD_RECORD
- See Also:
- Constant Field Values
-
UPDATE_RECORD
public static final byte UPDATE_RECORD
- See Also:
- Constant Field Values
-
SIZE_ADD_RECORD_TX
public static final int SIZE_ADD_RECORD_TX
- See Also:
- Constant Field Values
-
ADD_RECORD_TX
public static final byte ADD_RECORD_TX
- See Also:
- Constant Field Values
-
UPDATE_RECORD_TX
public static final byte UPDATE_RECORD_TX
- See Also:
- Constant Field Values
-
SIZE_DELETE_RECORD_TX
public static final int SIZE_DELETE_RECORD_TX
- See Also:
- Constant Field Values
-
DELETE_RECORD_TX
public static final byte DELETE_RECORD_TX
- See Also:
- Constant Field Values
-
SIZE_DELETE_RECORD
public static final int SIZE_DELETE_RECORD
- See Also:
- Constant Field Values
-
DELETE_RECORD
public static final byte DELETE_RECORD
- See Also:
- Constant Field Values
-
SIZE_COMPLETE_TRANSACTION_RECORD
public static final int SIZE_COMPLETE_TRANSACTION_RECORD
- See Also:
- Constant Field Values
-
SIZE_PREPARE_RECORD
public static final int SIZE_PREPARE_RECORD
- See Also:
- Constant Field Values
-
PREPARE_RECORD
public static final byte PREPARE_RECORD
- See Also:
- Constant Field Values
-
SIZE_COMMIT_RECORD
public static final int SIZE_COMMIT_RECORD
- See Also:
- Constant Field Values
-
COMMIT_RECORD
public static final byte COMMIT_RECORD
- See Also:
- Constant Field Values
-
SIZE_ROLLBACK_RECORD
public static final int SIZE_ROLLBACK_RECORD
- See Also:
- Constant Field Values
-
ROLLBACK_RECORD
public static final byte ROLLBACK_RECORD
- See Also:
- Constant Field Values
-
FILL_CHARACTER
protected static final byte FILL_CHARACTER
- See Also:
- Constant Field Values
-
ioExecutorFactory
protected ExecutorFactory ioExecutorFactory
-
fileSize
protected final int fileSize
-
removeExtraFilesOnLoad
protected boolean removeExtraFilesOnLoad
-
-
Constructor Detail
-
JournalImpl
public JournalImpl(int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO)
-
JournalImpl
public JournalImpl(int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)
-
JournalImpl
public JournalImpl(int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, int journalFileOpenTimeout, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)
-
JournalImpl
public JournalImpl(ExecutorFactory ioExecutors, int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)
-
JournalImpl
public JournalImpl(ExecutorFactory ioExecutors, int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, int journalFileOpenTimeout, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion)
-
JournalImpl
public JournalImpl(ExecutorFactory ioExecutors, int fileSize, int minFiles, int poolSize, int compactMinFiles, int compactPercentage, int journalFileOpenTimeout, SequentialFileFactory fileFactory, String filePrefix, String fileExtension, int maxAIO, int userVersion, IOCriticalErrorListener criticalErrorListener, int maxAtticFiles)
-
-
Method Detail
-
setHistoryFolder
public JournalImpl setHistoryFolder(File historyFolder, long maxBytes, long period) throws Exception
- Specified by:
setHistoryFolderin interfaceJournal- Throws:
Exception
-
replaceableRecord
public void replaceableRecord(byte recordType)
This will declare a record type as being replaceable on updates. Certain update records only need the last value, and they could be replaceable during compacting.- Specified by:
replaceableRecordin interfaceJournal
-
getCompactPercentage
public float getCompactPercentage()
-
getCompactMinFiles
public int getCompactMinFiles()
-
getFilesRepository
public JournalFilesRepository getFilesRepository()
-
getCriticalErrorListener
public IOCriticalErrorListener getCriticalErrorListener()
- Specified by:
getCriticalErrorListenerin interfaceJournal
-
setCriticalErrorListener
public JournalImpl setCriticalErrorListener(IOCriticalErrorListener criticalErrorListener)
- Specified by:
setCriticalErrorListenerin interfaceJournal
-
getRecords
public ConcurrentLongHashMap<JournalRecord> getRecords()
- Specified by:
getRecordsin interfaceJournalRecordProvider
-
getCurrentFile
public JournalFile getCurrentFile()
- Specified by:
getCurrentFilein interfaceTestableJournal
-
getCompactor
public JournalCompactor getCompactor()
- Specified by:
getCompactorin interfaceJournalRecordProvider
-
orderFiles
public List<JournalFile> orderFiles() throws Exception
this method is used internally only however tools may use it to maintenance. It won't be part of the interface as the tools should be specific to the implementation- Throws:
Exception
-
readJournalFile
public static int readJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader, AtomicReference<ByteBuffer> wholeFileBufferReference) throws Exception
- Throws:
Exception
-
readJournalFile
public static int readJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader, AtomicReference<ByteBuffer> wholeFileBufferReference, boolean reclaimed) throws Exception
- Throws:
Exception
-
readJournalFile
public static int readJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader, AtomicReference<ByteBuffer> wholeFileBufferReference, boolean reclaimed, io.netty.util.collection.ByteObjectHashMap<Boolean> replaceableRecords) throws Exception
- Throws:
Exception
-
readJournalFile
public static int readJournalFile(SequentialFileFactory fileFactory, JournalFile file, JournalReaderCallback reader) throws Exception
this method is used internally only however tools may use it to maintenance.- Throws:
Exception
-
appendAddRecord
public void appendAddRecord(long id, byte recordType, Persister persister, Object record, boolean sync, IOCompletion callback) throws Exception- Specified by:
appendAddRecordin interfaceJournal- Throws:
Exception
-
appendAddEvent
public void appendAddEvent(long id, byte recordType, Persister persister, Object record, boolean sync, IOCompletion callback) throws ExceptionDescription copied from interface:JournalAn event is data recorded on the journal, but it won't have any weight or deletes. It's always ready to be removed. It is useful on recovery data while in use with backup history journal.- Specified by:
appendAddEventin interfaceJournal- Throws:
Exception
-
appendUpdateRecord
public void appendUpdateRecord(long id, byte recordType, Persister persister, Object record, boolean sync, IOCompletion callback) throws Exception- Specified by:
appendUpdateRecordin interfaceJournal- Throws:
Exception
-
tryAppendUpdateRecord
public void tryAppendUpdateRecord(long id, byte recordType, Persister persister, Object record, boolean sync, boolean replaceableUpdate, JournalUpdateCallback updateCallback, IOCompletion callback) throws Exception- Specified by:
tryAppendUpdateRecordin interfaceJournal- Throws:
Exception
-
appendDeleteRecord
public void appendDeleteRecord(long id, boolean sync, IOCompletion callback) throws Exception- Specified by:
appendDeleteRecordin interfaceJournal- Throws:
Exception
-
tryAppendDeleteRecord
public void tryAppendDeleteRecord(long id, boolean sync, JournalUpdateCallback updateCallback, IOCompletion callback) throws Exception- Specified by:
tryAppendDeleteRecordin interfaceJournal- Throws:
Exception
-
appendAddRecordTransactional
public void appendAddRecordTransactional(long txID, long id, byte recordType, Persister persister, Object record) throws Exception- Specified by:
appendAddRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendUpdateRecordTransactional
public void appendUpdateRecordTransactional(long txID, long id, byte recordType, Persister persister, Object record) throws Exception- Specified by:
appendUpdateRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendDeleteRecordTransactional
public void appendDeleteRecordTransactional(long txID, long id, EncodingSupport record) throws Exception- Specified by:
appendDeleteRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendPrepareRecord
public void appendPrepareRecord(long txID, EncodingSupport transactionData, boolean sync, IOCompletion callback) throws ExceptionIf the system crashed after a prepare was called, it should store information that is required to bring the transaction back to a state it could be committed.
transactionData allows you to store any other supporting user-data related to the transaction
This method also uses the same logic applied on
Journal.appendCommitRecord(long, boolean)- Specified by:
appendPrepareRecordin interfaceJournal- Parameters:
txID-transactionData- extra user data for the prepare- Throws:
Exception
-
lineUpContext
public void lineUpContext(IOCompletion callback)
- Specified by:
lineUpContextin interfaceJournal
-
appendCommitRecord
public void appendCommitRecord(long txID, boolean sync, IOCompletion callback, boolean lineUpContext) throws ExceptionRegarding the number of operations in a given file seeJournalCompleteRecordTX.- Specified by:
appendCommitRecordin interfaceJournallineUpContext- if appendCommitRecord should call a storeLineUp. This is because the caller may have already taken into account- Throws:
Exception
-
appendRollbackRecord
public void appendRollbackRecord(long txID, boolean sync, IOCompletion callback) throws Exception- Specified by:
appendRollbackRecordin interfaceJournal- Throws:
Exception
-
getAlignment
public int getAlignment() throws Exception- Specified by:
getAlignmentin interfaceJournal- Throws:
Exception
-
loadInternalOnly
public JournalLoadInformation loadInternalOnly() throws Exception
Description copied from interface:JournalLoad internal data structures and not expose any data. This is only useful if you're using the journal but not interested on the current data. Useful in situations where the journal is being replicated, copied... etc.- Specified by:
loadInternalOnlyin interfaceJournal- Throws:
Exception
-
loadSyncOnly
public JournalLoadInformation loadSyncOnly(Journal.JournalState syncState) throws Exception
Description copied from interface:JournalLoad internal data structures, and remain waiting for synchronization to complete.- Specified by:
loadSyncOnlyin interfaceJournal- Parameters:
syncState- the current state of the journal, this parameter ensures consistency.- Throws:
Exception
-
load
public JournalLoadInformation load(List<RecordInfo> committedRecords, List<PreparedTransactionInfo> preparedTransactions, TransactionFailureCallback transactionFailure, boolean fixBadTx) throws Exception
-
load
public JournalLoadInformation load(SparseArrayLinkedList<RecordInfo> committedRecords, List<PreparedTransactionInfo> preparedTransactions, TransactionFailureCallback failureCallback, boolean fixBadTX) throws Exception
- Specified by:
loadin interfaceJournal- Throws:
Exception- See Also:
load(LoaderCallback)
-
scheduleCompactAndBlock
public void scheduleCompactAndBlock(int timeout) throws ExceptionDescription copied from interface:JournalThis method will start compact using the compactorExecutor and block up to timeout seconds- Specified by:
scheduleCompactAndBlockin interfaceJournal- Parameters:
timeout- the timeout in seconds or block forever if<= 0- Throws:
Exception
-
compact
public void compact()
Note: This method can't be called from the main executor, as it will invoke other methods depending on it. Note: only synchronized methods on journal are methods responsible for the life-cycle such as stop, start records will still come as this is being executed
-
load
public JournalLoadInformation load(LoaderCallback loadManager) throws Exception
Load data accordingly to the record layouts
Basic record layout:
Field Name Size RecordType Byte (1) FileID Integer (4 bytes) Compactor Counter 1 byte TransactionID (if record is transactional) Long (8 bytes) RecordID Long (8 bytes) BodySize(Add, update and delete) Integer (4 bytes) UserDefinedRecordType (If add/update only) Byte (1) RecordBody Byte Array (size=BodySize) Check Size Integer (4 bytes) The check-size is used to validate if the record is valid and complete
Commit/Prepare record layout:
Field Name Size RecordType Byte (1) FileID Integer (4 bytes) Compactor Counter 1 byte TransactionID (if record is transactional) Long (8 bytes) ExtraDataLength (Prepares only) Integer (4 bytes) Number Of Files (N) Integer (4 bytes) ExtraDataBytes Bytes (sized by ExtraDataLength) * FileID(n) Integer (4 bytes) * NumberOfElements(n) Integer (4 bytes) CheckSize Integer (4 bytes) * FileID and NumberOfElements are the transaction summary, and they will be repeated (N)umberOfFiles times
-
processBackupCleanup
public void processBackupCleanup()
Description copied from interface:JournalIt will check max files and max days on files and remove extra files.- Specified by:
processBackupCleanupin interfaceJournal
-
processBackup
public void processBackup()
With the exception of initialization, this has to be always called within the compactorExecutor- Specified by:
processBackupin interfaceJournal
-
getDatePortionMillis
public long getDatePortionMillis(String name)
-
checkReclaimStatus
public final boolean checkReclaimStatus() throws ExceptionDescription copied from interface:TestableJournalThis method is called automatically when a new file is opened.It will among other things, remove stale files and make them available for reuse.
This method locks the journal.
- Specified by:
checkReclaimStatusin interfaceTestableJournal- Returns:
- true if cleanup was called
- Throws:
Exception
-
setAutoReclaim
public final void setAutoReclaim(boolean autoReclaim)
- Specified by:
setAutoReclaimin interfaceTestableJournal
-
isAutoReclaim
public final boolean isAutoReclaim()
- Specified by:
isAutoReclaimin interfaceTestableJournal
-
debug
public String debug() throws Exception
- Specified by:
debugin interfaceTestableJournal- Throws:
Exception
-
debugWait
public void debugWait() throws InterruptedExceptionMethod for use on testcases. It will call waitComplete on every transaction, so any assertions on the file system will be correct after this- Specified by:
debugWaitin interfaceTestableJournal- Throws:
InterruptedException
-
flush
public void flush() throws ExceptionDescription copied from interface:JournalIt will make sure there are no more pending operations on the Executors.
-
getMaxRecordSize
public long getMaxRecordSize()
The max size record that can be stored in the journal- Specified by:
getMaxRecordSizein interfaceJournal- Returns:
-
flushAppendExecutor
public boolean flushAppendExecutor(long timeout, TimeUnit unit) throws InterruptedException- Throws:
InterruptedException
-
getDataFilesCount
public int getDataFilesCount()
- Specified by:
getDataFilesCountin interfaceTestableJournal
-
getDataFiles
public JournalFile[] getDataFiles()
Description copied from interface:JournalReturns theJournalFiles in use.- Specified by:
getDataFilesin interfaceJournal- Specified by:
getDataFilesin interfaceTestableJournal- Returns:
- array with all
JournalFiles in use
-
getFreeFilesCount
public int getFreeFilesCount()
- Specified by:
getFreeFilesCountin interfaceTestableJournal
-
getOpenedFilesCount
public int getOpenedFilesCount()
- Specified by:
getOpenedFilesCountin interfaceTestableJournal
-
getIDMapSize
public int getIDMapSize()
- Specified by:
getIDMapSizein interfaceTestableJournal
-
getFileSize
public int getFileSize()
- Specified by:
getFileSizein interfaceJournal- Specified by:
getFileSizein interfaceTestableJournal
-
getMinFiles
public int getMinFiles()
- Specified by:
getMinFilesin interfaceTestableJournal
-
getFilePrefix
public String getFilePrefix()
- Specified by:
getFilePrefixin interfaceTestableJournal
-
getFileExtension
public String getFileExtension()
- Specified by:
getFileExtensionin interfaceTestableJournal
-
getMaxAIO
public int getMaxAIO()
- Specified by:
getMaxAIOin interfaceTestableJournal
-
getUserVersion
public int getUserVersion()
- Specified by:
getUserVersionin interfaceJournal
-
forceMoveNextFile
public void forceMoveNextFile() throws ExceptionDescription copied from interface:JournalForce the usage of a newJournalFile.- Specified by:
forceMoveNextFilein interfaceJournal- Specified by:
forceMoveNextFilein interfaceTestableJournal- Throws:
Exception
-
forceBackup
public void forceBackup(int timeout, TimeUnit unit) throws Exception- Specified by:
forceBackupin interfaceJournal- Throws:
Exception
-
isStarted
public boolean isStarted()
- Specified by:
isStartedin interfaceActiveMQComponent
-
start
public void start()
- Specified by:
startin interfaceActiveMQComponent
-
stop
public void stop() throws Exception- Specified by:
stopin interfaceActiveMQComponent- Throws:
Exception
-
getNumberOfRecords
public int getNumberOfRecords()
- Specified by:
getNumberOfRecordsin interfaceJournal
-
createControlFile
protected SequentialFile createControlFile(List<JournalFile> files, List<JournalFile> newFiles, Pair<String,String> cleanupRename) throws Exception
- Throws:
Exception
-
writeControlFile
protected SequentialFile writeControlFile(SequentialFileFactory fileFactory, List<JournalFile> files, List<JournalFile> newFiles, List<Pair<String,String>> renames) throws Exception
- Throws:
Exception
-
deleteControlFile
protected void deleteControlFile(SequentialFile controlFile) throws Exception
- Throws:
Exception
-
renameFiles
protected void renameFiles(List<JournalFile> oldFiles, List<JournalFile> newFiles) throws Exception
being protected as testcases can override this method- Throws:
Exception
-
renameExtensionFile
protected static String renameExtensionFile(String name, String extension)
- Parameters:
name-- Returns:
-
onCompactStart
protected void onCompactStart() throws ExceptionThis is an interception point for testcases, when the compacted files are written, before replacing the data structures- Throws:
Exception
-
onCompactLockingTheJournal
protected void onCompactLockingTheJournal() throws ExceptionThis is an interception point for testcases, when the compacted files are written, to be called as soon as the compactor gets a writeLock- Throws:
Exception
-
onCompactDone
protected void onCompactDone()
This is an interception point for testcases, when the compacted files are written, before replacing the data structures
-
readFileHeader
public JournalFileImpl readFileHeader(SequentialFile file) throws Exception
- Parameters:
file-- Returns:
- Throws:
Exception
-
initFileHeader
public static int initFileHeader(SequentialFileFactory fileFactory, SequentialFile sequentialFile, int userVersion, long fileID) throws Exception
- Parameters:
fileID-sequentialFile-- Throws:
Exception
-
writeHeader
public static void writeHeader(ActiveMQBuffer buffer, int userVersion, long fileID)
- Parameters:
buffer-userVersion-fileID-
-
synchronizationLock
public final void synchronizationLock()
Description copied from interface:JournalWrite lock the Journal and write lock the compacting process. Necessary only during replication for backup synchronization.- Specified by:
synchronizationLockin interfaceJournal
-
synchronizationUnlock
public final void synchronizationUnlock()
Description copied from interface:JournalUnlock the Journal and the compacting process.- Specified by:
synchronizationUnlockin interfaceJournal- See Also:
Journal.synchronizationLock()
-
createFilesForBackupSync
public Map<Long,JournalFile> createFilesForBackupSync(long[] fileIds) throws Exception
Returns Map with aJournalFilefor all existing files. These are the files needed to be sent to a backup in order to synchronize it.- Specified by:
createFilesForBackupSyncin interfaceJournal- Parameters:
fileIds-- Returns:
- map with the IDs and corresponding
JournalFiles - Throws:
Exception
-
getFileFactory
public SequentialFileFactory getFileFactory()
- Specified by:
getFileFactoryin interfaceJournal
-
setUpCurrentFile
protected JournalFile setUpCurrentFile(int lastDataPos) throws Exception
- Parameters:
lastDataPos-- Returns:
- Throws:
Exception
-
switchFileIfNecessary
protected JournalFile switchFileIfNecessary(int size) throws Exception
- Parameters:
size-- Returns:
- Throws:
Exception
-
moveNextFile
protected void moveNextFile(boolean scheduleReclaim, boolean blockOnClose) throws ExceptionYou need to guarantee lock.acquire() before calling this method!- Throws:
Exception
-
replicationSyncPreserveOldFiles
public void replicationSyncPreserveOldFiles()
Description copied from interface:JournalStops any operation that may delete or modify old (stale) data.Meant to be used during synchronization of data between a live server and its replicating (remote) backup. Old files must not be compacted or deleted during synchronization.
- Specified by:
replicationSyncPreserveOldFilesin interfaceJournal
-
replicationSyncFinished
public void replicationSyncFinished()
Description copied from interface:JournalRestarts file reclaim and compacting on the journal.Meant to be used to revert the effect of
Journal.replicationSyncPreserveOldFiles(). it should only be called once the synchronization of the backup and live servers is completed.- Specified by:
replicationSyncFinishedin interfaceJournal
-
testCompact
public void testCompact()
- Specified by:
testCompactin interfaceTestableJournal
-
getCompactCount
public int getCompactCount()
For tests only
-
setRemoveExtraFilesOnLoad
public void setRemoveExtraFilesOnLoad(boolean setting)
- Specified by:
setRemoveExtraFilesOnLoadin interfaceJournal
-
isRemoveExtraFilesOnLoad
public boolean isRemoveExtraFilesOnLoad()
- Specified by:
isRemoveExtraFilesOnLoadin interfaceJournal
-
appendAddRecord
public void appendAddRecord(long id, byte recordType, byte[] record, boolean sync) throws Exception- Specified by:
appendAddRecordin interfaceJournal- Throws:
Exception
-
appendAddRecord
public void appendAddRecord(long id, byte recordType, Persister persister, Object record, boolean sync) throws Exception- Specified by:
appendAddRecordin interfaceJournal- Throws:
Exception
-
appendCommitRecord
public void appendCommitRecord(long txID, boolean sync) throws Exception- Specified by:
appendCommitRecordin interfaceJournal- Throws:
Exception
-
appendCommitRecord
public void appendCommitRecord(long txID, boolean sync, IOCompletion callback) throws Exception- Specified by:
appendCommitRecordin interfaceJournal- Throws:
Exception
-
appendUpdateRecord
public void appendUpdateRecord(long id, byte recordType, byte[] record, boolean sync) throws Exception- Specified by:
appendUpdateRecordin interfaceJournal- Throws:
Exception
-
tryAppendUpdateRecord
public void tryAppendUpdateRecord(long id, byte recordType, byte[] record, JournalUpdateCallback updateCallback, boolean sync, boolean replaceableRecord) throws Exception- Specified by:
tryAppendUpdateRecordin interfaceJournal- Throws:
Exception
-
appendUpdateRecordTransactional
public void appendUpdateRecordTransactional(long txID, long id, byte recordType, byte[] record) throws Exception- Specified by:
appendUpdateRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendAddRecordTransactional
public void appendAddRecordTransactional(long txID, long id, byte recordType, byte[] record) throws Exception- Specified by:
appendAddRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendDeleteRecordTransactional
public void appendDeleteRecordTransactional(long txID, long id) throws Exception- Specified by:
appendDeleteRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendPrepareRecord
public void appendPrepareRecord(long txID, byte[] transactionData, boolean sync) throws Exception- Specified by:
appendPrepareRecordin interfaceJournal- Throws:
Exception
-
appendPrepareRecord
public void appendPrepareRecord(long txID, EncodingSupport transactionData, boolean sync) throws ExceptionDescription copied from interface:JournalIf the system crashed after a prepare was called, it should store information that is required to bring the transaction back to a state it could be committed.
transactionData allows you to store any other supporting user-data related to the transaction
- Specified by:
appendPrepareRecordin interfaceJournaltransactionData- - extra user data for the prepare- Throws:
Exception
-
appendDeleteRecordTransactional
public void appendDeleteRecordTransactional(long txID, long id, byte[] record) throws Exception- Specified by:
appendDeleteRecordTransactionalin interfaceJournal- Throws:
Exception
-
appendUpdateRecord
public void appendUpdateRecord(long id, byte recordType, Persister persister, Object record, boolean sync) throws Exception- Specified by:
appendUpdateRecordin interfaceJournal- Throws:
Exception
-
tryAppendUpdateRecord
public void tryAppendUpdateRecord(long id, byte recordType, Persister persister, Object record, JournalUpdateCallback updateCallback, boolean sync, boolean replaceableUpdate) throws Exception- Specified by:
tryAppendUpdateRecordin interfaceJournal- Throws:
Exception
-
appendRollbackRecord
public void appendRollbackRecord(long txID, boolean sync) throws Exception- Specified by:
appendRollbackRecordin interfaceJournal- Throws:
Exception
-
appendDeleteRecord
public void appendDeleteRecord(long id, boolean sync) throws Exception- Specified by:
appendDeleteRecordin interfaceJournal- Throws:
Exception
-
tryAppendDeleteRecord
public void tryAppendDeleteRecord(long id, JournalUpdateCallback updateCallback, boolean sync) throws Exception- Specified by:
tryAppendDeleteRecordin interfaceJournal- Throws:
Exception
-
getSyncCallback
protected SyncIOCompletion getSyncCallback(boolean sync)
-
-