Package org.apache.pulsar.broker.transaction.buffer

Interface TransactionBuffer

All Known Implementing Classes:
TopicTransactionBuffer, TransactionBufferDisable
@Beta public interface TransactionBuffer
A class represent a transaction buffer. The transaction buffer is per partition. All the messages published within transactions are appended to a transaction buffer. They are not visible to consumers or readers before the transaction is committed.

When committing transaction starts, the broker will append a `COMMITTED` marker to the data partition first to mark the transaction is committed. The broker knows the data ledger of the commit marker and calls commitTxn(TxnID, long) to commit and seal the buffer.

When the marker is appended to the data partition, all the entries are visible to the consumers. So a transaction reader TransactionBufferReader will be opened to read the entries when the broker sees the commit marker. There is a chance broker crashes after writing the marker to data partition but before committing the transaction in transaction buffer. That is fine. Because the transaction buffer will fail opening the transaction buffer reader since the transaction is still marked as open. The broker can keep retry until the TC (transaction coordinator) eventually commits the buffer again.

  • Method Details

    • getTransactionMeta

      CompletableFuture<TransactionMeta> getTransactionMeta(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Return the metadata of a transaction in the buffer.
      Parameters:
      txnID - the transaction id
      Returns:
      a future represents the result of the operation
      Throws:
      org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

    • appendBufferToTxn

      CompletableFuture<org.apache.bookkeeper.mledger.Position> appendBufferToTxn(org.apache.pulsar.client.api.transaction.TxnID txnId, long sequenceId, io.netty.buffer.ByteBuf buffer)
      Append the buffer to the transaction buffer.

      The entry will be indexed by txnId and sequenceId.

      Parameters:
      txnId - the transaction id
      sequenceId - the sequence id of the entry in this transaction buffer.
      buffer - the entry buffer
      Returns:
      a future represents the result of the operation.
      Throws:
      org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionSealedException - if the transaction has been sealed.

    • openTransactionBufferReader

      CompletableFuture<TransactionBufferReader> openTransactionBufferReader(org.apache.pulsar.client.api.transaction.TxnID txnID, long startSequenceId)
      Open a TransactionBufferReader to read entries of a given transaction starting from the provided sequenceId.
      Parameters:
      txnID - transaction id
      startSequenceId - the sequence id to start read
      Returns:
      a future represents the result of open operation.
      Throws:
      org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

    • commitTxn

      CompletableFuture<Void> commitTxn(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark)
      Commit the transaction and seal the buffer for this transaction.

      If a transaction is sealed, no more entries can be appendBufferToTxn(TxnID, long, ByteBuf).

      Parameters:
      txnID - the transaction id
      lowWaterMark - the low water mark of this transaction
      Returns:
      a future represents the result of commit operation.
      Throws:
      org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

    • abortTxn

      CompletableFuture<Void> abortTxn(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark)
      Abort the transaction and all the entries of this transaction will be discarded.
      Parameters:
      txnID - the transaction id
      lowWaterMark - the low water mark of this transaction
      Returns:
      a future represents the result of abort operation.
      Throws:
      org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

    • purgeTxns

      CompletableFuture<Void> purgeTxns(List<Long> dataLedgers)
      Purge all the data of the transactions who are committed and stored in the provided data ledgers.

      This method will be called by the broker before they delete the ledgers. It ensures that all the transactions committed in those ledgers can be purged.

      Parameters:
      dataLedgers - the list of data ledgers.
      Returns:
      a future represents the result of purge operations.

    • clearSnapshot

      CompletableFuture<Void> clearSnapshot()
      Clear up the snapshot of the TransactionBuffer.
      Returns:
      Clear up operation result.

    • closeAsync

      CompletableFuture<Void> closeAsync()
      Close the buffer asynchronously.
      Returns:

    • isTxnAborted

      boolean isTxnAborted(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Close the buffer asynchronously.
      Parameters:
      txnID - TxnID txnId.
      Returns:
      the txnId is aborted.

    • syncMaxReadPositionForNormalPublish

      void syncMaxReadPositionForNormalPublish(org.apache.bookkeeper.mledger.impl.PositionImpl position)
      Sync max read position for normal publish.
      Parameters:
      position - PositionImpl the position to sync.

    • getMaxReadPosition

      org.apache.bookkeeper.mledger.impl.PositionImpl getMaxReadPosition()
      Get the can read max position.
      Returns:
      the stable position.

    • getTransactionInBufferStats

      org.apache.pulsar.common.policies.data.TransactionInBufferStats getTransactionInBufferStats(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Get transaction in buffer stats.
      Returns:
      the transaction in buffer stats.

    • getStats

      org.apache.pulsar.common.policies.data.TransactionBufferStats getStats(boolean lowWaterMarks)
      Get transaction stats in buffer.
      Returns:
      the transaction stats in buffer.

    • checkIfTBRecoverCompletely

      CompletableFuture<Void> checkIfTBRecoverCompletely(boolean isTxn)
      Wait TransactionBuffer Recovers completely. Take snapshot after TB Recovers completely.
      Parameters:
      isTxn -
      Returns:
      a future which has completely if isTxn = false. Or a future return by takeSnapshot.

    • getOngoingTxnCount

      long getOngoingTxnCount()

    • getAbortedTxnCount

      long getAbortedTxnCount()

    • getCommittedTxnCount

      long getCommittedTxnCount()