Package org.apache.pulsar.broker.transaction.buffer

Interface TransactionBuffer

  • All Known Implementing Classes:
    TopicTransactionBuffer, TransactionBufferDisable
    @Beta
public interface TransactionBuffer
    A class represent a transaction buffer. The transaction buffer is per partition. All the messages published within transactions are appended to a transaction buffer. They are not visible to consumers or readers before the transaction is committed.

    When committing transaction starts, the broker will append a `COMMITTED` marker to the data partition first to mark the transaction is committed. The broker knows the data ledger of the commit marker and calls #commitTxn(TxnID, long, long) to commit and seal the buffer.

    When the marker is appended to the data partition, all the entries are visible to the consumers. So a transaction reader TransactionBufferReader will be opened to read the entries when the broker sees the commit marker. There is a chance broker crashes after writing the marker to data partition but before committing the transaction in transaction buffer. That is fine. Because the transaction buffer will fail opening the transaction buffer reader since the transaction is still marked as open. The broker can keep retry until the TC (transaction coordinator) eventually commits the buffer again.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      java.util.concurrent.CompletableFuture<java.lang.Void> abortTxn​(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark)
      Abort the transaction and all the entries of this transaction will be discarded.
      java.util.concurrent.CompletableFuture<org.apache.bookkeeper.mledger.Position> appendBufferToTxn​(org.apache.pulsar.client.api.transaction.TxnID txnId, long sequenceId, io.netty.buffer.ByteBuf buffer)
      Append the buffer to the transaction buffer.
      java.util.concurrent.CompletableFuture<java.lang.Void> checkIfTBRecoverCompletely​(boolean isTxn)
      Wait TransactionBuffer Recovers completely.
      java.util.concurrent.CompletableFuture<java.lang.Void> clearSnapshot()
      Clear up the snapshot of the TransactionBuffer.
      java.util.concurrent.CompletableFuture<java.lang.Void> closeAsync()
      Close the buffer asynchronously.
      java.util.concurrent.CompletableFuture<java.lang.Void> commitTxn​(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark)
      Commit the transaction and seal the buffer for this transaction.
      org.apache.bookkeeper.mledger.impl.PositionImpl getMaxReadPosition()
      Get the can read max position.
      org.apache.pulsar.common.policies.data.TransactionBufferStats getStats()
      Get transaction stats in buffer.
      org.apache.pulsar.common.policies.data.TransactionInBufferStats getTransactionInBufferStats​(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Get transaction in buffer stats.
      java.util.concurrent.CompletableFuture<TransactionMeta> getTransactionMeta​(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Return the metadata of a transaction in the buffer.
      boolean isTxnAborted​(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Close the buffer asynchronously.
      java.util.concurrent.CompletableFuture<TransactionBufferReader> openTransactionBufferReader​(org.apache.pulsar.client.api.transaction.TxnID txnID, long startSequenceId)
      Open a TransactionBufferReader to read entries of a given transaction starting from the provided sequenceId.
      java.util.concurrent.CompletableFuture<java.lang.Void> purgeTxns​(java.util.List<java.lang.Long> dataLedgers)
      Purge all the data of the transactions who are committed and stored in the provided data ledgers.
      void syncMaxReadPositionForNormalPublish​(org.apache.bookkeeper.mledger.impl.PositionImpl position)
      Sync max read position for normal publish.

    • Method Detail

      • getTransactionMeta

        java.util.concurrent.CompletableFuture<TransactionMeta> getTransactionMeta​(org.apache.pulsar.client.api.transaction.TxnID txnID)
        Return the metadata of a transaction in the buffer.
        Parameters:
        txnID - the transaction id
        Returns:
        a future represents the result of the operation
        Throws:
        org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

      • appendBufferToTxn

        java.util.concurrent.CompletableFuture<org.apache.bookkeeper.mledger.Position> appendBufferToTxn​(org.apache.pulsar.client.api.transaction.TxnID txnId,
                                                                                                 long sequenceId,
                                                                                                 io.netty.buffer.ByteBuf buffer)
        Append the buffer to the transaction buffer.

        The entry will be indexed by txnId and sequenceId.

        Parameters:
        txnId - the transaction id
        sequenceId - the sequence id of the entry in this transaction buffer.
        buffer - the entry buffer
        Returns:
        a future represents the result of the operation.
        Throws:
        org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionSealedException - if the transaction has been sealed.

      • openTransactionBufferReader

        java.util.concurrent.CompletableFuture<TransactionBufferReader> openTransactionBufferReader​(org.apache.pulsar.client.api.transaction.TxnID txnID,
                                                                                            long startSequenceId)
        Open a TransactionBufferReader to read entries of a given transaction starting from the provided sequenceId.
        Parameters:
        txnID - transaction id
        startSequenceId - the sequence id to start read
        Returns:
        a future represents the result of open operation.
        Throws:
        org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

      • commitTxn

        java.util.concurrent.CompletableFuture<java.lang.Void> commitTxn​(org.apache.pulsar.client.api.transaction.TxnID txnID,
                                                                 long lowWaterMark)
        Commit the transaction and seal the buffer for this transaction.

        If a transaction is sealed, no more entries can be appendBufferToTxn(TxnID, long, ByteBuf).

        Parameters:
        txnID - the transaction id
        lowWaterMark - the low water mark of this transaction
        Returns:
        a future represents the result of commit operation.
        Throws:
        org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

      • abortTxn

        java.util.concurrent.CompletableFuture<java.lang.Void> abortTxn​(org.apache.pulsar.client.api.transaction.TxnID txnID,
                                                                long lowWaterMark)
        Abort the transaction and all the entries of this transaction will be discarded.
        Parameters:
        txnID - the transaction id
        lowWaterMark - the low water mark of this transaction
        Returns:
        a future represents the result of abort operation.
        Throws:
        org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

      • purgeTxns

        java.util.concurrent.CompletableFuture<java.lang.Void> purgeTxns​(java.util.List<java.lang.Long> dataLedgers)
        Purge all the data of the transactions who are committed and stored in the provided data ledgers.

        This method will be called by the broker before they delete the ledgers. It ensures that all the transactions committed in those ledgers can be purged.

        Parameters:
        dataLedgers - the list of data ledgers.
        Returns:
        a future represents the result of purge operations.

      • clearSnapshot

        java.util.concurrent.CompletableFuture<java.lang.Void> clearSnapshot()
        Clear up the snapshot of the TransactionBuffer.
        Returns:
        Clear up operation result.

      • closeAsync

        java.util.concurrent.CompletableFuture<java.lang.Void> closeAsync()
        Close the buffer asynchronously.
        Returns:

      • isTxnAborted

        boolean isTxnAborted​(org.apache.pulsar.client.api.transaction.TxnID txnID)
        Close the buffer asynchronously.
        Parameters:
        txnID - TxnID txnId.
        Returns:
        the txnId is aborted.

      • syncMaxReadPositionForNormalPublish

        void syncMaxReadPositionForNormalPublish​(org.apache.bookkeeper.mledger.impl.PositionImpl position)
        Sync max read position for normal publish.
        Parameters:
        position - PositionImpl the position to sync.

      • getMaxReadPosition

        org.apache.bookkeeper.mledger.impl.PositionImpl getMaxReadPosition()
        Get the can read max position.
        Returns:
        the stable position.

      • getTransactionInBufferStats

        org.apache.pulsar.common.policies.data.TransactionInBufferStats getTransactionInBufferStats​(org.apache.pulsar.client.api.transaction.TxnID txnID)
        Get transaction in buffer stats.
        Returns:
        the transaction in buffer stats.

      • getStats

        org.apache.pulsar.common.policies.data.TransactionBufferStats getStats()
        Get transaction stats in buffer.
        Returns:
        the transaction stats in buffer.

      • checkIfTBRecoverCompletely

        java.util.concurrent.CompletableFuture<java.lang.Void> checkIfTBRecoverCompletely​(boolean isTxn)
        Wait TransactionBuffer Recovers completely. Take snapshot after TB Recovers completely.
        Parameters:
        isTxn -
        Returns:
        a future which has completely if isTxn = false. Or a future return by takeSnapshot.