Package org.apache.bookkeeper.mledger

Interface ManagedLedger

  • All Known Implementing Classes:
    ManagedLedgerImpl, ReadOnlyManagedLedgerImpl
    @LimitedPrivate
@Stable
public interface ManagedLedger
    A ManagedLedger it's a superset of a BookKeeper ledger concept.

    It mimics the concept of an appender log that:

    • has a unique name (chosen by clients) by which it can be created/opened/deleted
    • is always writable: if a writer process crashes, a new writer can re-open the ManagedLedger and continue writing into it
    • has multiple persisted consumers (see ManagedCursor), each of them with an associated position
    • when all the consumers have processed all the entries contained in a Bookkeeper ledger, the ledger is deleted

    Caveats:

    • A single ManagedLedger can only be open once at any time. Implementation can protect double access from the same VM, but accesses from different machines to the same ManagedLedger need to be avoided through an external source of coordination.

    • Method Detail

      • getName

        java.lang.String getName()
        Returns:
        the unique name of this ManagedLedger

      • addEntry

        Position addEntry​(byte[] data)
           throws java.lang.InterruptedException,
                  ManagedLedgerException
        Append a new entry to the end of a managed ledger.
        Parameters:
        data - data entry to be persisted
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • addEntry

        Position addEntry​(byte[] data,
                  int numberOfMessages)
           throws java.lang.InterruptedException,
                  ManagedLedgerException
        Append a new entry to the end of a managed ledger.
        Parameters:
        data - data entry to be persisted
        numberOfMessages - numberOfMessages of entry
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • asyncAddEntry

        void asyncAddEntry​(byte[] data,
                   AsyncCallbacks.AddEntryCallback callback,
                   java.lang.Object ctx)
        Append a new entry asynchronously.
        Parameters:
        data - data entry to be persisted
        callback - callback object
        ctx - opaque context
        See Also:
        addEntry(byte[])

      • addEntry

        Position addEntry​(byte[] data,
                  int offset,
                  int length)
           throws java.lang.InterruptedException,
                  ManagedLedgerException
        Append a new entry to the end of a managed ledger.
        Parameters:
        data - data entry to be persisted
        offset - offset in the data array
        length - number of bytes
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • addEntry

        Position addEntry​(byte[] data,
                  int numberOfMessages,
                  int offset,
                  int length)
           throws java.lang.InterruptedException,
                  ManagedLedgerException
        Append a new entry to the end of a managed ledger.
        Parameters:
        data - data entry to be persisted
        numberOfMessages - numberOfMessages of entry
        offset - offset in the data array
        length - number of bytes
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • asyncAddEntry

        void asyncAddEntry​(byte[] data,
                   int offset,
                   int length,
                   AsyncCallbacks.AddEntryCallback callback,
                   java.lang.Object ctx)
        Append a new entry asynchronously.
        Parameters:
        data - data entry to be persisted
        offset - offset in the data array
        length - number of bytes
        callback - callback object
        ctx - opaque context
        See Also:
        addEntry(byte[])

      • asyncAddEntry

        void asyncAddEntry​(byte[] data,
                   int numberOfMessages,
                   int offset,
                   int length,
                   AsyncCallbacks.AddEntryCallback callback,
                   java.lang.Object ctx)
        Append a new entry asynchronously.
        Parameters:
        data - data entry to be persisted
        numberOfMessages - numberOfMessages of entry
        offset - offset in the data array
        length - number of bytes
        callback - callback object
        ctx - opaque context
        See Also:
        addEntry(byte[])

      • asyncAddEntry

        void asyncAddEntry​(io.netty.buffer.ByteBuf buffer,
                   AsyncCallbacks.AddEntryCallback callback,
                   java.lang.Object ctx)
        Append a new entry asynchronously.
        Parameters:
        buffer - buffer with the data entry
        callback - callback object
        ctx - opaque context
        See Also:
        addEntry(byte[])

      • asyncAddEntry

        void asyncAddEntry​(io.netty.buffer.ByteBuf buffer,
                   int numberOfMessages,
                   AsyncCallbacks.AddEntryCallback callback,
                   java.lang.Object ctx)
        Append a new entry asynchronously.
        Parameters:
        buffer - buffer with the data entry
        numberOfMessages - numberOfMessages for data entry
        callback - callback object
        ctx - opaque context
        See Also:
        addEntry(byte[])

      • openCursor

        ManagedCursor openCursor​(java.lang.String name)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Open a ManagedCursor in this ManagedLedger.

        If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

        Parameters:
        name - the name associated with the ManagedCursor
        Returns:
        the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • openCursor

        ManagedCursor openCursor​(java.lang.String name,
                         org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Open a ManagedCursor in this ManagedLedger.

        If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

        Parameters:
        name - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at latest position or not when first created default is true
        Returns:
        the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • openCursor

        ManagedCursor openCursor​(java.lang.String name,
                         org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition,
                         java.util.Map<java.lang.String,​java.lang.Long> properties)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Open a ManagedCursor in this ManagedLedger.

        If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

        Parameters:
        name - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at latest position or not when first created default is true
        properties - user defined properties that will be attached to the first position of the cursor, if the open operation will trigger the creation of the cursor.
        Returns:
        the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • newNonDurableCursor

        ManagedCursor newNonDurableCursor​(Position startCursorPosition)
                           throws ManagedLedgerException
        Creates a new cursor whose metadata is not backed by durable storage. A caller can treat the non-durable cursor exactly like a normal cursor, with the only difference in that after restart it will not remember which entries were deleted. Also it does not prevent data from being deleted.

        The cursor is anonymous and can be positioned on an arbitrary position.

        This method is not-blocking.

        Parameters:
        startCursorPosition - the position where the cursor should be initialized, or null to start from the current latest entry. When starting on a particular cursor position, the first entry to be returned will be the entry next to the specified position
        Returns:
        the new NonDurableCursor
        Throws:
        ManagedLedgerException

      • asyncDeleteCursor

        void asyncDeleteCursor​(java.lang.String name,
                       AsyncCallbacks.DeleteCursorCallback callback,
                       java.lang.Object ctx)
        Delete a ManagedCursor asynchronously.
        Parameters:
        name - the name associated with the ManagedCursor
        callback - callback object
        ctx - opaque context
        See Also:
        deleteCursor(String)

      • deleteCursor

        void deleteCursor​(java.lang.String name)
           throws java.lang.InterruptedException,
                  ManagedLedgerException
        Remove a ManagedCursor from this ManagedLedger.

        If the cursor doesn't exist, the operation will still succeed.

        Parameters:
        name - the name associated with the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • asyncOpenCursor

        void asyncOpenCursor​(java.lang.String name,
                     AsyncCallbacks.OpenCursorCallback callback,
                     java.lang.Object ctx)
        Open a ManagedCursor asynchronously.
        Parameters:
        name - the name associated with the ManagedCursor
        callback - callback object
        ctx - opaque context
        See Also:
        openCursor(String)

      • asyncOpenCursor

        void asyncOpenCursor​(java.lang.String name,
                     org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition,
                     AsyncCallbacks.OpenCursorCallback callback,
                     java.lang.Object ctx)
        Open a ManagedCursor asynchronously.
        Parameters:
        name - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at lastest position or not when first created default is true
        callback - callback object
        ctx - opaque context
        See Also:
        openCursor(String)

      • asyncOpenCursor

        void asyncOpenCursor​(java.lang.String name,
                     org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition,
                     java.util.Map<java.lang.String,​java.lang.Long> properties,
                     AsyncCallbacks.OpenCursorCallback callback,
                     java.lang.Object ctx)
        Open a ManagedCursor asynchronously.
        Parameters:
        name - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at lastest position or not when first created default is true
        callback - callback object
        ctx - opaque context
        See Also:
        openCursor(String)

      • getCursors

        java.lang.Iterable<ManagedCursor> getCursors()
        Get a list of all the cursors reading from this ManagedLedger
        Returns:
        a list of cursors

      • getActiveCursors

        java.lang.Iterable<ManagedCursor> getActiveCursors()
        Get a list of all the active cursors reading from this ManagedLedger.
        Returns:
        a list of cursors

      • getNumberOfEntries

        long getNumberOfEntries()
        Get the total number of entries for this managed ledger.

        This is defined by the number of entries in all the BookKeeper ledgers that are being maintained by this ManagedLedger.

        This method is non-blocking.

        Returns:
        the number of entries

      • getNumberOfActiveEntries

        long getNumberOfActiveEntries()
        Get the total number of active entries for this managed ledger.

        This is defined by the number of non consumed entries in all the BookKeeper ledgers that are being maintained by this ManagedLedger.

        This method is non-blocking.

        Returns:
        the number of entries

      • getTotalSize

        long getTotalSize()
        Get the total sizes in bytes of the managed ledger, without accounting for replicas.

        This is defined by the sizes of all the BookKeeper ledgers that are being maintained by this ManagedLedger.

        This method is non-blocking.

        Returns:
        total size in bytes

      • getEstimatedBacklogSize

        long getEstimatedBacklogSize()
        Get estimated total unconsumed or backlog size in bytes for the managed ledger, without accounting for replicas.
        Returns:
        estimated total backlog size

      • getOffloadedSize

        long getOffloadedSize()
        Return the size of all ledgers offloaded to 2nd tier storage

      • terminate

        Position terminate()
            throws java.lang.InterruptedException,
                   ManagedLedgerException
        Terminate the managed ledger and return the last committed entry.

        Once the managed ledger is terminated, it will not accept any more write

        Returns:
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • close

        void close()
    throws java.lang.InterruptedException,
           ManagedLedgerException
        Close the ManagedLedger.

        This will close all the underlying BookKeeper ledgers. All the ManagedCursors associated will be invalidated.

        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • asyncClose

        void asyncClose​(AsyncCallbacks.CloseCallback callback,
                java.lang.Object ctx)
        Close the ManagedLedger asynchronously.
        Parameters:
        callback - callback object
        ctx - opaque context
        See Also:
        close()

      • asyncOffloadPrefix

        void asyncOffloadPrefix​(Position pos,
                        AsyncCallbacks.OffloadCallback callback,
                        java.lang.Object ctx)
        Offload as many entries before position as possible to longterm storage. As internally, entries is stored in ledgers, and ledgers can only be operated on as a whole, it is likely not possible to offload every entry before the passed in position. Only complete ledgers will be offloaded. On completion a position will be passed to the callback. This position is the earliest entry which was not offloaded.
        Parameters:
        pos - the position before which entries will be offloaded
        callback - a callback which will be supplied with the earliest unoffloaded position on completion
        ctx - a context object which will be passed to the callback on completion

      • getSlowestConsumer

        ManagedCursor getSlowestConsumer()
        Get the slowest consumer.
        Returns:
        the slowest consumer

      • isTerminated

        boolean isTerminated()
        Returns whether the managed ledger was terminated.

      • setConfig

        void setConfig​(ManagedLedgerConfig config)
        Updates managed-ledger config.
        Parameters:
        config -

      • getLastConfirmedEntry

        Position getLastConfirmedEntry()
        Gets last confirmed entry of the managed ledger.
        Returns:
        the last confirmed entry id

      • readyToCreateNewLedger

        void readyToCreateNewLedger()
        Signaling managed ledger that we can resume after BK write failure

      • getProperties

        java.util.Map<java.lang.String,​java.lang.String> getProperties()
        Returns managed-ledger's properties.
        Returns:
        key-values of properties

      • setProperty

        void setProperty​(java.lang.String key,
                 java.lang.String value)
          throws java.lang.InterruptedException,
                 ManagedLedgerException
        Add key-value to propertiesMap.
        Parameters:
        key - key of property to add
        value - value of property to add
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • asyncSetProperty

        void asyncSetProperty​(java.lang.String key,
                      java.lang.String value,
                      AsyncCallbacks.UpdatePropertiesCallback callback,
                      java.lang.Object ctx)
        Async add key-value to propertiesMap.
        Parameters:
        key - key of property to add
        value - value of property to add
        callback - a callback which will be supplied with the newest properties in managedLedger.
        ctx - a context object which will be passed to the callback on completion.

      • deleteProperty

        void deleteProperty​(java.lang.String key)
             throws java.lang.InterruptedException,
                    ManagedLedgerException
        Delete the property by key.
        Parameters:
        key - key of property to delete
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • asyncDeleteProperty

        void asyncDeleteProperty​(java.lang.String key,
                         AsyncCallbacks.UpdatePropertiesCallback callback,
                         java.lang.Object ctx)
        Async delete the property by key.
        Parameters:
        key - key of property to delete
        callback - a callback which will be supplied with the newest properties in managedLedger.
        ctx - a context object which will be passed to the callback on completion.

      • setProperties

        void setProperties​(java.util.Map<java.lang.String,​java.lang.String> properties)
            throws java.lang.InterruptedException,
                   ManagedLedgerException
        Update managed-ledger's properties.
        Parameters:
        properties - key-values of properties
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • asyncSetProperties

        void asyncSetProperties​(java.util.Map<java.lang.String,​java.lang.String> properties,
                        AsyncCallbacks.UpdatePropertiesCallback callback,
                        java.lang.Object ctx)
        Async update managed-ledger's properties.
        Parameters:
        properties - key-values of properties.
        callback - a callback which will be supplied with the newest properties in managedLedger.
        ctx - a context object which will be passed to the callback on completion.

      • trimConsumedLedgersInBackground

        void trimConsumedLedgersInBackground​(java.util.concurrent.CompletableFuture<?> promise)
        Trim consumed ledgers in background
        Parameters:
        promise -

      • rollCurrentLedgerIfFull

        @Deprecated
void rollCurrentLedgerIfFull()
        Deprecated.
        Roll current ledger if it is full

      • asyncFindPosition

        java.util.concurrent.CompletableFuture<Position> asyncFindPosition​(com.google.common.base.Predicate<Entry> predicate)
        Find position by sequenceId.

      • getManagedLedgerInterceptor

        ManagedLedgerInterceptor getManagedLedgerInterceptor()
        Get the ManagedLedgerInterceptor for ManagedLedger.

      • asyncTruncate

        java.util.concurrent.CompletableFuture<java.lang.Void> asyncTruncate()
        Truncate ledgers The truncate operation will move all cursors to the end of the topic and delete all inactive ledgers.

      • getManagedLedgerInternalStats

        java.util.concurrent.CompletableFuture<org.apache.pulsar.common.policies.data.ManagedLedgerInternalStats> getManagedLedgerInternalStats​(boolean includeLedgerMetadata)
        Get managed ledger internal stats
        Parameters:
        includeLedgerMetadata - the flag to control managed ledger internal stats include ledger metadata
        Returns:
        the future of managed ledger internal stats