Package org.apache.bookkeeper.mledger.impl

Class ManagedLedgerImpl

  • All Implemented Interfaces:
    org.apache.bookkeeper.client.AsyncCallback.CreateCallback, ManagedLedger
    Direct Known Subclasses:
    ReadOnlyManagedLedgerImpl
    public class ManagedLedgerImpl
extends java.lang.Object
implements ManagedLedger, org.apache.bookkeeper.client.AsyncCallback.CreateCallback

    • Field Detail

      • AsyncOperationTimeoutSeconds

        protected static final int AsyncOperationTimeoutSeconds
        See Also:
        Constant Field Values

      • bookKeeper

        protected final org.apache.bookkeeper.client.BookKeeper bookKeeper

      • name

        protected final java.lang.String name

      • propertiesMap

        protected java.util.Map<java.lang.String,​java.lang.String> propertiesMap

      • mlOwnershipChecker

        protected final java.util.function.Supplier<java.lang.Boolean> mlOwnershipChecker

      • DEFAULT_LEDGER_DELETE_RETRIES

        protected static final int DEFAULT_LEDGER_DELETE_RETRIES
        See Also:
        Constant Field Values

      • DEFAULT_LEDGER_DELETE_BACKOFF_TIME_SEC

        protected static final int DEFAULT_LEDGER_DELETE_BACKOFF_TIME_SEC
        See Also:
        Constant Field Values

      • clock

        protected final java.time.Clock clock

      • ADD_OP_COUNT_UPDATER

        protected static final java.util.concurrent.atomic.AtomicLongFieldUpdater<ManagedLedgerImpl> ADD_OP_COUNT_UPDATER

    • Constructor Detail

      • ManagedLedgerImpl

        public ManagedLedgerImpl​(ManagedLedgerFactoryImpl factory,
                         org.apache.bookkeeper.client.BookKeeper bookKeeper,
                         MetaStore store,
                         ManagedLedgerConfig config,
                         org.apache.bookkeeper.common.util.OrderedScheduler scheduledExecutor,
                         java.lang.String name)

      • ManagedLedgerImpl

        public ManagedLedgerImpl​(ManagedLedgerFactoryImpl factory,
                         org.apache.bookkeeper.client.BookKeeper bookKeeper,
                         MetaStore store,
                         ManagedLedgerConfig config,
                         org.apache.bookkeeper.common.util.OrderedScheduler scheduledExecutor,
                         java.lang.String name,
                         java.util.function.Supplier<java.lang.Boolean> mlOwnershipChecker)

    • Method Detail

      • getName

        public java.lang.String getName()
        Specified by:
        getName in interface ManagedLedger
        Returns:
        the unique name of this ManagedLedger

      • addEntry

        public Position addEntry​(byte[] data)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Description copied from interface: ManagedLedger
        Append a new entry to the end of a managed ledger.
        Specified by:
        addEntry in interface ManagedLedger
        Parameters:
        data - data entry to be persisted
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • addEntry

        public Position addEntry​(byte[] data,
                         int numberOfMessages)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Description copied from interface: ManagedLedger
        Append a new entry to the end of a managed ledger.
        Specified by:
        addEntry in interface ManagedLedger
        Parameters:
        data - data entry to be persisted
        numberOfMessages - numberOfMessages of entry
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • addEntry

        public Position addEntry​(byte[] data,
                         int offset,
                         int length)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Description copied from interface: ManagedLedger
        Append a new entry to the end of a managed ledger.
        Specified by:
        addEntry in interface ManagedLedger
        Parameters:
        data - data entry to be persisted
        offset - offset in the data array
        length - number of bytes
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • addEntry

        public Position addEntry​(byte[] data,
                         int numberOfMessages,
                         int offset,
                         int length)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Description copied from interface: ManagedLedger
        Append a new entry to the end of a managed ledger.
        Specified by:
        addEntry in interface ManagedLedger
        Parameters:
        data - data entry to be persisted
        numberOfMessages - numberOfMessages of entry
        offset - offset in the data array
        length - number of bytes
        Returns:
        the Position at which the entry has been inserted
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • asyncAddEntry

        public void asyncAddEntry​(byte[] data,
                          int numberOfMessages,
                          int offset,
                          int length,
                          AsyncCallbacks.AddEntryCallback callback,
                          java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Append a new entry asynchronously.
        Specified by:
        asyncAddEntry in interface ManagedLedger
        Parameters:
        data - data entry to be persisted
        numberOfMessages - numberOfMessages of entry
        offset - offset in the data array
        length - number of bytes
        callback - callback object
        ctx - opaque context
        See Also:
        ManagedLedger.addEntry(byte[])

      • readyToCreateNewLedger

        public void readyToCreateNewLedger()
        Description copied from interface: ManagedLedger
        Signaling managed ledger that we can resume after BK write failure
        Specified by:
        readyToCreateNewLedger in interface ManagedLedger

      • openCursor

        public ManagedCursor openCursor​(java.lang.String cursorName)
                         throws java.lang.InterruptedException,
                                ManagedLedgerException
        Description copied from interface: ManagedLedger
        Open a ManagedCursor in this ManagedLedger.

        If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

        Specified by:
        openCursor in interface ManagedLedger
        Parameters:
        cursorName - the name associated with the ManagedCursor
        Returns:
        the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • openCursor

        public ManagedCursor openCursor​(java.lang.String cursorName,
                                org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition)
                         throws java.lang.InterruptedException,
                                ManagedLedgerException
        Description copied from interface: ManagedLedger
        Open a ManagedCursor in this ManagedLedger.

        If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

        Specified by:
        openCursor in interface ManagedLedger
        Parameters:
        cursorName - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at latest position or not when first created default is true
        Returns:
        the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • openCursor

        public ManagedCursor openCursor​(java.lang.String cursorName,
                                org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition,
                                java.util.Map<java.lang.String,​java.lang.Long> properties)
                         throws java.lang.InterruptedException,
                                ManagedLedgerException
        Description copied from interface: ManagedLedger
        Open a ManagedCursor in this ManagedLedger.

        If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

        Specified by:
        openCursor in interface ManagedLedger
        Parameters:
        cursorName - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at latest position or not when first created default is true
        properties - user defined properties that will be attached to the first position of the cursor, if the open operation will trigger the creation of the cursor.
        Returns:
        the ManagedCursor
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • asyncOpenCursor

        public void asyncOpenCursor​(java.lang.String cursorName,
                            org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition,
                            AsyncCallbacks.OpenCursorCallback callback,
                            java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Open a ManagedCursor asynchronously.
        Specified by:
        asyncOpenCursor in interface ManagedLedger
        Parameters:
        cursorName - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at lastest position or not when first created default is true
        callback - callback object
        ctx - opaque context
        See Also:
        ManagedLedger.openCursor(String)

      • asyncOpenCursor

        public void asyncOpenCursor​(java.lang.String cursorName,
                            org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition initialPosition,
                            java.util.Map<java.lang.String,​java.lang.Long> properties,
                            AsyncCallbacks.OpenCursorCallback callback,
                            java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Open a ManagedCursor asynchronously.
        Specified by:
        asyncOpenCursor in interface ManagedLedger
        Parameters:
        cursorName - the name associated with the ManagedCursor
        initialPosition - the cursor will be set at lastest position or not when first created default is true
        callback - callback object
        ctx - opaque context
        See Also:
        ManagedLedger.openCursor(String)

      • deleteCursor

        public void deleteCursor​(java.lang.String name)
                  throws java.lang.InterruptedException,
                         ManagedLedgerException
        Description copied from interface: ManagedLedger
        Remove a ManagedCursor from this ManagedLedger.

        If the cursor doesn't exist, the operation will still succeed.

        Specified by:
        deleteCursor in interface ManagedLedger
        Parameters:
        name - the name associated with the ManagedCursor
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • newNonDurableCursor

        public ManagedCursor newNonDurableCursor​(Position startCursorPosition)
                                  throws ManagedLedgerException
        Description copied from interface: ManagedLedger
        Creates a new cursor whose metadata is not backed by durable storage. A caller can treat the non-durable cursor exactly like a normal cursor, with the only difference in that after restart it will not remember which entries were deleted. Also it does not prevent data from being deleted.

        The cursor is anonymous and can be positioned on an arbitrary position.

        This method is not-blocking.

        Specified by:
        newNonDurableCursor in interface ManagedLedger
        Parameters:
        startCursorPosition - the position where the cursor should be initialized, or null to start from the current latest entry. When starting on a particular cursor position, the first entry to be returned will be the entry next to the specified position
        Returns:
        the new NonDurableCursor
        Throws:
        ManagedLedgerException

      • hasActiveCursors

        public boolean hasActiveCursors()
        Tells whether the managed ledger has any active-cursor registered.
        Returns:
        true if at least a cursor exists

      • getNumberOfEntries

        public long getNumberOfEntries()
        Description copied from interface: ManagedLedger
        Get the total number of entries for this managed ledger.

        This is defined by the number of entries in all the BookKeeper ledgers that are being maintained by this ManagedLedger.

        This method is non-blocking.

        Specified by:
        getNumberOfEntries in interface ManagedLedger
        Returns:
        the number of entries

      • getNumberOfActiveEntries

        public long getNumberOfActiveEntries()
        Description copied from interface: ManagedLedger
        Get the total number of active entries for this managed ledger.

        This is defined by the number of non consumed entries in all the BookKeeper ledgers that are being maintained by this ManagedLedger.

        This method is non-blocking.

        Specified by:
        getNumberOfActiveEntries in interface ManagedLedger
        Returns:
        the number of entries

      • getTotalSize

        public long getTotalSize()
        Description copied from interface: ManagedLedger
        Get the total sizes in bytes of the managed ledger, without accounting for replicas.

        This is defined by the sizes of all the BookKeeper ledgers that are being maintained by this ManagedLedger.

        This method is non-blocking.

        Specified by:
        getTotalSize in interface ManagedLedger
        Returns:
        total size in bytes

      • getEstimatedBacklogSize

        public long getEstimatedBacklogSize()
        Description copied from interface: ManagedLedger
        Get estimated total unconsumed or backlog size in bytes for the managed ledger, without accounting for replicas.
        Specified by:
        getEstimatedBacklogSize in interface ManagedLedger
        Returns:
        estimated total backlog size

      • getEstimatedBacklogSize

        public long getEstimatedBacklogSize​(PositionImpl pos)
        Get estimated backlog size from a specific position.

      • isTerminated

        public boolean isTerminated()
        Description copied from interface: ManagedLedger
        Returns whether the managed ledger was terminated.
        Specified by:
        isTerminated in interface ManagedLedger

      • close

        public void close()
           throws java.lang.InterruptedException,
                  ManagedLedgerException
        Description copied from interface: ManagedLedger
        Close the ManagedLedger.

        This will close all the underlying BookKeeper ledgers. All the ManagedCursors associated will be invalidated.

        Specified by:
        close in interface ManagedLedger
        Throws:
        ManagedLedgerException
        java.lang.InterruptedException

      • createComplete

        public void createComplete​(int rc,
                           org.apache.bookkeeper.client.LedgerHandle lh,
                           java.lang.Object ctx)
        Specified by:
        createComplete in interface org.apache.bookkeeper.client.AsyncCallback.CreateCallback

      • updateLedgersIdsComplete

        public void updateLedgersIdsComplete​(org.apache.pulsar.metadata.api.Stat stat)

      • asyncFindPosition

        public java.util.concurrent.CompletableFuture<Position> asyncFindPosition​(com.google.common.base.Predicate<Entry> predicate)
        Description copied from interface: ManagedLedger
        Find position by sequenceId.
        Specified by:
        asyncFindPosition in interface ManagedLedger

      • getLedgerMetadata

        public java.util.concurrent.CompletableFuture<java.lang.String> getLedgerMetadata​(long ledgerId)

      • asyncReadEntry

        protected void asyncReadEntry​(org.apache.bookkeeper.client.api.ReadHandle ledger,
                              long firstEntry,
                              long lastEntry,
                              boolean isSlowestReader,
                              org.apache.bookkeeper.mledger.impl.OpReadEntry opReadEntry,
                              java.lang.Object ctx)

      • hasMoreEntries

        public boolean hasMoreEntries​(PositionImpl position)

      • maybeUpdateCursorBeforeTrimmingConsumedLedger

        public void maybeUpdateCursorBeforeTrimmingConsumedLedger()

      • trimConsumedLedgersInBackground

        public void trimConsumedLedgersInBackground​(java.util.concurrent.CompletableFuture<?> promise)
        Description copied from interface: ManagedLedger
        Trim consumed ledgers in background
        Specified by:
        trimConsumedLedgersInBackground in interface ManagedLedger

      • trimConsumedLedgersInBackground

        public void trimConsumedLedgersInBackground​(boolean isTruncate,
                                            java.util.concurrent.CompletableFuture<?> promise)

      • asyncOffloadPrefix

        public void asyncOffloadPrefix​(Position pos,
                               AsyncCallbacks.OffloadCallback callback,
                               java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Offload as many entries before position as possible to longterm storage. As internally, entries is stored in ledgers, and ledgers can only be operated on as a whole, it is likely not possible to offload every entry before the passed in position. Only complete ledgers will be offloaded. On completion a position will be passed to the callback. This position is the earliest entry which was not offloaded.
        Specified by:
        asyncOffloadPrefix in interface ManagedLedger
        Parameters:
        pos - the position before which entries will be offloaded
        callback - a callback which will be supplied with the earliest unoffloaded position on completion
        ctx - a context object which will be passed to the callback on completion

      • getPositionAfterN

        public PositionImpl getPositionAfterN​(PositionImpl startPosition,
                                      long n,
                                      ManagedLedgerImpl.PositionBound startRange)
        Get the entry position at a given distance from a given position.
        Parameters:
        startPosition - starting position
        n - number of entries to skip ahead
        startRange - specifies whether or not to include the start position in calculating the distance
        Returns:
        the new position that is n entries ahead

      • getPreviousPosition

        public PositionImpl getPreviousPosition​(PositionImpl position)
        Get the entry position that come before the specified position in the message stream, using information from the ledger list and each ledger entries count.
        Parameters:
        position - the current position
        Returns:
        the previous position

      • isValidPosition

        public boolean isValidPosition​(PositionImpl position)
        Validate whether a specified position is valid for the current managed ledger.
        Parameters:
        position - the position to validate
        Returns:
        true if the position is valid, false otherwise

      • ledgerExists

        public boolean ledgerExists​(long ledgerId)

      • getNextValidLedger

        public java.lang.Long getNextValidLedger​(long ledgerId)

      • activateCursor

        public void activateCursor​(ManagedCursor cursor)

      • deactivateCursor

        public void deactivateCursor​(ManagedCursor cursor)

      • isCursorActive

        public boolean isCursorActive​(ManagedCursor cursor)

      • getEntriesAddedCounter

        public long getEntriesAddedCounter()

      • getCurrentLedgerEntries

        public long getCurrentLedgerEntries()

      • getCurrentLedgerSize

        public long getCurrentLedgerSize()

      • getLastLedgerCreatedTimestamp

        public long getLastLedgerCreatedTimestamp()

      • getLastLedgerCreationFailureTimestamp

        public long getLastLedgerCreationFailureTimestamp()

      • getWaitingCursorsCount

        public int getWaitingCursorsCount()

      • getPendingAddEntriesCount

        public int getPendingAddEntriesCount()

      • getCacheSize

        public long getCacheSize()

      • isReadOnly

        protected boolean isReadOnly()

      • createManagedLedgerException

        public static ManagedLedgerException createManagedLedgerException​(int bkErrorCode)

      • createManagedLedgerException

        public static ManagedLedgerException createManagedLedgerException​(java.lang.Throwable t)

      • asyncCreateLedger

        protected void asyncCreateLedger​(org.apache.bookkeeper.client.BookKeeper bookKeeper,
                                 ManagedLedgerConfig config,
                                 org.apache.bookkeeper.client.BookKeeper.DigestType digestType,
                                 org.apache.bookkeeper.client.AsyncCallback.CreateCallback cb,
                                 java.util.Map<java.lang.String,​byte[]> metadata)
        Create ledger async and schedule a timeout task to check ledger-creation is complete else it fails the callback with TimeoutException.
        Parameters:
        bookKeeper -
        config -
        digestType -
        cb -
        metadata -

      • getClock

        public java.time.Clock getClock()

      • checkAndCompleteLedgerOpTask

        protected boolean checkAndCompleteLedgerOpTask​(int rc,
                                               org.apache.bookkeeper.client.LedgerHandle lh,
                                               java.lang.Object ctx)
        check if ledger-op task is already completed by timeout-task. If completed then delete the created ledger
        Parameters:
        rc -
        lh -
        ctx -
        Returns:

      • getOffloadedSize

        public long getOffloadedSize()
        Description copied from interface: ManagedLedger
        Return the size of all ledgers offloaded to 2nd tier storage
        Specified by:
        getOffloadedSize in interface ManagedLedger

      • getProperties

        public java.util.Map<java.lang.String,​java.lang.String> getProperties()
        Description copied from interface: ManagedLedger
        Returns managed-ledger's properties.
        Specified by:
        getProperties in interface ManagedLedger
        Returns:
        key-values of properties

      • setProperty

        public void setProperty​(java.lang.String key,
                        java.lang.String value)
                 throws java.lang.InterruptedException,
                        ManagedLedgerException
        Description copied from interface: ManagedLedger
        Add key-value to propertiesMap.
        Specified by:
        setProperty in interface ManagedLedger
        Parameters:
        key - key of property to add
        value - value of property to add
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • asyncSetProperty

        public void asyncSetProperty​(java.lang.String key,
                             java.lang.String value,
                             AsyncCallbacks.UpdatePropertiesCallback callback,
                             java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Async add key-value to propertiesMap.
        Specified by:
        asyncSetProperty in interface ManagedLedger
        Parameters:
        key - key of property to add
        value - value of property to add
        callback - a callback which will be supplied with the newest properties in managedLedger.
        ctx - a context object which will be passed to the callback on completion.

      • asyncDeleteProperty

        public void asyncDeleteProperty​(java.lang.String key,
                                AsyncCallbacks.UpdatePropertiesCallback callback,
                                java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Async delete the property by key.
        Specified by:
        asyncDeleteProperty in interface ManagedLedger
        Parameters:
        key - key of property to delete
        callback - a callback which will be supplied with the newest properties in managedLedger.
        ctx - a context object which will be passed to the callback on completion.

      • setProperties

        public void setProperties​(java.util.Map<java.lang.String,​java.lang.String> properties)
                   throws java.lang.InterruptedException,
                          ManagedLedgerException
        Description copied from interface: ManagedLedger
        Update managed-ledger's properties.
        Specified by:
        setProperties in interface ManagedLedger
        Parameters:
        properties - key-values of properties
        Throws:
        java.lang.InterruptedException
        ManagedLedgerException

      • asyncSetProperties

        public void asyncSetProperties​(java.util.Map<java.lang.String,​java.lang.String> properties,
                               AsyncCallbacks.UpdatePropertiesCallback callback,
                               java.lang.Object ctx)
        Description copied from interface: ManagedLedger
        Async update managed-ledger's properties.
        Specified by:
        asyncSetProperties in interface ManagedLedger
        Parameters:
        properties - key-values of properties.
        callback - a callback which will be supplied with the newest properties in managedLedger.
        ctx - a context object which will be passed to the callback on completion.

      • setEntriesAddedCounter

        public void setEntriesAddedCounter​(long count)

      • asyncTruncate

        public java.util.concurrent.CompletableFuture<java.lang.Void> asyncTruncate()
        Description copied from interface: ManagedLedger
        Truncate ledgers The truncate operation will move all cursors to the end of the topic and delete all inactive ledgers.
        Specified by:
        asyncTruncate in interface ManagedLedger

      • getManagedLedgerInternalStats

        public java.util.concurrent.CompletableFuture<org.apache.pulsar.common.policies.data.ManagedLedgerInternalStats> getManagedLedgerInternalStats​(boolean includeLedgerMetadata)
        Description copied from interface: ManagedLedger
        Get managed ledger internal stats
        Specified by:
        getManagedLedgerInternalStats in interface ManagedLedger
        Parameters:
        includeLedgerMetadata - the flag to control managed ledger internal stats include ledger metadata
        Returns:
        the future of managed ledger internal stats

      • getEnsemblesAsync

        public java.util.concurrent.CompletableFuture<java.util.Set<org.apache.bookkeeper.net.BookieId>> getEnsemblesAsync​(long ledgerId)