Package org.apache.bookkeeper.mledger

Interface ManagedLedger

All Known Implementing Classes:
ManagedLedgerImpl, ReadOnlyManagedLedgerImpl, ShadowManagedLedgerImpl
@LimitedPrivate @Stable public interface ManagedLedger
A ManagedLedger it's a superset of a BookKeeper ledger concept.

It mimics the concept of an appender log that:

  • has a unique name (chosen by clients) by which it can be created/opened/deleted
  • is always writable: if a writer process crashes, a new writer can re-open the ManagedLedger and continue writing into it
  • has multiple persisted consumers (see ManagedCursor), each of them with an associated position
  • when all the consumers have processed all the entries contained in a Bookkeeper ledger, the ledger is deleted

Caveats:

  • A single ManagedLedger can only be open once at any time. Implementation can protect double access from the same VM, but accesses from different machines to the same ManagedLedger need to be avoided through an external source of coordination.

  • Method Details

    • getName

      String getName()
      Returns:
      the unique name of this ManagedLedger

    • addEntry

      Position addEntry(byte[] data) throws InterruptedException, ManagedLedgerException
      Append a new entry to the end of a managed ledger.
      Parameters:
      data - data entry to be persisted
      Returns:
      the Position at which the entry has been inserted
      Throws:
      ManagedLedgerException
      InterruptedException

    • addEntry

      Position addEntry(byte[] data, int numberOfMessages) throws InterruptedException, ManagedLedgerException
      Append a new entry to the end of a managed ledger.
      Parameters:
      data - data entry to be persisted
      numberOfMessages - numberOfMessages of entry
      Returns:
      the Position at which the entry has been inserted
      Throws:
      ManagedLedgerException
      InterruptedException

    • asyncAddEntry

      void asyncAddEntry(byte[] data, AsyncCallbacks.AddEntryCallback callback, Object ctx)
      Append a new entry asynchronously.
      Parameters:
      data - data entry to be persisted
      callback - callback object
      ctx - opaque context
      See Also:

    • addEntry

      Position addEntry(byte[] data, int offset, int length) throws InterruptedException, ManagedLedgerException
      Append a new entry to the end of a managed ledger.
      Parameters:
      data - data entry to be persisted
      offset - offset in the data array
      length - number of bytes
      Returns:
      the Position at which the entry has been inserted
      Throws:
      ManagedLedgerException
      InterruptedException

    • addEntry

      Position addEntry(byte[] data, int numberOfMessages, int offset, int length) throws InterruptedException, ManagedLedgerException
      Append a new entry to the end of a managed ledger.
      Parameters:
      data - data entry to be persisted
      numberOfMessages - numberOfMessages of entry
      offset - offset in the data array
      length - number of bytes
      Returns:
      the Position at which the entry has been inserted
      Throws:
      ManagedLedgerException
      InterruptedException

    • asyncAddEntry

      void asyncAddEntry(byte[] data, int offset, int length, AsyncCallbacks.AddEntryCallback callback, Object ctx)
      Append a new entry asynchronously.
      Parameters:
      data - data entry to be persisted
      offset - offset in the data array
      length - number of bytes
      callback - callback object
      ctx - opaque context
      See Also:

    • asyncAddEntry

      void asyncAddEntry(byte[] data, int numberOfMessages, int offset, int length, AsyncCallbacks.AddEntryCallback callback, Object ctx)
      Append a new entry asynchronously.
      Parameters:
      data - data entry to be persisted
      numberOfMessages - numberOfMessages of entry
      offset - offset in the data array
      length - number of bytes
      callback - callback object
      ctx - opaque context
      See Also:

    • asyncAddEntry

      void asyncAddEntry(io.netty.buffer.ByteBuf buffer, AsyncCallbacks.AddEntryCallback callback, Object ctx)
      Append a new entry asynchronously.
      Parameters:
      buffer - buffer with the data entry
      callback - callback object
      ctx - opaque context
      See Also:

    • asyncAddEntry

      void asyncAddEntry(io.netty.buffer.ByteBuf buffer, int numberOfMessages, AsyncCallbacks.AddEntryCallback callback, Object ctx)
      Append a new entry asynchronously.
      Parameters:
      buffer - buffer with the data entry
      numberOfMessages - numberOfMessages for data entry
      callback - callback object
      ctx - opaque context
      See Also:

    • openCursor

      ManagedCursor openCursor(String name) throws InterruptedException, ManagedLedgerException
      Open a ManagedCursor in this ManagedLedger.

      If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

      Parameters:
      name - the name associated with the ManagedCursor
      Returns:
      the ManagedCursor
      Throws:
      ManagedLedgerException
      InterruptedException

    • openCursor

      ManagedCursor openCursor(String name, CommandSubscribe.InitialPosition initialPosition) throws InterruptedException, ManagedLedgerException
      Open a ManagedCursor in this ManagedLedger.

      If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

      Parameters:
      name - the name associated with the ManagedCursor
      initialPosition - if null, the cursor will be set at latest position when first created
      Returns:
      the ManagedCursor
      Throws:
      ManagedLedgerException
      InterruptedException

    • openCursor

      ManagedCursor openCursor(String name, CommandSubscribe.InitialPosition initialPosition, Map<String,Long> properties, Map<String,String> cursorProperties) throws InterruptedException, ManagedLedgerException
      Open a ManagedCursor in this ManagedLedger.

      If the cursors doesn't exist, a new one will be created and its position will be at the end of the ManagedLedger.

      Parameters:
      name - the name associated with the ManagedCursor
      initialPosition - if null, the cursor will be set at latest position when first created
      properties - user defined properties that will be attached to the first position of the cursor, if the open operation will trigger the creation of the cursor.
      cursorProperties - the properties for the Cursor
      Returns:
      the ManagedCursor
      Throws:
      ManagedLedgerException
      InterruptedException

    • newNonDurableCursor

      ManagedCursor newNonDurableCursor(Position startCursorPosition) throws ManagedLedgerException
      Creates a new cursor whose metadata is not backed by durable storage. A caller can treat the non-durable cursor exactly like a normal cursor, with the only difference in that after restart it will not remember which entries were deleted. Also it does not prevent data from being deleted.

      The cursor is anonymous and can be positioned on an arbitrary position.

      This method is not-blocking.

      Parameters:
      startCursorPosition - the position where the cursor should be initialized, or null to start from the current latest entry. When starting on a particular cursor position, the first entry to be returned will be the entry next to the specified position
      Returns:
      the new NonDurableCursor
      Throws:
      ManagedLedgerException

    • newNonDurableCursor

      ManagedCursor newNonDurableCursor(Position startPosition, String subscriptionName) throws ManagedLedgerException
      Throws:
      ManagedLedgerException

    • newNonDurableCursor

      ManagedCursor newNonDurableCursor(Position startPosition, String subscriptionName, CommandSubscribe.InitialPosition initialPosition, boolean isReadCompacted) throws ManagedLedgerException
      Throws:
      ManagedLedgerException

    • asyncDeleteCursor

      void asyncDeleteCursor(String name, AsyncCallbacks.DeleteCursorCallback callback, Object ctx)
      Delete a ManagedCursor asynchronously.
      Parameters:
      name - the name associated with the ManagedCursor
      callback - callback object
      ctx - opaque context
      See Also:

    • deleteCursor

      void deleteCursor(String name) throws InterruptedException, ManagedLedgerException
      Remove a ManagedCursor from this ManagedLedger.

      If the cursor doesn't exist, the operation will still succeed.

      Parameters:
      name - the name associated with the ManagedCursor
      Throws:
      ManagedLedgerException
      InterruptedException

    • removeWaitingCursor

      void removeWaitingCursor(ManagedCursor cursor)
      Remove a ManagedCursor from this ManagedLedger's waitingCursors.
      Parameters:
      cursor - the ManagedCursor

    • asyncOpenCursor

      void asyncOpenCursor(String name, AsyncCallbacks.OpenCursorCallback callback, Object ctx)
      Open a ManagedCursor asynchronously.
      Parameters:
      name - the name associated with the ManagedCursor
      callback - callback object
      ctx - opaque context
      See Also:

    • asyncOpenCursor

      void asyncOpenCursor(String name, CommandSubscribe.InitialPosition initialPosition, AsyncCallbacks.OpenCursorCallback callback, Object ctx)
      Open a ManagedCursor asynchronously.
      Parameters:
      name - the name associated with the ManagedCursor
      initialPosition - if null, the cursor will be set at latest position when first created
      callback - callback object
      ctx - opaque context
      See Also:

    • asyncOpenCursor

      void asyncOpenCursor(String name, CommandSubscribe.InitialPosition initialPosition, Map<String,Long> properties, Map<String,String> cursorProperties, AsyncCallbacks.OpenCursorCallback callback, Object ctx)
      Open a ManagedCursor asynchronously.
      Parameters:
      name - the name associated with the ManagedCursor
      initialPosition - if null, the cursor will be set at latest position when first created
      cursorProperties - the properties for the Cursor
      callback - callback object
      ctx - opaque context
      See Also:

    • getCursors

      Iterable<ManagedCursor> getCursors()
      Get a list of all the cursors reading from this ManagedLedger.
      Returns:
      a list of cursors

    • getActiveCursors

      Iterable<ManagedCursor> getActiveCursors()
      Get a list of all the active cursors reading from this ManagedLedger.
      Returns:
      a list of cursors

    • getNumberOfEntries

      long getNumberOfEntries()
      Get the total number of entries for this managed ledger.

      This is defined by the number of entries in all the BookKeeper ledgers that are being maintained by this ManagedLedger.

      This method is non-blocking.

      Returns:
      the number of entries

    • getNumberOfActiveEntries

      long getNumberOfActiveEntries()
      Get the total number of active entries for this managed ledger.

      This is defined by the number of non consumed entries in all the BookKeeper ledgers that are being maintained by this ManagedLedger.

      This method is non-blocking.

      Returns:
      the number of entries

    • getTotalSize

      long getTotalSize()
      Get the total sizes in bytes of the managed ledger, without accounting for replicas.

      This is defined by the sizes of all the BookKeeper ledgers that are being maintained by this ManagedLedger.

      This method is non-blocking.

      Returns:
      total size in bytes

    • getEstimatedBacklogSize

      long getEstimatedBacklogSize()
      Get estimated total unconsumed or backlog size in bytes for the managed ledger, without accounting for replicas.
      Returns:
      estimated total backlog size

    • getEarliestMessagePublishTimeInBacklog

      CompletableFuture<Long> getEarliestMessagePublishTimeInBacklog()
      Get the publishing time of the oldest message in the backlog.
      Returns:
      the publishing time of the oldest message

    • getOffloadedSize

      long getOffloadedSize()
      Return the size of all ledgers offloaded to 2nd tier storage.

    • getLastOffloadedLedgerId

      long getLastOffloadedLedgerId()
      Get last offloaded ledgerId. If no offloaded yet, it returns 0.
      Returns:
      last offloaded ledgerId

    • getLastOffloadedSuccessTimestamp

      long getLastOffloadedSuccessTimestamp()
      Get last suceessful offloaded timestamp. If no successful offload, it returns 0.
      Returns:
      last successful offloaded timestamp

    • getLastOffloadedFailureTimestamp

      long getLastOffloadedFailureTimestamp()
      Get last failed offloaded timestamp. If no failed offload, it returns 0.
      Returns:
      last failed offloaded timestamp

    • asyncTerminate

      void asyncTerminate(AsyncCallbacks.TerminateCallback callback, Object ctx)

    • asyncMigrate

      CompletableFuture<Position> asyncMigrate()

    • terminate

      Position terminate() throws InterruptedException, ManagedLedgerException
      Terminate the managed ledger and return the last committed entry.

      Once the managed ledger is terminated, it will not accept any more write

      Returns:
      Throws:
      InterruptedException
      ManagedLedgerException

    • close

      void close() throws InterruptedException, ManagedLedgerException
      Close the ManagedLedger.

      This will close all the underlying BookKeeper ledgers. All the ManagedCursors associated will be invalidated.

      Throws:
      ManagedLedgerException
      InterruptedException

    • asyncClose

      void asyncClose(AsyncCallbacks.CloseCallback callback, Object ctx)
      Close the ManagedLedger asynchronously.
      Parameters:
      callback - callback object
      ctx - opaque context
      See Also:

    • getStats

      ManagedLedgerMXBean getStats()
      Returns:
      the managed ledger stats MBean

    • delete

      void delete() throws InterruptedException, ManagedLedgerException
      Delete the ManagedLedger.
      Throws:
      InterruptedException
      ManagedLedgerException

    • asyncDelete

      void asyncDelete(AsyncCallbacks.DeleteLedgerCallback callback, Object ctx)
      Async delete a ledger.
      Parameters:
      callback -
      ctx -
      Throws:
      InterruptedException
      ManagedLedgerException

    • offloadPrefix

      Position offloadPrefix(Position pos) throws InterruptedException, ManagedLedgerException
      Offload as many entries before position as possible to longterm storage.
      Parameters:
      pos - the position before which entries will be offloaded
      Returns:
      the earliest position which was not offloaded
      Throws:
      InterruptedException
      ManagedLedgerException
      See Also:

    • asyncOffloadPrefix

      void asyncOffloadPrefix(Position pos, AsyncCallbacks.OffloadCallback callback, Object ctx)
      Offload as many entries before position as possible to longterm storage. As internally, entries is stored in ledgers, and ledgers can only be operated on as a whole, it is likely not possible to offload every entry before the passed in position. Only complete ledgers will be offloaded. On completion a position will be passed to the callback. This position is the earliest entry which was not offloaded.
      Parameters:
      pos - the position before which entries will be offloaded
      callback - a callback which will be supplied with the earliest unoffloaded position on completion
      ctx - a context object which will be passed to the callback on completion

    • getSlowestConsumer

      ManagedCursor getSlowestConsumer()
      Get the slowest consumer.
      Returns:
      the slowest consumer

    • isTerminated

      boolean isTerminated()
      Returns whether the managed ledger was terminated.

    • isMigrated

      boolean isMigrated()
      Returns whether the managed ledger was migrated.

    • getConfig

      ManagedLedgerConfig getConfig()
      Returns managed-ledger config.

    • setConfig

      void setConfig(ManagedLedgerConfig config)
      Updates managed-ledger config.
      Parameters:
      config -

    • getLastConfirmedEntry

      Position getLastConfirmedEntry()
      Gets last confirmed entry of the managed ledger.
      Returns:
      the last confirmed entry id

    • readyToCreateNewLedger

      void readyToCreateNewLedger()
      Signaling managed ledger that we can resume after BK write failure.

    • getProperties

      Map<String,String> getProperties()
      Returns managed-ledger's properties.
      Returns:
      key-values of properties

    • setProperty

      void setProperty(String key, String value) throws InterruptedException, ManagedLedgerException
      Add key-value to propertiesMap.
      Parameters:
      key - key of property to add
      value - value of property to add
      Throws:
      InterruptedException
      ManagedLedgerException

    • asyncSetProperty

      void asyncSetProperty(String key, String value, AsyncCallbacks.UpdatePropertiesCallback callback, Object ctx)
      Async add key-value to propertiesMap.
      Parameters:
      key - key of property to add
      value - value of property to add
      callback - a callback which will be supplied with the newest properties in managedLedger.
      ctx - a context object which will be passed to the callback on completion.

    • deleteProperty

      void deleteProperty(String key) throws InterruptedException, ManagedLedgerException
      Delete the property by key.
      Parameters:
      key - key of property to delete
      Throws:
      InterruptedException
      ManagedLedgerException

    • asyncDeleteProperty

      void asyncDeleteProperty(String key, AsyncCallbacks.UpdatePropertiesCallback callback, Object ctx)
      Async delete the property by key.
      Parameters:
      key - key of property to delete
      callback - a callback which will be supplied with the newest properties in managedLedger.
      ctx - a context object which will be passed to the callback on completion.

    • setProperties

      void setProperties(Map<String,String> properties) throws InterruptedException, ManagedLedgerException
      Update managed-ledger's properties.
      Parameters:
      properties - key-values of properties
      Throws:
      InterruptedException
      ManagedLedgerException

    • asyncSetProperties

      void asyncSetProperties(Map<String,String> properties, AsyncCallbacks.UpdatePropertiesCallback callback, Object ctx)
      Async update managed-ledger's properties.
      Parameters:
      properties - key-values of properties.
      callback - a callback which will be supplied with the newest properties in managedLedger.
      ctx - a context object which will be passed to the callback on completion.

    • trimConsumedLedgersInBackground

      void trimConsumedLedgersInBackground(CompletableFuture<?> promise)
      Trim consumed ledgers in background.
      Parameters:
      promise -

    • skipNonRecoverableLedger

      default void skipNonRecoverableLedger(long ledgerId)
      If a ledger is lost, this ledger will be skipped after enabled "autoSkipNonRecoverableData", and the method is used to delete information about this ledger in the ManagedCursor.

    • rollCurrentLedgerIfFull

      @Deprecated void rollCurrentLedgerIfFull()
      Deprecated.
      Roll current ledger if it is full.

    • asyncFindPosition

      CompletableFuture<Position> asyncFindPosition(Predicate<Entry> predicate)
      Find position by sequenceId.

    • getManagedLedgerInterceptor

      ManagedLedgerInterceptor getManagedLedgerInterceptor()
      Get the ManagedLedgerInterceptor for ManagedLedger.

    • getLedgerInfo

      CompletableFuture<MLDataFormats.ManagedLedgerInfo.LedgerInfo> getLedgerInfo(long ledgerId)
      Get basic ledger summary. will got null if corresponding ledger not exists.

    • getOptionalLedgerInfo

      Optional<MLDataFormats.ManagedLedgerInfo.LedgerInfo> getOptionalLedgerInfo(long ledgerId)
      Get basic ledger summary. will get Optional.empty() if corresponding ledger not exists.

    • asyncTruncate

      CompletableFuture<Void> asyncTruncate()
      Truncate ledgers The truncate operation will move all cursors to the end of the topic and delete all inactive ledgers.

    • getManagedLedgerInternalStats

      CompletableFuture<ManagedLedgerInternalStats> getManagedLedgerInternalStats(boolean includeLedgerMetadata)
      Get managed ledger internal stats.
      Parameters:
      includeLedgerMetadata - the flag to control managed ledger internal stats include ledger metadata
      Returns:
      the future of managed ledger internal stats

    • checkInactiveLedgerAndRollOver

      boolean checkInactiveLedgerAndRollOver()
      Check current inactive ledger (based on ManagedLedgerConfig.getInactiveLedgerRollOverTimeMs() and roll over that ledger if inactive.
      Returns:
      true if ledger is considered for rolling over

    • checkCursorsToCacheEntries

      void checkCursorsToCacheEntries()
      Check if managed ledger should cache backlog reads.