Package org.projectnessie.versioned.persist.adapter

Interface DatabaseAdapter

  • All Known Implementing Classes:
    AbstractDatabaseAdapter, TracingDatabaseAdapter
    public interface DatabaseAdapter
    Database-Adapter interface that encapsulates all database related logic, an abstraction between a VersionStore implementation and a variety of different databases that share common core implementations for example for the commit/merge/transplant operations.

    One or more adapter instances may use the same storage (database instance / schema). In this case adapter instances usually differ by their repository ID configuration parameters.

    Database-adapters treat the actual "Nessie content" and "Nessie commit metadata" as an opaque value ("BLOB") without interpreting the content. Database-adapter must persist serialized values for commit-metadata and content as is and must return those in the exact same representation on read.

    Actual implementation usually extend either org.projectnessie.versioned.persist.nontx.NonTxDatabaseAdapter (NoSQL databases) or org.projectnessie.versioned.persist.tx.TxDatabaseAdapter (JDBC/transactional). Both in turn extend AbstractDatabaseAdapter.

    All returned Streams must be closed.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void assertCleanStateForTests()  
      void assign​(org.projectnessie.versioned.NamedRef assignee, java.util.Optional<org.projectnessie.versioned.Hash> expectedHead, org.projectnessie.versioned.Hash assignTo)
      Updates assignee's HEAD to assignTo.
      org.projectnessie.versioned.Hash commit​(CommitParams commitParams)
      Commit operation, see CommitParams for a description of the parameters.
      java.util.stream.Stream<CommitLogEntry> commitLog​(org.projectnessie.versioned.Hash offset)
      Retrieve the commit-log starting at the commit referenced by offset.
      boolean consistentPutAttachment​(org.projectnessie.versioned.ContentAttachment attachment, java.util.Optional<java.lang.String> expectedVersion)
      Consistent put-attachment operation.
      org.projectnessie.versioned.Hash create​(org.projectnessie.versioned.NamedRef ref, org.projectnessie.versioned.Hash target)
      Create a new named reference.
      org.projectnessie.versioned.Hash delete​(org.projectnessie.versioned.NamedRef reference, java.util.Optional<org.projectnessie.versioned.Hash> expectedHead)
      Delete the given reference.
      void deleteAttachments​(java.util.stream.Stream<org.projectnessie.versioned.ContentAttachmentKey> keys)
      Unconditionally delete the content attachments identified via keys.
      java.util.stream.Stream<Difference> diff​(org.projectnessie.versioned.Hash from, org.projectnessie.versioned.Hash to, KeyFilterPredicate keyFilter)
      Compute the difference of the content for the two commits identified by from and to.
      void eraseRepo()
      Forces all repository data managed by this adapter instance to be deleted.
      java.util.stream.Stream<CommitLogEntry> fetchCommitLogEntries​(java.util.stream.Stream<org.projectnessie.versioned.Hash> hashes)
      Loads commit log entries.
      RepoDescription fetchRepositoryDescription()
      Fetches the current version and descriptive attributes of the repository.
      java.util.stream.Stream<org.projectnessie.versioned.ContentAttachmentKey> getAttachmentKeys​(java.lang.String contentId)
      Retrieve the known attachment keys for a content ID.
      DatabaseAdapterConfig getConfig()  
      java.util.Optional<ContentIdAndBytes> globalContent​(ContentId contentId)
      Retrieves the global content for the given contents-id.
      org.projectnessie.versioned.Hash hashOnReference​(org.projectnessie.versioned.NamedRef namedReference, java.util.Optional<org.projectnessie.versioned.Hash> hashOnReference)
      Verifies that the given namedReference exists and that hashOnReference, if present, is reachable via that reference.
      void initializeRepo​(java.lang.String defaultBranchName)
      Ensures that mandatory data is present in the repository, does not change an existing repo.
      java.util.stream.Stream<KeyListEntry> keys​(org.projectnessie.versioned.Hash commit, KeyFilterPredicate keyFilter)
      Retrieve the content-keys that are "present" for the specified commit.
      java.util.stream.Stream<org.projectnessie.versioned.ContentAttachment> mapToAttachment​(java.util.stream.Stream<org.projectnessie.versioned.ContentAttachmentKey> keys)
      Retrieve the content attachments identified via keys.
      org.projectnessie.versioned.MergeResult<CommitLogEntry> merge​(MergeParams mergeParams)
      Merge all commits on from since the common ancestor of from and to and commit those onto to.
      org.projectnessie.versioned.ReferenceInfo<com.google.protobuf.ByteString> namedRef​(java.lang.String ref, org.projectnessie.versioned.GetNamedRefsParams params)
      Resolve the current HEAD of the given named-reference and optionally additional information.
      java.util.stream.Stream<org.projectnessie.versioned.ReferenceInfo<com.google.protobuf.ByteString>> namedRefs​(org.projectnessie.versioned.GetNamedRefsParams params)
      Get all named references including their current HEAD.
      org.projectnessie.versioned.Hash noAncestorHash()
      Get the Hash for "beginning of time".
      void putAttachments​(java.util.stream.Stream<org.projectnessie.versioned.ContentAttachment> attachments)
      Bulk-write the given content attachments.
      CommitLogEntry rebuildKeyList​(CommitLogEntry entry, java.util.function.Function<org.projectnessie.versioned.Hash,​CommitLogEntry> inMemoryCommits)
      Populates the aggregated key-list for the given entry and returns it.
      java.util.stream.Stream<RefLog> refLog​(org.projectnessie.versioned.Hash offset)
      Retrieve the refLog starting at the refLog referenced by offset.
      java.util.Map<java.lang.String,​java.util.Map<java.lang.String,​java.lang.String>> repoMaintenance​(RepoMaintenanceParams repoMaintenanceParams)  
      java.util.stream.Stream<CommitLogEntry> scanAllCommitLogEntries()
      Scan all commit log entries, no guarantees about order nor about the behavior when commits happen while the returned Stream is consumed.
      org.projectnessie.versioned.MergeResult<CommitLogEntry> transplant​(TransplantParams transplantParams)
      Cherry-pick the commits with the hashes sequenceToTransplant from named reference source onto the reference targetBranch.
      void updateMultipleCommits​(java.util.List<CommitLogEntry> commitLogEntries)
      Updates multiple commit-entries, the given commit entries are to be persisted as is.
      void updateRepositoryDescription​(java.util.function.Function<RepoDescription,​RepoDescription> updater)
      Updates the repository description.
      java.util.Map<org.projectnessie.versioned.Key,​ContentAndState> values​(org.projectnessie.versioned.Hash commit, java.util.Collection<org.projectnessie.versioned.Key> keys, KeyFilterPredicate keyFilter)
      Retrieve the reference-local and global state for the given keys for the specified commit.
      void writeMultipleCommits​(java.util.List<CommitLogEntry> commitLogEntries)
      Write multiple new commit-entries, the given commit entries are to be persisted as is.

    • Method Detail

      • initializeRepo

        void initializeRepo​(java.lang.String defaultBranchName)
        Ensures that mandatory data is present in the repository, does not change an existing repo.

      • eraseRepo

        void eraseRepo()
        Forces all repository data managed by this adapter instance to be deleted.

        This includes all data for the configured repository ID.

        After erasing a repository initializeRepo(String) may be called to reinitialize the minimal required data structures for the same repository ID.

      • noAncestorHash

        org.projectnessie.versioned.Hash noAncestorHash()
        Get the Hash for "beginning of time".

      • hashOnReference

        org.projectnessie.versioned.Hash hashOnReference​(org.projectnessie.versioned.NamedRef namedReference,
                                                 java.util.Optional<org.projectnessie.versioned.Hash> hashOnReference)
                                          throws org.projectnessie.versioned.ReferenceNotFoundException
        Verifies that the given namedReference exists and that hashOnReference, if present, is reachable via that reference.
        Returns:
        verified hashOnReference or, if hashOnReference is not present, the current HEAD of namedReference
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if namedReference does not exist or hashOnReference, if present, is not reachable from that reference

      • values

        java.util.Map<org.projectnessie.versioned.Key,​ContentAndState> values​(org.projectnessie.versioned.Hash commit,
                                                                            java.util.Collection<org.projectnessie.versioned.Key> keys,
                                                                            KeyFilterPredicate keyFilter)
                                                                     throws org.projectnessie.versioned.ReferenceNotFoundException
        Retrieve the reference-local and global state for the given keys for the specified commit.
        Parameters:
        commit - commit to retrieve the values for.
        keys - keys to retrieve the values (reference-local and global) for
        keyFilter - predicate to optionally skip specific keys in the result and return those as "not present", for example to implement a security policy.
        Returns:
        Ordered stream
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if commit does not exist.

      • commitLog

        @MustBeClosed
java.util.stream.Stream<CommitLogEntry> commitLog​(org.projectnessie.versioned.Hash offset)
                                           throws org.projectnessie.versioned.ReferenceNotFoundException
        Retrieve the commit-log starting at the commit referenced by offset.
        Parameters:
        offset - hash to start at
        Returns:
        stream of CommitLogEntrys
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if offset does not exist.

      • fetchCommitLogEntries

        @MustBeClosed
java.util.stream.Stream<CommitLogEntry> fetchCommitLogEntries​(java.util.stream.Stream<org.projectnessie.versioned.Hash> hashes)
        Loads commit log entries.
        Returns:
        the loaded CommitLogEntrys, non-existing entries will not be returned.

      • keys

        @MustBeClosed
java.util.stream.Stream<KeyListEntry> keys​(org.projectnessie.versioned.Hash commit,
                                           KeyFilterPredicate keyFilter)
                                    throws org.projectnessie.versioned.ReferenceNotFoundException
        Retrieve the content-keys that are "present" for the specified commit.
        Parameters:
        commit - commit to retrieve the values for.
        keyFilter - predicate to optionally skip specific keys in the result and return those as "not present", for example to implement a security policy.
        Returns:
        Ordered stream with content-keys, content-ids and content-types
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if commit does not exist.

      • commit

        org.projectnessie.versioned.Hash commit​(CommitParams commitParams)
                                 throws org.projectnessie.versioned.ReferenceConflictException,
                                        org.projectnessie.versioned.ReferenceNotFoundException
        Commit operation, see CommitParams for a description of the parameters.
        Parameters:
        commitParams - parameters for the commit
        Returns:
        optimistically written commit-log-entry
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if either the named reference in ToBranchParams.getToBranch() or the commit on that reference, if specified, does not exist.
        org.projectnessie.versioned.ReferenceConflictException - if any of the commits could not be committed onto the target branch due to a conflicting change or if the expected hash in ToBranchParams.getToBranch()is not its expected hEAD

      • transplant

        org.projectnessie.versioned.MergeResult<CommitLogEntry> transplant​(TransplantParams transplantParams)
                                                            throws org.projectnessie.versioned.ReferenceNotFoundException,
                                                                   org.projectnessie.versioned.ReferenceConflictException
        Cherry-pick the commits with the hashes sequenceToTransplant from named reference source onto the reference targetBranch.
        Returns:
        the hash of the last cherry-picked commit, in other words the new HEAD of the target branch
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if either the named reference in commitOnReference or the commit on that reference, if specified, does not exist.
        org.projectnessie.versioned.ReferenceConflictException - if any of the commits could not be committed onto the target branch due to a conflicting change or if the expected hash of toBranch is not its expected hEAD

      • merge

        org.projectnessie.versioned.MergeResult<CommitLogEntry> merge​(MergeParams mergeParams)
                                                       throws org.projectnessie.versioned.ReferenceNotFoundException,
                                                              org.projectnessie.versioned.ReferenceConflictException
        Merge all commits on from since the common ancestor of from and to and commit those onto to.

        The implementation first identifies the common-ancestor (the most-recent commit that is both reachable via from and to).

        Returns:
        the hash of the last cherry-picked commit, in other words the new HEAD of the target branch
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if either the named reference in toBranch or the commit on that reference, if specified, does not exist.
        org.projectnessie.versioned.ReferenceConflictException - if any of the commits could not be committed onto the target branch due to a conflicting change or if the expected hash of toBranch is not its expected hEAD

      • namedRef

        org.projectnessie.versioned.ReferenceInfo<com.google.protobuf.ByteString> namedRef​(java.lang.String ref,
                                                                                   org.projectnessie.versioned.GetNamedRefsParams params)
                                                                            throws org.projectnessie.versioned.ReferenceNotFoundException
        Resolve the current HEAD of the given named-reference and optionally additional information.

        This is actually a convenience for hashOnReference(ref, Optional.empty().

        Parameters:
        ref - named reference to resolve
        params - options that control which information shall be returned in ReferenceInfo, see GetNamedRefsParams for details.
        Returns:
        current HEAD of ref
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if the named reference ref does not exist.

      • namedRefs

        @MustBeClosed
java.util.stream.Stream<org.projectnessie.versioned.ReferenceInfo<com.google.protobuf.ByteString>> namedRefs​(org.projectnessie.versioned.GetNamedRefsParams params)
                                                                                                      throws org.projectnessie.versioned.ReferenceNotFoundException
        Get all named references including their current HEAD.
        Parameters:
        params - options that control which information shall be returned in each ReferenceInfo, see ReferenceInfo for details.
        Returns:
        stream with all named references.
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException

      • create

        org.projectnessie.versioned.Hash create​(org.projectnessie.versioned.NamedRef ref,
                                        org.projectnessie.versioned.Hash target)
                                 throws org.projectnessie.versioned.ReferenceAlreadyExistsException,
                                        org.projectnessie.versioned.ReferenceNotFoundException
        Create a new named reference.
        Parameters:
        ref - Named reference to create - either a BranchName or TagName.
        target - The already existing named reference with an optional hash on that branch. This parameter can be null for the edge case when the default branch is re-created after it has been dropped.
        Returns:
        the current HEAD of the created branch or tag
        Throws:
        org.projectnessie.versioned.ReferenceAlreadyExistsException - if the reference ref already exists.
        org.projectnessie.versioned.ReferenceNotFoundException - if target does not exist.

      • delete

        org.projectnessie.versioned.Hash delete​(org.projectnessie.versioned.NamedRef reference,
                                        java.util.Optional<org.projectnessie.versioned.Hash> expectedHead)
                                 throws org.projectnessie.versioned.ReferenceNotFoundException,
                                        org.projectnessie.versioned.ReferenceConflictException
        Delete the given reference.
        Parameters:
        reference - named-reference to delete. If a value for the hash is specified, it must be equal to the current HEAD.
        expectedHead - if present, reference's current HEAD must be equal to this value
        Returns:
        head of deleted reference
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if the named reference in reference does not exist.
        org.projectnessie.versioned.ReferenceConflictException - if the named reference's HEAD is not equal to the expected HEAD

      • assign

        void assign​(org.projectnessie.versioned.NamedRef assignee,
            java.util.Optional<org.projectnessie.versioned.Hash> expectedHead,
            org.projectnessie.versioned.Hash assignTo)
     throws org.projectnessie.versioned.ReferenceNotFoundException,
            org.projectnessie.versioned.ReferenceConflictException
        Updates assignee's HEAD to assignTo.
        Parameters:
        assignee - named reference to re-assign
        expectedHead - if present, assignee's current HEAD must be equal to this value
        assignTo - commit to update assignee's HEAD to
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if either the named reference in assignTo or the commit on that reference, if specified, does not exist or if the named reference specified in assignee does not exist.
        org.projectnessie.versioned.ReferenceConflictException - if the HEAD of the named reference assignee is not equal to the expected HEAD

      • diff

        @MustBeClosed
java.util.stream.Stream<Difference> diff​(org.projectnessie.versioned.Hash from,
                                         org.projectnessie.versioned.Hash to,
                                         KeyFilterPredicate keyFilter)
                                  throws org.projectnessie.versioned.ReferenceNotFoundException
        Compute the difference of the content for the two commits identified by from and to.
        Parameters:
        from - "From" side of the diff
        to - "To" side of the diff
        keyFilter - predicate to optionally skip specific keys in the diff result and not return those, for example to implement a security policy.
        Returns:
        stream containing the difference of the content, excluding both equal values and values that were excluded via keyFilter
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException - if from or to does not exist.

      • fetchRepositoryDescription

        RepoDescription fetchRepositoryDescription()
        Fetches the current version and descriptive attributes of the repository.

      • updateRepositoryDescription

        void updateRepositoryDescription​(java.util.function.Function<RepoDescription,​RepoDescription> updater)
                          throws org.projectnessie.versioned.ReferenceConflictException
        Updates the repository description. Takes a function that receives the current repository description and returns the updated description.
        Parameters:
        updater - updater function, the input argument is never null, if updater return null, the update will be aborted
        Throws:
        org.projectnessie.versioned.ReferenceConflictException - thrown if the repository description could not be updated due to other concurrent updates

      • globalContent

        java.util.Optional<ContentIdAndBytes> globalContent​(ContentId contentId)
        Retrieves the global content for the given contents-id.
        Parameters:
        contentId - contents-id to retrieve the global content for
        Returns:
        global content, if present or an empty optional, never null.

      • repoMaintenance

        java.util.Map<java.lang.String,​java.util.Map<java.lang.String,​java.lang.String>> repoMaintenance​(RepoMaintenanceParams repoMaintenanceParams)

      • refLog

        @MustBeClosed
java.util.stream.Stream<RefLog> refLog​(org.projectnessie.versioned.Hash offset)
                                throws org.projectnessie.versioned.RefLogNotFoundException
        Retrieve the refLog starting at the refLog referenced by offset.
        Parameters:
        offset - initial reflog id to read from
        Returns:
        stream of RefLogs
        Throws:
        org.projectnessie.versioned.RefLogNotFoundException

      • scanAllCommitLogEntries

        @MustBeClosed
java.util.stream.Stream<CommitLogEntry> scanAllCommitLogEntries()
        Scan all commit log entries, no guarantees about order nor about the behavior when commits happen while the returned Stream is consumed.

      • getAttachmentKeys

        @MustBeClosed
java.util.stream.Stream<org.projectnessie.versioned.ContentAttachmentKey> getAttachmentKeys​(java.lang.String contentId)
        Retrieve the known attachment keys for a content ID.

        Implementations may return keys that do not or no longer exist. Especially non-transactional database adapter implementations allow this as a compromise. See mapToAttachment(Stream), which only returns existing content attachments.

      • mapToAttachment

        @MustBeClosed
java.util.stream.Stream<org.projectnessie.versioned.ContentAttachment> mapToAttachment​(java.util.stream.Stream<org.projectnessie.versioned.ContentAttachmentKey> keys)
        Retrieve the content attachments identified via keys. Attachments that do not exist are not returned.

        Whether the input stream is "terminated" or "fluently" mapped, can vary between implementations.

      • consistentPutAttachment

        boolean consistentPutAttachment​(org.projectnessie.versioned.ContentAttachment attachment,
                                java.util.Optional<java.lang.String> expectedVersion)
        Consistent put-attachment operation.

        Either a "put-if-absent", if expectedVersion is empty or a compare-and-swap based on the value of expectedVersion.

        Note: this method uses conditional put operations, unlike putAttachments(Stream). Users should not use this method and putAttachments(Stream) for the same keys.

        Parameters:
        attachment - the attachment to write
        expectedVersion - indicator for put-if-absent or the expected value on an existing item

      • putAttachments

        void putAttachments​(java.util.stream.Stream<org.projectnessie.versioned.ContentAttachment> attachments)
        Bulk-write the given content attachments. The values of attachments written with this method should be immutable and deterministic for the respective attachment keys.

        The behavior of the implementation whether an already existing attachment will be overwritten or not is undefined.

        The outcome of this method is undefined when an error occurred.

        Note: this method uses unconditional put operations, unlike consistentPutAttachment(ContentAttachment, Optional). Should not use this method and consistentPutAttachment(ContentAttachment, Optional) for the same keys.

      • deleteAttachments

        void deleteAttachments​(java.util.stream.Stream<org.projectnessie.versioned.ContentAttachmentKey> keys)
        Unconditionally delete the content attachments identified via keys.

      • assertCleanStateForTests

        void assertCleanStateForTests()

      • writeMultipleCommits

        void writeMultipleCommits​(java.util.List<CommitLogEntry> commitLogEntries)
                   throws org.projectnessie.versioned.ReferenceConflictException
        Write multiple new commit-entries, the given commit entries are to be persisted as is. All values of the given CommitLogEntry can be considered valid and consistent.

        Callers must call updateMultipleCommits(List) for already existing CommitLogEntrys and writeMultipleCommits(List) for new CommitLogEntrys. Implementations can rely on this assumption (think: SQL INSERT + UPDATE compared to a "simple put" for NoSQL databases).

        Implementations however can enforce strict consistency checks/guarantees, like a best-effort approach to prevent hash-collisions but without any other consistency checks/guarantees.

        Throws:
        org.projectnessie.versioned.ReferenceConflictException

      • updateMultipleCommits

        void updateMultipleCommits​(java.util.List<CommitLogEntry> commitLogEntries)
                    throws org.projectnessie.versioned.ReferenceNotFoundException
        Updates multiple commit-entries, the given commit entries are to be persisted as is. All values of the given CommitLogEntry can be considered valid and consistent.

        Callers must call updateMultipleCommits(List) for already existing CommitLogEntrys and writeMultipleCommits(List) for new CommitLogEntrys. Implementations can rely on this assumption (think: SQL INSERT + UPDATE compared to a "simple put" for NoSQL databases).

        Implementations however can enforce strict consistency checks/guarantees.

        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException

      • rebuildKeyList

        CommitLogEntry rebuildKeyList​(CommitLogEntry entry,
                              @Nonnull @Nonnull
                              java.util.function.Function<org.projectnessie.versioned.Hash,​CommitLogEntry> inMemoryCommits)
                       throws org.projectnessie.versioned.ReferenceNotFoundException
        Populates the aggregated key-list for the given entry and returns it.
        Parameters:
        entry - the CommitLogEntry to build the aggregated key list for
        inMemoryCommits - function to retrieve not-yet-written commit-log-entries
        Returns:
        commit-log-entry with the aggregated key-list. The returned CommitLogEntry has not been persisted.
        Throws:
        org.projectnessie.versioned.ReferenceNotFoundException