org.apache.nifi.controller.repository

Interface FlowFileRepository

All Superinterfaces:

AutoCloseable, Closeable

public interface FlowFileRepository
extends Closeable

Implementations must be thread safe

Method Summary

Modifier and Type	Method and Description
default Set<ResourceClaim>	findOrphanedResourceClaims() Returns the set of Resource Claims that are referenced by FlowFiles that have been "orphaned" because they belong to FlowFile Queues/Connections that did not exist in the flow when NiFi started
Set<String>	findQueuesWithFlowFiles(FlowFileSwapManager flowFileSwapManager) Searches through the repository to find the ID's of all FlowFile Queues that currently have data queued
default Map<ResourceClaim,Set<ResourceClaimReference>>	findResourceClaimReferences(Set<ResourceClaim> resourceClaims, FlowFileSwapManager swapManager) Scans the FlowFile Repository to locate any FlowFiles that reference the given Resource Claims.
String	getFileStoreName() Returns the name of the FileStore that the repository is stored on, or null if not applicable or unable to determine the file store name
long	getMaxFlowFileIdentifier()
long	getNextFlowFileSequence()
long	getStorageCapacity()
long	getUsableStorageSpace()
void	initialize(ResourceClaimManager claimManager) Initializes the Content Repository, providing to it the ContentClaimManager that is to be used for interacting with Content Claims
boolean	isValidSwapLocationSuffix(String swapLocationSuffix) Determines whether or not the given swap location suffix is a valid, known location according to this FlowFileRepository.
boolean	isVolatile()
long	loadFlowFiles(QueueProvider queueProvider) Loads all flow files found within the repository, establishes the content claims and their reference count
void	swapFlowFilesIn(String swapLocation, List<FlowFileRecord> flowFileRecords, FlowFileQueue flowFileQueue) Updates the Repository to indicate that the given FlowFileRecords were Swapped In to memory
void	swapFlowFilesOut(List<FlowFileRecord> swappedOut, FlowFileQueue flowFileQueue, String swapLocation) Updates the Repository to indicate that the given FlowFileRecords were Swapped Out of memory
void	updateMaxFlowFileIdentifier(long maxId) Notifies the FlowFile Repository that the given identifier has been identified as the maximum value that has been encountered for an 'external' (swapped out) FlowFile.
void	updateRepository(Collection<RepositoryRecord> records) Updates the repository with the given RepositoryRecords.

Methods inherited from interface java.io.Closeable

close

Method Detail

initialize

void initialize(ResourceClaimManager claimManager)
         throws IOException

Initializes the Content Repository, providing to it the ContentClaimManager that is to be used for interacting with Content Claims

Parameters:

claimManager - for handling claims

Throws:

IOException - if unable to initialize repository

getStorageCapacity

long getStorageCapacity()
                 throws IOException

Returns:

the maximum number of bytes that can be stored in the underlying storage mechanism

Throws:

IOException - if computing capacity fails

getUsableStorageSpace

long getUsableStorageSpace()
                    throws IOException

Returns:

the number of bytes currently available for use by the underlying storage mechanism

Throws:

IOException - if computing usable space fails

getFileStoreName

String getFileStoreName()

Returns the name of the FileStore that the repository is stored on, or null if not applicable or unable to determine the file store name

Returns:

the name of the FileStore

updateRepository

void updateRepository(Collection<RepositoryRecord> records)
               throws IOException

Updates the repository with the given RepositoryRecords.

Parameters:

records - the records to update the repository with

Throws:

IOException - if update fails

loadFlowFiles

long loadFlowFiles(QueueProvider queueProvider)
            throws IOException

Loads all flow files found within the repository, establishes the content claims and their reference count

Parameters:

queueProvider - the provider of FlowFile Queues into which the FlowFiles should be enqueued

Returns:

index of highest flow file identifier

Throws:

IOException - if load fails

findQueuesWithFlowFiles

Set<String> findQueuesWithFlowFiles(FlowFileSwapManager flowFileSwapManager)
                             throws IOException

Searches through the repository to find the ID's of all FlowFile Queues that currently have data queued

Returns:

the set of all FlowFileQueue identifiers for which a FlowFile is queued

Throws:

IOException - if unable to read from the FlowFile Repository

isVolatile

boolean isVolatile()

Returns:

true if the Repository is volatile (i.e., its data is lost upon application restart), false otherwise

getNextFlowFileSequence

long getNextFlowFileSequence()

Returns:

the next ID in sequence for creating FlowFiles.

getMaxFlowFileIdentifier

long getMaxFlowFileIdentifier()
                       throws IOException

Returns:

the max ID of all FlowFiles that currently exist in the repository.

Throws:

IOException - if computing max identifier fails

updateMaxFlowFileIdentifier

void updateMaxFlowFileIdentifier(long maxId)

Notifies the FlowFile Repository that the given identifier has been identified as the maximum value that has been encountered for an 'external' (swapped out) FlowFile.

Parameters:

maxId - the max id of any FlowFile encountered

swapFlowFilesOut

void swapFlowFilesOut(List<FlowFileRecord> swappedOut,
                      FlowFileQueue flowFileQueue,
                      String swapLocation)
               throws IOException

Updates the Repository to indicate that the given FlowFileRecords were Swapped Out of memory

Parameters:

swappedOut - the FlowFiles that were swapped out of memory

flowFileQueue - the queue that the FlowFiles belong to

swapLocation - the location to which the FlowFiles were swapped

Throws:

IOException - if swap fails

swapFlowFilesIn

void swapFlowFilesIn(String swapLocation,
                     List<FlowFileRecord> flowFileRecords,
                     FlowFileQueue flowFileQueue)
              throws IOException

Updates the Repository to indicate that the given FlowFileRecords were Swapped In to memory

Parameters:

swapLocation - the location (e.g., a filename) from which FlowFiles were recovered

flowFileRecords - the records that were swapped in

flowFileQueue - the queue that the FlowFiles belong to

Throws:

IOException - if swap fails

isValidSwapLocationSuffix

boolean isValidSwapLocationSuffix(String swapLocationSuffix)

Determines whether or not the given swap location suffix is a valid, known location according to this FlowFileRepository. Note that while the swapFlowFilesIn(String, List, FlowFileQueue) and swapFlowFilesOut(List, FlowFileQueue, String) methods expect a full "swap location" this method expects only the "suffix" of a swap location. For example, if the location points to a file, this method would expect only the filename, not the full path.

This method differs from the others because the other methods want to store the swap location or recover from a given location. However, this method is used to verify that the location is known. If for any reason, NiFi is stopped, its FlowFile Repository relocated to a new location (for example, a different disk partition), and restarted, the swap location would not match if we used the full location. Therefore, by using only the "suffix" (i.e. the filename for a file-based implementation), we can avoid worrying about relocation.

Parameters:

swapLocationSuffix - the suffix of the location to check

Returns:

true if the swap location is known and valid, false otherwise

findResourceClaimReferences

default Map<ResourceClaim,Set<ResourceClaimReference>> findResourceClaimReferences(Set<ResourceClaim> resourceClaims,
                                                                                   FlowFileSwapManager swapManager)
                                                                            throws IOException

Scans the FlowFile Repository to locate any FlowFiles that reference the given Resource Claims. If the FlowFile Repository does not implement this capability, it will return null.

Parameters:

resourceClaims - the resource claims whose references should be found

swapManager - the swap manager to use for scanning swap files

Returns:

a Mapping of Resource Claim to a representation of the FlowFiles/Swap Files that reference those Resource Claims

Throws:

IOException - if an IO failure occurs when attempting to find references

findOrphanedResourceClaims

default Set<ResourceClaim> findOrphanedResourceClaims()

Returns the set of Resource Claims that are referenced by FlowFiles that have been "orphaned" because they belong to FlowFile Queues/Connections that did not exist in the flow when NiFi started

Returns:

the set of orphaned Resource Claims