org.apache.hudi.common.fs

Class FailSafeConsistencyGuard

java.lang.Object

org.apache.hudi.common.fs.FailSafeConsistencyGuard

All Implemented Interfaces:

ConsistencyGuard

Direct Known Subclasses:

OptimisticConsistencyGuard

public class FailSafeConsistencyGuard
extends Object
implements ConsistencyGuard

A consistency checker that fails if it is unable to meet the required condition within a specified timeout.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hudi.common.fs.ConsistencyGuard

ConsistencyGuard.FileVisibility

Field Summary

Fields

Modifier and Type	Field and Description
protected ConsistencyGuardConfig	consistencyGuardConfig
protected HoodieStorage	storage

Constructor Summary

Constructors

Constructor and Description
FailSafeConsistencyGuard(HoodieStorage storage, ConsistencyGuardConfig consistencyGuardConfig)

Method Summary

Modifier and Type	Method and Description
protected boolean	checkFilesVisibility(int retryNum, StoragePath dir, List<String> files, ConsistencyGuard.FileVisibility event) Helper to check for file visibility based on ConsistencyGuard.FileVisibility event.
protected boolean	checkFileVisibility(StoragePath filePath, ConsistencyGuard.FileVisibility visibility) Helper to check of file visibility.
protected List<String>	getFilesWithoutSchemeAndAuthority(List<String> files) Generate file names without scheme and authority.
void	waitForFilesVisibility(String dirPath, List<String> files, ConsistencyGuard.FileVisibility event) Helper function to wait for all files belonging to single directory to appear.
void	waitTillAllFilesAppear(String dirPath, List<String> files)
void	waitTillAllFilesDisappear(String dirPath, List<String> files)
void	waitTillFileAppears(StoragePath filePath)
void	waitTillFileDisappears(StoragePath filePath)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.fs.ConsistencyGuard

waitTill

Field Detail

storage

protected final HoodieStorage storage

consistencyGuardConfig

protected final ConsistencyGuardConfig consistencyGuardConfig

Constructor Detail

FailSafeConsistencyGuard

public FailSafeConsistencyGuard(HoodieStorage storage,
                                ConsistencyGuardConfig consistencyGuardConfig)

Method Detail

waitTillFileAppears

public void waitTillFileAppears(StoragePath filePath)
                         throws TimeoutException

Specified by:

waitTillFileAppears in interface ConsistencyGuard

Throws:

TimeoutException

waitTillFileDisappears

public void waitTillFileDisappears(StoragePath filePath)
                            throws TimeoutException

Specified by:

waitTillFileDisappears in interface ConsistencyGuard

Throws:

TimeoutException

waitTillAllFilesAppear

public void waitTillAllFilesAppear(String dirPath,
                                   List<String> files)
                            throws TimeoutException

Specified by:

waitTillAllFilesAppear in interface ConsistencyGuard

Throws:

TimeoutException

waitTillAllFilesDisappear

public void waitTillAllFilesDisappear(String dirPath,
                                      List<String> files)
                               throws TimeoutException

Specified by:

waitTillAllFilesDisappear in interface ConsistencyGuard

Throws:

TimeoutException

waitForFilesVisibility

public void waitForFilesVisibility(String dirPath,
                                   List<String> files,
                                   ConsistencyGuard.FileVisibility event)
                            throws TimeoutException

Helper function to wait for all files belonging to single directory to appear.

Parameters:

dirPath - Dir Path

files - Files to appear/disappear

event - Appear/Disappear

Throws:

TimeoutException

checkFileVisibility

protected boolean checkFileVisibility(StoragePath filePath,
                                      ConsistencyGuard.FileVisibility visibility)
                               throws IOException

Helper to check of file visibility.

Parameters:

filePath - File Path

visibility - Visibility

Returns:

true (if file visible in Path), false (otherwise)

Throws:

IOException - -

checkFilesVisibility

protected boolean checkFilesVisibility(int retryNum,
                                       StoragePath dir,
                                       List<String> files,
                                       ConsistencyGuard.FileVisibility event)

Helper to check for file visibility based on ConsistencyGuard.FileVisibility event.

Parameters:

retryNum - retry attempt count.

dir - directory of interest in which list of files are checked for visibility

files - List of files to check for visibility

event - ConsistencyGuard.FileVisibility event of interest.

Returns:

true if condition succeeded. else false.

getFilesWithoutSchemeAndAuthority

protected List<String> getFilesWithoutSchemeAndAuthority(List<String> files)

Generate file names without scheme and authority.

Parameters:

files - list of files of interest.

Returns:

the filenames without scheme and authority.