org.apache.hudi.integ.testsuite.helpers

Class DFSTestSuitePathSelector

java.lang.Object

org.apache.hudi.utilities.sources.helpers.DFSPathSelector

org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

All Implemented Interfaces:

Serializable

public class DFSTestSuitePathSelector
extends DFSPathSelector

A custom dfs path selector used only for the hudi test suite. To be used only if workload is not run inline.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hudi.utilities.sources.helpers.DFSPathSelector

DFSPathSelector.Config

Field Summary

Fields inherited from class org.apache.hudi.utilities.sources.helpers.DFSPathSelector

fs, IGNORE_FILEPREFIX_LIST, props

Constructor Summary

Constructors

Constructor and Description
DFSTestSuitePathSelector(org.apache.hudi.common.config.TypedProperties props, org.apache.hadoop.conf.Configuration hadoopConf)

Method Summary

Modifier and Type	Method and Description
org.apache.hudi.common.util.collection.Pair<org.apache.hudi.common.util.Option<String>,String>	getNextFilePathsAndMaxModificationTime(org.apache.hudi.common.util.Option<String> lastCheckpointStr, long sourceLimit)

Methods inherited from class org.apache.hudi.utilities.sources.helpers.DFSPathSelector

createSourceSelector, getNextFilePathsAndMaxModificationTime, listEligibleFiles

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DFSTestSuitePathSelector

public DFSTestSuitePathSelector(org.apache.hudi.common.config.TypedProperties props,
                                org.apache.hadoop.conf.Configuration hadoopConf)

Method Detail

getNextFilePathsAndMaxModificationTime

public org.apache.hudi.common.util.collection.Pair<org.apache.hudi.common.util.Option<String>,String> getNextFilePathsAndMaxModificationTime(org.apache.hudi.common.util.Option<String> lastCheckpointStr,
                                                                                                                                             long sourceLimit)

Overrides:

getNextFilePathsAndMaxModificationTime in class DFSPathSelector