org.apache.gobblin.dataset.test

Class StaticDatasetsFinderForTesting

java.lang.Object

org.apache.gobblin.dataset.test.StaticDatasetsFinderForTesting

All Implemented Interfaces:

DatasetsFinder<Dataset>, IterableDatasetFinder<Dataset>

public class StaticDatasetsFinderForTesting
extends Object
implements IterableDatasetFinder<Dataset>

A DatasetsFinder that returns a predefined set of Datasets for testing.

Constructor Summary

Constructors

Constructor and Description
StaticDatasetsFinderForTesting()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path	commonDatasetRoot()
List<Dataset>	findDatasets() Find all Datasets in the file system.
Iterator<Dataset>	getDatasetsIterator()
java.util.stream.Stream<Dataset>	getDatasetsStream(int desiredCharacteristics, Comparator<Dataset> suggestedOrder) Get a stream of Datasets found.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StaticDatasetsFinderForTesting

public StaticDatasetsFinderForTesting()

Method Detail

findDatasets

public List<Dataset> findDatasets()
                           throws IOException

Description copied from interface: DatasetsFinder

Find all Datasets in the file system.

Specified by:

findDatasets in interface DatasetsFinder<Dataset>

Returns:

List of Datasets in the file system.

Throws:

IOException

commonDatasetRoot

public org.apache.hadoop.fs.Path commonDatasetRoot()

Specified by:

commonDatasetRoot in interface DatasetsFinder<Dataset>

Returns:

The deepest common root shared by all Datasets root paths returned by this finder.

getDatasetsIterator

public Iterator<Dataset> getDatasetsIterator()
                                      throws IOException

Specified by:

getDatasetsIterator in interface IterableDatasetFinder<Dataset>

Returns:

An Iterator over the Datasets found.

Throws:

IOException

getDatasetsStream

public java.util.stream.Stream<Dataset> getDatasetsStream(int desiredCharacteristics,
                                                          Comparator<Dataset> suggestedOrder)
                                                   throws IOException

Description copied from interface: IterableDatasetFinder

Get a stream of Datasets found.

Specified by:

getDatasetsStream in interface IterableDatasetFinder<Dataset>

Parameters:

desiredCharacteristics - desired Spliterator characteristics of this stream. The returned stream need not satisfy these characteristics, this argument merely implies that the caller will run optimally when those characteristics are present, allowing pushdown of those characteristics. For example Spliterator.SORTED can sometimes be pushed down at a cost, so the DatasetsFinder would only push it down if it is valuable for the caller.

suggestedOrder - suggested order of the datasets in the stream. Implementation may or may not return the entries in that order. If the entries are in that order, implementation should ensure the spliterator is annotated as such.

Returns:

a stream of Datasets found.

Throws:

IOException