org.apache.hudi.execution

Class HoodieLazyInsertIterable<T extends HoodieRecordPayload>

java.lang.Object

org.apache.hudi.client.utils.LazyIterableIterator<HoodieRecord<T>,List<WriteStatus>>

org.apache.hudi.execution.HoodieLazyInsertIterable<T>

All Implemented Interfaces:

Iterable<List<WriteStatus>>, Iterator<List<WriteStatus>>

public abstract class HoodieLazyInsertIterable<T extends HoodieRecordPayload>
extends LazyIterableIterator<HoodieRecord<T>,List<WriteStatus>>

Lazy Iterable, that writes a stream of HoodieRecords sorted by the partitionPath, into new files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HoodieLazyInsertIterable.HoodieInsertValueGenResult<T extends HoodieRecord>

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	areRecordsSorted
protected HoodieWriteConfig	hoodieConfig
protected HoodieTable	hoodieTable
protected String	idPrefix
protected String	instantTime
protected TaskContextSupplier	taskContextSupplier
protected WriteHandleFactory	writeHandleFactory

Fields inherited from class org.apache.hudi.client.utils.LazyIterableIterator

inputItr

Constructor Summary

Constructors

Constructor and Description
HoodieLazyInsertIterable(Iterator<HoodieRecord<T>> recordItr, boolean areRecordsSorted, HoodieWriteConfig config, String instantTime, HoodieTable hoodieTable, String idPrefix, TaskContextSupplier taskContextSupplier)
HoodieLazyInsertIterable(Iterator<HoodieRecord<T>> recordItr, boolean areRecordsSorted, HoodieWriteConfig config, String instantTime, HoodieTable hoodieTable, String idPrefix, TaskContextSupplier taskContextSupplier, WriteHandleFactory writeHandleFactory)

Method Summary

Modifier and Type	Method and Description
protected void	end() Called once, after all elements are processed.
protected CopyOnWriteInsertHandler	getInsertHandler()
protected void	start() Called once, before any elements are processed.

Methods inherited from class org.apache.hudi.client.utils.LazyIterableIterator

computeNext, hasNext, iterator, next, remove

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

hoodieConfig

protected final HoodieWriteConfig hoodieConfig

instantTime

protected final String instantTime

areRecordsSorted

protected boolean areRecordsSorted

hoodieTable

protected final HoodieTable hoodieTable

idPrefix

protected final String idPrefix

taskContextSupplier

protected TaskContextSupplier taskContextSupplier

writeHandleFactory

protected WriteHandleFactory writeHandleFactory

Constructor Detail

HoodieLazyInsertIterable

public HoodieLazyInsertIterable(Iterator<HoodieRecord<T>> recordItr,
                                boolean areRecordsSorted,
                                HoodieWriteConfig config,
                                String instantTime,
                                HoodieTable hoodieTable,
                                String idPrefix,
                                TaskContextSupplier taskContextSupplier)

HoodieLazyInsertIterable

public HoodieLazyInsertIterable(Iterator<HoodieRecord<T>> recordItr,
                                boolean areRecordsSorted,
                                HoodieWriteConfig config,
                                String instantTime,
                                HoodieTable hoodieTable,
                                String idPrefix,
                                TaskContextSupplier taskContextSupplier,
                                WriteHandleFactory writeHandleFactory)

Method Detail

start

protected void start()

Description copied from class: LazyIterableIterator

Called once, before any elements are processed.

Overrides:

start in class LazyIterableIterator<HoodieRecord<T extends HoodieRecordPayload>,List<WriteStatus>>

end

protected void end()

Description copied from class: LazyIterableIterator

Called once, after all elements are processed.

Overrides:

end in class LazyIterableIterator<HoodieRecord<T extends HoodieRecordPayload>,List<WriteStatus>>

getInsertHandler

protected CopyOnWriteInsertHandler getInsertHandler()