class SnapshotProcessor extends TableProcessorBase
This implementation of TableProcessor process data from Delta table Snapshot.
- Alphabetic
- By Inheritance
- SnapshotProcessor
- TableProcessorBase
- TableProcessor
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new SnapshotProcessor(deltaTablePath: Path, snapshot: Snapshot, fileEnumerator: AddFileEnumerator[DeltaSourceSplit], alreadyProcessedPaths: Collection[Path])
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getSnapshotVersion(): Long
- returns
A
Snapshotversion that this processor reads.
- Definition Classes
- SnapshotProcessor → TableProcessor
- Annotations
- @Override()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
prepareSplits(changes: ChangesPerVersion[AddFile], splitFilter: SplitFilter[Path]): List[DeltaSourceSplit]
- Attributes
- protected[processor]
- Definition Classes
- TableProcessorBase
-
def
process(processCallback: Consumer[List[DeltaSourceSplit]]): Unit
Process all
AddFilefromSnapshotpassed to thisSnapshotProcessorconstructor by converting them toDeltaSourceSplitobjects.Process all
AddFilefromSnapshotpassed to thisSnapshotProcessorconstructor by converting them toDeltaSourceSplitobjects.- processCallback
A
Consumercallback that will be called after converting allAddFiletoDeltaSourceSplit.
- Definition Classes
- SnapshotProcessor → TableProcessor
- Annotations
- @Override()
-
def
setUpEnumeratorContext(addFiles: List[AddFile], snapshotVersion: Long): AddFileEnumeratorContext
- Attributes
- protected[processor]
- Definition Classes
- TableProcessorBase
-
def
snapshotState(checkpointBuilder: DeltaEnumeratorStateCheckpointBuilder[DeltaSourceSplit]): DeltaEnumeratorStateCheckpointBuilder[DeltaSourceSplit]
Add
TableProcessorstate information toDeltaEnumeratorStateCheckpointBuilderto be stored in Flink's checkpoint.Add
TableProcessorstate information toDeltaEnumeratorStateCheckpointBuilderto be stored in Flink's checkpoint.The implementation of this method should add the latest state information to
DeltaEnumeratorStateCheckpointBuilderneeded to recreateTableProcessorinstance during Flink recovery.- checkpointBuilder
the
DeltaEnumeratorStateCheckpointBuilderinstance that should be updated withTableProcessorstate information.- returns
the
DeltaEnumeratorStateCheckpointBuilderinstance withTableProcessorstate information.
- Definition Classes
- SnapshotProcessor → TableProcessor
- Annotations
- @Override()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()