org.apache.beam.sdk.io.gcp.bigtable.changestreams.action

Class ResumeFromPreviousPipelineAction

java.lang.Object

org.apache.beam.sdk.io.gcp.bigtable.changestreams.action.ResumeFromPreviousPipelineAction

public class ResumeFromPreviousPipelineAction
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description
ResumeFromPreviousPipelineAction(ChangeStreamMetrics metrics, MetadataTableDao metadataTableDao, org.joda.time.Instant endTime, ProcessNewPartitionsAction processNewPartitionsAction)

Method Summary

Modifier and Type	Method and Description
void	run(org.apache.beam.sdk.transforms.DoFn.OutputReceiver<PartitionRecord> receiver) Resume from previously drained pipeline.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ResumeFromPreviousPipelineAction

public ResumeFromPreviousPipelineAction(ChangeStreamMetrics metrics,
                                        MetadataTableDao metadataTableDao,
                                        @Nullable
                                        org.joda.time.Instant endTime,
                                        ProcessNewPartitionsAction processNewPartitionsAction)

Method Detail

run

public void run(org.apache.beam.sdk.transforms.DoFn.OutputReceiver<PartitionRecord> receiver)
         throws com.google.protobuf.InvalidProtocolBufferException

Resume from previously drained pipeline. Restart StreamPartitions by reading the last metadata write and output PartitionRecord based on that. Process NewPartitions normally.

1. Clear out missing partitions metadata in the metadata table
2. Read all StreamPartitions and output them
3. Read all NewPartitions and output them normally

Parameters:

receiver - output receiver to output

Throws:

com.google.protobuf.InvalidProtocolBufferException - when failing to read from metadata table.