org.apache.beam.sdk.io.gcp.spanner.changestreams.restriction

Class ReadChangeStreamPartitionRangeTracker

java.lang.Object

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<org.apache.beam.sdk.io.range.OffsetRange,java.lang.Long>

org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker

org.apache.beam.sdk.io.gcp.spanner.changestreams.restriction.ReadChangeStreamPartitionRangeTracker

All Implemented Interfaces:

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.HasProgress

public class ReadChangeStreamPartitionRangeTracker
extends org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker

This restriction tracker is a decorator on top of the OffsetRangeTracker. It modifies the behaviour of OffsetRangeTracker.tryClaim(Long) to ignore claims for the same long multiple times. This is because several change stream records might have the same timestamp, thus leading to multiple claims of the same Long. Other than that, it modifies the OffsetRangeTracker.trySplit(double) method to always deny splits for the InitialPartition.PARTITION_TOKEN, since we only need to perform this query once.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.HasProgress, org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.IsBounded, org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.Progress, org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.TruncateResult<RestrictionT>

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker

lastAttemptedOffset, lastClaimedOffset, range

Constructor Summary

Constructors

Constructor and Description
ReadChangeStreamPartitionRangeTracker(PartitionMetadata partition, org.apache.beam.sdk.io.range.OffsetRange range) Receives the partition that will be queried and be using this tracker, alongside the range itself.

Method Summary

Modifier and Type	Method and Description
boolean	tryClaim(java.lang.Long i) Attempts to claim the given offset.
org.apache.beam.sdk.transforms.splittabledofn.SplitResult<org.apache.beam.sdk.io.range.OffsetRange>	trySplit(double fractionOfRemainder) If the partition token is the InitialPartition.PARTITION_TOKEN, it does not allow for splits (returns null).

Methods inherited from class org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker

checkDone, currentRestriction, getProgress, isBounded, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ReadChangeStreamPartitionRangeTracker

public ReadChangeStreamPartitionRangeTracker(PartitionMetadata partition,
                                             org.apache.beam.sdk.io.range.OffsetRange range)

Receives the partition that will be queried and be using this tracker, alongside the range itself.

Parameters:

partition - the partition that will use the tracker

range - closed / open range interval representing the start / end times for a partition

Method Detail

tryClaim

public boolean tryClaim(java.lang.Long i)

Attempts to claim the given offset.

Must be equal or larger than the last successfully claimed offset.

Overrides:

tryClaim in class org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker

Returns:

true if the offset was successfully claimed, false if it is outside the current OffsetRange of this tracker (in that case this operation is a no-op).

trySplit

public org.apache.beam.sdk.transforms.splittabledofn.SplitResult<org.apache.beam.sdk.io.range.OffsetRange> trySplit(double fractionOfRemainder)

If the partition token is the InitialPartition.PARTITION_TOKEN, it does not allow for splits (returns null).

Overrides:

trySplit in class org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker