org.apache.beam.sdk.io.gcp.spanner.changestreams.restriction

Class TimestampRangeTracker

java.lang.Object

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>

org.apache.beam.sdk.io.gcp.spanner.changestreams.restriction.TimestampRangeTracker

All Implemented Interfaces:

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.HasProgress

Direct Known Subclasses:

DetectNewPartitionsRangeTracker, ReadChangeStreamPartitionRangeTracker

public class TimestampRangeTracker
extends org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>
implements org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.HasProgress

A RestrictionTracker for claiming positions in a TimestampRange in a monotonically increasing fashion.

The smallest position is Timestamp.MIN_VALUE and the largest position is Timestamp.MAX_VALUE - 1 nanosecond.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.HasProgress, org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.IsBounded, org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.Progress, org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.TruncateResult<RestrictionT>

Field Summary

Fields

Modifier and Type	Field and Description
protected @Nullable com.google.cloud.Timestamp	lastAttemptedPosition
protected @Nullable com.google.cloud.Timestamp	lastClaimedPosition
protected TimestampRange	range

Constructor Summary

Constructors

Constructor and Description
TimestampRangeTracker(TimestampRange range)

Method Summary

Modifier and Type	Method and Description
void	checkDone() Checks if the restriction has been processed successfully.
TimestampRange	currentRestriction()
org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.Progress	getProgress() Returns the progress made within the restriction so far.
org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.IsBounded	isBounded()
boolean	tryClaim(com.google.cloud.Timestamp position) Attempts to claim the given position.
@Nullable org.apache.beam.sdk.transforms.splittabledofn.SplitResult<TimestampRange>	trySplit(double fractionOfRemainder) Splits the restriction through the following algorithm:

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

range

protected TimestampRange range

lastAttemptedPosition

protected @Nullable com.google.cloud.Timestamp lastAttemptedPosition

lastClaimedPosition

protected @Nullable com.google.cloud.Timestamp lastClaimedPosition

Constructor Detail

TimestampRangeTracker

public TimestampRangeTracker(TimestampRange range)

Method Detail

tryClaim

public boolean tryClaim(com.google.cloud.Timestamp position)

Attempts to claim the given position. Depending on the position following outcomes are possible:

• If the position is less than or equal to a previous attempted one, an IllegalArgumentException will be thrown
• If the position is less than the restriction range start, an IllegalArgumentException will be thrown
• If the position is greater than or equal to the range end, the position will not be claimed
• If the position is greater or equal to the range start and less than the range end, the position will be claimed

If an error is not thrown, this function will register the position as the lastAttemptedPosition.

Specified by:

tryClaim in class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>

Returns:

true if the position was successfully claimed, false otherwise

trySplit

public @Nullable org.apache.beam.sdk.transforms.splittabledofn.SplitResult<TimestampRange> trySplit(double fractionOfRemainder)

Splits the restriction through the following algorithm:

    currentPosition = lastAttemptedPosition == null ? (from - 1ns) : lastAttemptedPosition
    splitPosition = currentPosition + max(1, (range.to - currentPosition) * fractionOfRemainder)
    primary = [range.from, splitPosition)
    residual = [splitPosition, range.to)
    this.range = primary
 

If the splitPosition is greater than the range.to, null will be returned. For checkpoints the fractionOfRemainder will always be zero.

Specified by:

trySplit in class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>

Returns:

a SplitResult if a split was possible or null if the splitPosition is beyond the end of the range.

checkDone

public void checkDone()
               throws java.lang.IllegalStateException

Checks if the restriction has been processed successfully. If not, throws an IllegalStateException.

The restriction is considered processed successfully if:

• The range is empty (range.from == range.to)
• The lastAttemptedPosition + 1ns >= range.to

The restriction is considered not processed successfully if:

• No position claim was attempted for a non-empty range
• The lastAttemptedPosition + 1ns < range.to

Specified by:

checkDone in class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>

Throws:

java.lang.IllegalStateException

getProgress

public org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.Progress getProgress()

Returns the progress made within the restriction so far. This progress is returned in a normalized fashion from the interval [0, 1]. Zero means no work indicates no work (completed or remaining), while 1 indicates all work (completed or remaining).

If no position was attempted, it will return workCompleted as 0 and workRemaining as 1. If a position was attempted, it will return the fraction of work completed and work remaining based on the offset the position represents in the restriction range. If the last position attempted was greater than the end of the restriction range, it will return workCompleted as 1 and workRemaining as 0.

Specified by:

getProgress in interface org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.HasProgress

Returns:

work completed and work remaining as fractions between [0, 1]

currentRestriction

public TimestampRange currentRestriction()

Specified by:

currentRestriction in class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>

isBounded

public org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.IsBounded isBounded()

Specified by:

isBounded in class org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<TimestampRange,com.google.cloud.Timestamp>