Package org.apache.beam.sdk.io.kafka
Class TimestampPolicy<K,V>
- java.lang.Object
-
- org.apache.beam.sdk.io.kafka.TimestampPolicy<K,V>
-
- Direct Known Subclasses:
CustomTimestampPolicyWithLimitedDelay,TimestampPolicyFactory.LogAppendTimePolicy,TimestampPolicyFactory.ProcessingTimePolicy,TimestampPolicyFactory.TimestampFnPolicy
public abstract class TimestampPolicy<K,V> extends java.lang.ObjectA timestamp policy to assign event time for messages in a Kafka partition and watermark for it. KafkaIO reader creates one policy usingTimestampPolicyFactoryfor each each of the partitions it reads from. See @TimestampPolicyFactory.LogAppendTimePolicyfor example of a policy.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classTimestampPolicy.PartitionContextThe context contains state maintained in the reader for the partition.
-
Constructor Summary
Constructors Constructor Description TimestampPolicy()
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description abstract org.joda.time.InstantgetTimestampForRecord(TimestampPolicy.PartitionContext ctx, KafkaRecord<K,V> record)Returns record timestamp (aka event time).abstract org.joda.time.InstantgetWatermark(TimestampPolicy.PartitionContext ctx)Returns watermark for the partition.
-
-
-
Method Detail
-
getTimestampForRecord
public abstract org.joda.time.Instant getTimestampForRecord(TimestampPolicy.PartitionContext ctx, KafkaRecord<K,V> record)
Returns record timestamp (aka event time). This is often based on the timestamp of the Kafka record. This is invoked for each record when it is processed in the reader.
-
getWatermark
public abstract org.joda.time.Instant getWatermark(TimestampPolicy.PartitionContext ctx)
Returns watermark for the partition. It is the timestamp before or at the timestamps of all future records consumed from the partition. SeeUnboundedSource.UnboundedReader.getWatermark()for more guidance on watermarks. E.g. if the record timestamp is 'LogAppendTime', watermark would be the timestamp of the last record since 'LogAppendTime' monotonically increases within a partition.
-
-