com.solace.connector.beam

Class SolaceIO.Read<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<T>>

com.solace.connector.beam.SolaceIO.Read<T>

All Implemented Interfaces:

Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

SolaceIO

public abstract static class SolaceIO.Read<T>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<T>>

A PTransform to read from a Solace broker.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Constructor Summary

Constructors

Constructor and Description
Read()

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<T>	expand(org.apache.beam.sdk.values.PBegin input)
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
SolaceIO.Read<T>	withAdvanceTimeoutInMillis(int advanceTimeoutInMillis) Sets the message polling timeout for this SolaceIO.Read.
SolaceIO.Read<T>	withCoder(org.apache.beam.sdk.coders.Coder<T> coder) Sets the Coder that this SolaceIO.Read will use to encode and decode the elements of the resulting PCollection.
SolaceIO.Read<T>	withInboundMessageMapper(SolaceIO.InboundMessageMapper<T> inboundMessageMapper) Sets the SolaceIO.InboundMessageMapper that this SolaceIO.Read will use to map Solace messages into elements of the resulting PCollection.
SolaceIO.Read<T>	withJcsmpProperties(com.solacesystems.jcsmp.JCSMPProperties jcsmpProperties) Sets the JCSMP connection config for Solace PubSub+.
SolaceIO.Read<T>	withMaxNumRecords(long maxNumRecords) Sets the max number of records received by this SolaceIO.Read.
SolaceIO.Read<T>	withMaxReadTime(org.joda.time.Duration maxReadTime) Sets the max read time (duration) that this SolaceIO.Read will receive messages.
SolaceIO.Read<T>	withQueues(List<String> queues) Sets the list of pre-configured queues to which this SolaceIO.Read will consume messages from.
SolaceIO.Read<T>	withUseSenderTimestamp(boolean useSenderTimestamp) Sets whether or not for this SolaceIO.Read to use the sender timestamp to determine the freshness of its data.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Read

public Read()

Method Detail

withJcsmpProperties

public SolaceIO.Read<T> withJcsmpProperties(com.solacesystems.jcsmp.JCSMPProperties jcsmpProperties)

Sets the JCSMP connection config for Solace PubSub+.

Note: the JCSMP property JCSMPProperties.CLIENT_NAME must be null. This is because each Apache Beam split has its own Solace session, and multiple sessions cannot share the same client name.

Parameters:

jcsmpProperties - Solace PubSub+ JCSMP connection config

Returns:

a new copy of this SolaceIO.Read configured with the provided JCSMP

withQueues

public SolaceIO.Read<T> withQueues(List<String> queues)

Sets the list of pre-configured queues to which this SolaceIO.Read will consume messages from.

For non-exclusive queues, you may add duplicate queue names in this list to create additional concurrent readers for them.

Parameters:

queues - list of queues

Returns:

a new copy of this SolaceIO.Read configured with the provided queue list

withUseSenderTimestamp

public SolaceIO.Read<T> withUseSenderTimestamp(boolean useSenderTimestamp)

Sets whether or not for this SolaceIO.Read to use the sender timestamp to determine the freshness of its data. Otherwise, the time at which Beam receives the data will be used.

By default, the latency measurement is taken from the time the message enters Dataflow and does not take into account the time sitting in a Solace queue waiting to be processed. If messages are published with sender timestamps and useSenderTimestamp is enabled in the SolaceIO, then end to end latencies will be used and reported. For java clients the JCSMP property JCSMPProperties.GENERATE_SEND_TIMESTAMPS will ensure that each message is sent with a timestamp.

Default:

Parameters:

useSenderTimestamp - set to true to use the data's sender timestamp to determine their freshness

Returns:

a new copy of this SolaceIO.Read configured with the provided indication to use or not use sender timestamps

withAdvanceTimeoutInMillis

public SolaceIO.Read<T> withAdvanceTimeoutInMillis(int advanceTimeoutInMillis)

Sets the message polling timeout for this SolaceIO.Read. If the poll timeout is passed, then this SolaceIO.Read will treat that poll as a null message. i.e. no messages were available from the Solace PubSub+ source.

Default:

Parameters:

advanceTimeoutInMillis - the message polling timeout in milliseconds

Returns:

a new copy of this SolaceIO.Read configured with the provided advance timeout

withMaxNumRecords

public SolaceIO.Read<T> withMaxNumRecords(long maxNumRecords)

Sets the max number of records received by this SolaceIO.Read. When this max number of records is lower than Long.MAX_VALUE, the SolaceIO.Read will provide a bounded PCollection.

Default: Long.MAX_VALUE

Parameters:

maxNumRecords - the maximum number of records to receive

Returns:

a new copy of this SolaceIO.Read configured with the provided maximum number of records

withMaxReadTime

public SolaceIO.Read<T> withMaxReadTime(org.joda.time.Duration maxReadTime)

Sets the max read time (duration) that this SolaceIO.Read will receive messages. When this max read time is not null, the SolaceIO.Read will provide a bounded PCollection.

Parameters:

maxReadTime - the maximum time to receive messages

Returns:

a new copy of this SolaceIO.Read configured with the provided maximum read time

withInboundMessageMapper

public SolaceIO.Read<T> withInboundMessageMapper(SolaceIO.InboundMessageMapper<T> inboundMessageMapper)

Sets the SolaceIO.InboundMessageMapper that this SolaceIO.Read will use to map Solace messages into elements of the resulting PCollection.

Parameters:

inboundMessageMapper - the message mapper

Returns:

a new copy of this SolaceIO.Read configured with the provided mapper

See Also:

SolaceIO.InboundMessageMapper

withCoder

public SolaceIO.Read<T> withCoder(org.apache.beam.sdk.coders.Coder<T> coder)

Sets the Coder that this SolaceIO.Read will use to encode and decode the elements of the resulting PCollection.

Parameters:

coder - the output coder

Returns:

a new copy of this SolaceIO.Read configured with the provided output coder

See Also:

Coder

expand

public org.apache.beam.sdk.values.PCollection<T> expand(org.apache.beam.sdk.values.PBegin input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<T>>

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<T>>