Package io.debezium.connector.postgresql

Class PostgresConnectorTask

java.lang.Object

org.apache.kafka.connect.source.SourceTask

io.debezium.connector.common.BaseSourceTask<PostgresPartition,PostgresOffsetContext>

io.debezium.connector.postgresql.PostgresConnectorTask

All Implemented Interfaces:

org.apache.kafka.connect.connector.Task

public class PostgresConnectorTask
extends BaseSourceTask<PostgresPartition,PostgresOffsetContext>

Kafka connect source task which uses Postgres logical decoding over a streaming replication connection to process DB changes.

Author:

Horia Chiorean (hchiorea@redhat.com)

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.connector.common.BaseSourceTask

BaseSourceTask.State

Nested classes/interfaces inherited from class org.apache.kafka.connect.source.SourceTask

org.apache.kafka.connect.source.SourceTask.TransactionBoundary

Field Summary

Fields

Modifier and Type	Field	Description
private PostgresConnection	beanRegistryJdbcConnection
private final ReentrantLock	commitLock
private static final String	CONTEXT_NAME
private ErrorHandler	errorHandler
private PostgresConnection	jdbcConnection
private static final org.slf4j.Logger	LOGGER
private OffsetContext.Loader<PostgresOffsetContext>	offsetContextLoader
private Partition.Provider<PostgresPartition>	partitionProvider
private ChangeEventQueue<DataChangeEvent>	queue
private ReplicationConnection	replicationConnection
private PostgresSchema	schema
private PostgresTaskContext	taskContext

Fields inherited from class io.debezium.connector.common.BaseSourceTask

coordinator

Fields inherited from class org.apache.kafka.connect.source.SourceTask

context, TRANSACTION_BOUNDARY_CONFIG

Constructor Summary

Constructors

Constructor	Description
PostgresConnectorTask()

Method Summary

Modifier and Type	Method	Description
private static void	checkWalLevel(PostgresConnection connection, SnapshotterService snapshotterService)
void	commit()
void	commitRecord(org.apache.kafka.connect.source.SourceRecord record)
void	commitRecord(org.apache.kafka.connect.source.SourceRecord record, org.apache.kafka.clients.producer.RecordMetadata metadata)
ReplicationConnection	createReplicationConnection(PostgresTaskContext taskContext, int maxRetries, Duration retryDelay)
List<org.apache.kafka.connect.source.SourceRecord>	doPoll()
protected void	doStop()
protected Iterable<Field>	getAllConfigurationFields()
private SlotState	getSlotState(PostgresConnectorConfig connectorConfig)
PostgresTaskContext	getTaskContext()
ChangeEventSourceCoordinator<PostgresPartition,PostgresOffsetContext>	start(Configuration config)
private SlotCreationResult	tryToCreateSlot(io.debezium.spi.snapshot.Snapshotter snapshotter, PostgresConnectorConfig connectorConfig, SlotState slotInfo)
String	version()

Methods inherited from class io.debezium.connector.common.BaseSourceTask

getAvailableSignalChannels, getNotificationChannels, getPreviousOffsets, getTaskState, isLogPositionAvailable, logStatistics, poll, registerServiceProviders, resetErrorHandlerRetriesIfNeeded, start, stop, validateAndLoadSchemaHistory, withMaskedSensitiveOptions

Methods inherited from class org.apache.kafka.connect.source.SourceTask

initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

CONTEXT_NAME

private static final String CONTEXT_NAME

See Also:

• Constant Field Values

taskContext

private volatile PostgresTaskContext taskContext

queue

private volatile ChangeEventQueue<DataChangeEvent> queue

jdbcConnection

private volatile PostgresConnection jdbcConnection

beanRegistryJdbcConnection

private volatile PostgresConnection beanRegistryJdbcConnection

replicationConnection

private volatile ReplicationConnection replicationConnection

errorHandler

private volatile ErrorHandler errorHandler

schema

private volatile PostgresSchema schema

partitionProvider

private Partition.Provider<PostgresPartition> partitionProvider

offsetContextLoader

private OffsetContext.Loader<PostgresOffsetContext> offsetContextLoader

commitLock

private final ReentrantLock commitLock

Constructor Details

PostgresConnectorTask

public PostgresConnectorTask()

Method Details

start

public ChangeEventSourceCoordinator<PostgresPartition,PostgresOffsetContext> start(Configuration config)

Specified by:

start in class BaseSourceTask<PostgresPartition,PostgresOffsetContext>

tryToCreateSlot

private SlotCreationResult tryToCreateSlot(io.debezium.spi.snapshot.Snapshotter snapshotter,
 PostgresConnectorConfig connectorConfig,
 SlotState slotInfo)

getSlotState

private SlotState getSlotState(PostgresConnectorConfig connectorConfig)

createReplicationConnection

public ReplicationConnection createReplicationConnection(PostgresTaskContext taskContext,
 int maxRetries,
 Duration retryDelay)
                                                  throws org.apache.kafka.connect.errors.ConnectException

Throws:

org.apache.kafka.connect.errors.ConnectException

doPoll

public List<org.apache.kafka.connect.source.SourceRecord> doPoll()
                                                          throws InterruptedException

Specified by:

doPoll in class BaseSourceTask<PostgresPartition,PostgresOffsetContext>

Throws:

InterruptedException

doStop

protected void doStop()

Specified by:

doStop in class BaseSourceTask<PostgresPartition,PostgresOffsetContext>

version

public String version()

getAllConfigurationFields

protected Iterable<Field> getAllConfigurationFields()

Specified by:

getAllConfigurationFields in class BaseSourceTask<PostgresPartition,PostgresOffsetContext>

commitRecord

public void commitRecord(org.apache.kafka.connect.source.SourceRecord record,
 org.apache.kafka.clients.producer.RecordMetadata metadata)
                  throws InterruptedException

Overrides:

commitRecord in class org.apache.kafka.connect.source.SourceTask

Throws:

InterruptedException

commitRecord

public void commitRecord(org.apache.kafka.connect.source.SourceRecord record)
                  throws InterruptedException

Overrides:

commitRecord in class BaseSourceTask<PostgresPartition,PostgresOffsetContext>

Throws:

InterruptedException

commit

public void commit()
            throws InterruptedException

Overrides:

commit in class BaseSourceTask<PostgresPartition,PostgresOffsetContext>

Throws:

InterruptedException

getTaskContext

public PostgresTaskContext getTaskContext()

checkWalLevel

private static void checkWalLevel(PostgresConnection connection,
 SnapshotterService snapshotterService)
                           throws SQLException

Throws:

SQLException