io.debezium.connector.postgresql.connection

Class PostgresReplicationConnection

java.lang.Object

io.debezium.jdbc.JdbcConnection

io.debezium.connector.postgresql.connection.PostgresReplicationConnection

All Implemented Interfaces:

ReplicationConnection, AutoCloseable

public class PostgresReplicationConnection
extends JdbcConnection
implements ReplicationConnection

Implementation of a ReplicationConnection for Postgresql. Note that replication connections in PG cannot execute regular statements but only a limited number of replication-related commands.

Author:

Horia Chiorean (hchiorea@redhat.com)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	PostgresReplicationConnection.ReplicationConnectionBuilder

Nested classes/interfaces inherited from class io.debezium.jdbc.JdbcConnection

JdbcConnection.BlockingMultiResultSetConsumer, JdbcConnection.BlockingResultSetConsumer, JdbcConnection.CallPreparer, JdbcConnection.ConnectionFactory, JdbcConnection.MultiResultSetConsumer, JdbcConnection.Operations, JdbcConnection.ParameterResultSetConsumer, JdbcConnection.ResultSetConsumer, JdbcConnection.ResultSetMapper<T>, JdbcConnection.StatementFactory, JdbcConnection.StatementPreparer

Nested classes/interfaces inherited from interface io.debezium.connector.postgresql.connection.ReplicationConnection

ReplicationConnection.Builder

Field Summary

Fields

Modifier and Type	Field and Description
private long	defaultStartingPos
private boolean	dropSlotOnClose
private boolean	exportSnapshot
private boolean	hasInitedSlot
private static org.slf4j.Logger	LOGGER
private MessageDecoder	messageDecoder
private Configuration	originalConfig
private PostgresConnectorConfig.LogicalDecoder	plugin
private String	publicationName
private SlotCreationResult	slotCreationInfo
private String	slotName
private Duration	statusUpdateInterval
private Properties	streamParams
private TypeRegistry	typeRegistry

Constructor Summary

Constructors

Modifier	Constructor and Description
private	PostgresReplicationConnection(Configuration config, String slotName, String publicationName, PostgresConnectorConfig.LogicalDecoder plugin, boolean dropSlotOnClose, boolean exportSnapshot, Duration statusUpdateInterval, TypeRegistry typeRegistry, Properties streamParams, PostgresSchema schema) Creates a new replication connection with the given params.

Method Summary

Modifier and Type	Method and Description
void	close()
Optional<SlotCreationResult>	createReplicationSlot() Creates a new replication slot with the given option and returns the result of the command, which may contain results (depending on postgres versions)
private ReplicationStream	createReplicationStream(org.postgresql.replication.LogSequenceNumber startLsn, boolean skipFirstFlushRecord)
protected static void	defaultSettings(Configuration.Builder builder)
private ServerInfo.ReplicationSlot	getSlotInfo()
void	initConnection() Forces the connection to be created, is called by startStreaming, but can be called manually in cases where we want the connection but not to to start streaming yet.
protected void	initPublication()
protected void	initReplicationSlot()
private SlotCreationResult	parseSlotCreation(ResultSet rs)
protected org.postgresql.jdbc.PgConnection	pgConnection()
private org.postgresql.replication.PGReplicationStream	startPgReplicationStream(org.postgresql.replication.LogSequenceNumber lsn, Function<org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder,org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder> configurator)
ReplicationStream	startStreaming() creating a replication connection and starting to stream involves a few steps: 1.
ReplicationStream	startStreaming(Long offset) Opens a stream for reading logical replication changes from a given LSN position.
private boolean	useTemporarySlot()

Methods inherited from class io.debezium.jdbc.JdbcConnection

call, columnsFor, columnsFor, commit, config, connect, connection, connection, connectionString, database, execute, execute, executeWithoutCommitting, isConnected, isNullable, parseSqlStatementString, patternBasedFactory, patternBasedFactory, prepareQuery, prepareQuery, prepareQuery, prepareQuery, prepareQueryAndMap, prepareQueryWithBlockingConsumer, prepareUpdate, print, print, query, query, queryAndMap, queryAndMap, queryWithBlockingConsumer, readAllCatalogNames, readAllSchemaNames, readAllTableNames, readPrimaryKeyNames, readPrimaryKeyOrUniqueIndexNames, readSchema, readTableColumn, readTableNames, readTableUniqueIndices, resolveJdbcType, resolveNativeType, setAutoCommit, tableTypes, username

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.debezium.connector.postgresql.connection.ReplicationConnection

builder, format, isConnected

Field Detail

LOGGER

private static org.slf4j.Logger LOGGER

slotName

private final String slotName

publicationName

private final String publicationName

plugin

private final PostgresConnectorConfig.LogicalDecoder plugin

dropSlotOnClose

private final boolean dropSlotOnClose

exportSnapshot

private final boolean exportSnapshot

originalConfig

private final Configuration originalConfig

statusUpdateInterval

private final Duration statusUpdateInterval

messageDecoder

private final MessageDecoder messageDecoder

typeRegistry

private final TypeRegistry typeRegistry

streamParams

private final Properties streamParams

defaultStartingPos

private long defaultStartingPos

slotCreationInfo

private SlotCreationResult slotCreationInfo

hasInitedSlot

private boolean hasInitedSlot

Constructor Detail

PostgresReplicationConnection

private PostgresReplicationConnection(Configuration config,
                                      String slotName,
                                      String publicationName,
                                      PostgresConnectorConfig.LogicalDecoder plugin,
                                      boolean dropSlotOnClose,
                                      boolean exportSnapshot,
                                      Duration statusUpdateInterval,
                                      TypeRegistry typeRegistry,
                                      Properties streamParams,
                                      PostgresSchema schema)

Creates a new replication connection with the given params.

Parameters:

config - the JDBC configuration for the connection; may not be null

slotName - the name of the DB slot for logical replication; may not be null

publicationName - the name of the DB publication for logical replication; may not be null

plugin - decoder matching the server side plug-in used for streaming changes; may not be null

dropSlotOnClose - whether the replication slot should be dropped once the connection is closed

statusUpdateInterval - the interval at which the replication connection should periodically send status

exportSnapshot - whether the replication should export a snapshot when created

typeRegistry - registry with PostgreSQL types

streamParams - additional parameters to pass to the replication stream

schema - the schema; must not be null updates to the server

Method Detail

getSlotInfo

private ServerInfo.ReplicationSlot getSlotInfo()
                                        throws SQLException,
                                               InterruptedException

Throws:

SQLException

InterruptedException

initPublication

protected void initPublication()

initReplicationSlot

protected void initReplicationSlot()
                            throws SQLException,
                                   InterruptedException

Throws:

SQLException

InterruptedException

useTemporarySlot

private boolean useTemporarySlot()
                          throws SQLException

Throws:

SQLException

startStreaming

public ReplicationStream startStreaming()
                                 throws SQLException,
                                        InterruptedException

creating a replication connection and starting to stream involves a few steps: 1. we create the connection and ensure that a. the slot exists b. the slot isn't currently being used 2. we query to get our potential start position in the slot (lsn) 3. we try and start streaming, depending on our options (such as in wal2json) this may fail, which can result in the connection being killed and we need to start the process over if we are using a temporary slot 4. actually start the streamer This method takes care of all of these and this method queries for a default starting position If you know where you are starting from you should call startStreaming(Long), this method delegates to that method

Specified by:

startStreaming in interface ReplicationConnection

Returns:

Throws:

SQLException

InterruptedException

startStreaming

public ReplicationStream startStreaming(Long offset)
                                 throws SQLException,
                                        InterruptedException

Description copied from interface: ReplicationConnection

Opens a stream for reading logical replication changes from a given LSN position.

Note that it is possible for a server to have recycled old WAL segments (see the wal_keep_segments setting). If that is the case, then even though a LSN number may be valid, the server will not stream back any changes because they are not available.

Specified by:

startStreaming in interface ReplicationConnection

Parameters:

offset - a value representing the WAL sequence number where replication should start from; if the value is null or negative, this behaves exactly like ReplicationConnection.startStreaming().

Returns:

a PGReplicationStream from which data is read; never null

Throws:

SQLException - if anything fails

InterruptedException

See Also:

LogSequenceNumber

initConnection

public void initConnection()
                    throws SQLException,
                           InterruptedException

Description copied from interface: ReplicationConnection

Forces the connection to be created, is called by startStreaming, but can be called manually in cases where we want the connection but not to to start streaming yet. Can be called multiple times

Specified by:

initConnection in interface ReplicationConnection

Throws:

SQLException

InterruptedException

createReplicationSlot

public Optional<SlotCreationResult> createReplicationSlot()
                                                   throws SQLException

Description copied from interface: ReplicationConnection

Creates a new replication slot with the given option and returns the result of the command, which may contain results (depending on postgres versions)

Specified by:

createReplicationSlot in interface ReplicationConnection

Throws:

SQLException - if anything fails

pgConnection

protected org.postgresql.jdbc.PgConnection pgConnection()
                                                 throws SQLException

Throws:

SQLException

parseSlotCreation

private SlotCreationResult parseSlotCreation(ResultSet rs)

createReplicationStream

private ReplicationStream createReplicationStream(org.postgresql.replication.LogSequenceNumber startLsn,
                                                  boolean skipFirstFlushRecord)
                                           throws SQLException,
                                                  InterruptedException

Throws:

SQLException

InterruptedException

startPgReplicationStream

private org.postgresql.replication.PGReplicationStream startPgReplicationStream(org.postgresql.replication.LogSequenceNumber lsn,
                                                                                Function<org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder,org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder> configurator)
                                                                         throws SQLException

Throws:

SQLException

close

public void close()

Specified by:

close in interface AutoCloseable

Overrides:

close in class JdbcConnection

defaultSettings

protected static void defaultSettings(Configuration.Builder builder)