Class PostgresReplicationConnection
- java.lang.Object
-
- io.debezium.jdbc.JdbcConnection
-
- io.debezium.connector.postgresql.connection.PostgresReplicationConnection
-
- All Implemented Interfaces:
ReplicationConnection,AutoCloseable
public class PostgresReplicationConnection extends JdbcConnection implements ReplicationConnection
Implementation of aReplicationConnectionfor Postgresql. Note that replication connections in PG cannot execute regular statements but only a limited number of replication-related commands.- Author:
- Horia Chiorean (hchiorea@redhat.com)
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static classPostgresReplicationConnection.ReplicationConnectionBuilder-
Nested classes/interfaces inherited from class io.debezium.jdbc.JdbcConnection
JdbcConnection.BlockingMultiResultSetConsumer, JdbcConnection.BlockingResultSetConsumer, JdbcConnection.CallPreparer, JdbcConnection.ConnectionFactory, JdbcConnection.MultiResultSetConsumer, JdbcConnection.Operations, JdbcConnection.ParameterResultSetConsumer, JdbcConnection.ResultSetConsumer, JdbcConnection.ResultSetExtractor<T extends Object>, JdbcConnection.ResultSetMapper<T extends Object>, JdbcConnection.StatementFactory, JdbcConnection.StatementPreparer
-
Nested classes/interfaces inherited from interface io.debezium.connector.postgresql.connection.ReplicationConnection
ReplicationConnection.Builder
-
-
Field Summary
Fields Modifier and Type Field Description private LsndefaultStartingPosprivate booleandropSlotOnCloseprivate booleanhasInitedSlotprivate static org.slf4j.LoggerLOGGERprivate MessageDecodermessageDecoderprivate PostgresConnectorConfigoriginalConfigprivate PostgresConnectorConfig.LogicalDecoderpluginprivate PostgresConnectorConfig.AutoCreateModepublicationAutocreateModeprivate StringpublicationNameprivate SlotCreationResultslotCreationInfoprivate StringslotNameprivate DurationstatusUpdateIntervalprivate PropertiesstreamParamsprivate RelationalTableFilterstableFilterprivate TypeRegistrytypeRegistry
-
Constructor Summary
Constructors Modifier Constructor Description privatePostgresReplicationConnection(PostgresConnectorConfig config, String slotName, String publicationName, RelationalTableFilters tableFilter, PostgresConnectorConfig.AutoCreateMode publicationAutocreateMode, PostgresConnectorConfig.LogicalDecoder plugin, boolean dropSlotOnClose, boolean doSnapshot, Duration statusUpdateInterval, TypeRegistry typeRegistry, Properties streamParams, PostgresSchema schema)Creates a new replication connection with the given params.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidclose()voidclose(boolean dropSlot)Optional<SlotCreationResult>createReplicationSlot()Creates a new replication slot with the given option and returns the result of the command, which may contain results (depending on postgres versions)private ReplicationStreamcreateReplicationStream(Lsn startLsn, WalPositionLocator walPosition)protected static voiddefaultSettings(Configuration.Builder builder)private Set<TableId>determineCapturedTables()private ServerInfo.ReplicationSlotgetSlotInfo()voidinitConnection()Forces the connection to be created, is called by startStreaming, but can be called manually in cases where we want the connection but not to to start streaming yet.protected voidinitPublication()protected voidinitReplicationSlot()private SlotCreationResultparseSlotCreation(ResultSet rs)protected org.postgresql.core.BaseConnectionpgConnection()voidreconnect()private org.postgresql.replication.PGReplicationStreamstartPgReplicationStream(Lsn lsn, Function<org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder,org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder> configurator)ReplicationStreamstartStreaming(Lsn offset, WalPositionLocator walPosition)Opens a stream for reading logical replication changes from a given LSN position.ReplicationStreamstartStreaming(WalPositionLocator walPosition)creating a replication connection and starting to stream involves a few steps: 1.private booleanuseTemporarySlot()-
Methods inherited from class io.debezium.jdbc.JdbcConnection
buildSelectWithRowLimits, call, commit, config, connect, connection, connection, connectionString, database, execute, execute, executeWithoutCommitting, getColumnValue, getDefaultValue, isConnected, isNullable, isTableUniqueIndexIncluded, isValid, parseSqlStatementString, patternBasedFactory, patternBasedFactory, prepareQuery, prepareQuery, prepareQuery, prepareQuery, prepareQuery, prepareQueryAndMap, prepareQueryWithBlockingConsumer, prepareUpdate, print, print, query, query, queryAndMap, queryAndMap, querySingleValue, queryWithBlockingConsumer, quotedColumnIdString, quotedTableIdString, readAllCatalogNames, readAllSchemaNames, readAllTableNames, readPrimaryKeyNames, readPrimaryKeyOrUniqueIndexNames, readSchema, readTableColumn, readTableNames, readTablePreparedStatement, readTableStatement, readTableUniqueIndices, resolveJdbcType, resolveNativeType, rollback, rowToArray, setAutoCommit, singleResultMapper, tableTypes, username
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface io.debezium.connector.postgresql.connection.ReplicationConnection
isConnected
-
-
-
-
Field Detail
-
LOGGER
private static org.slf4j.Logger LOGGER
-
slotName
private final String slotName
-
publicationName
private final String publicationName
-
tableFilter
private final RelationalTableFilters tableFilter
-
publicationAutocreateMode
private final PostgresConnectorConfig.AutoCreateMode publicationAutocreateMode
-
plugin
private final PostgresConnectorConfig.LogicalDecoder plugin
-
dropSlotOnClose
private final boolean dropSlotOnClose
-
originalConfig
private final PostgresConnectorConfig originalConfig
-
statusUpdateInterval
private final Duration statusUpdateInterval
-
messageDecoder
private final MessageDecoder messageDecoder
-
typeRegistry
private final TypeRegistry typeRegistry
-
streamParams
private final Properties streamParams
-
defaultStartingPos
private Lsn defaultStartingPos
-
slotCreationInfo
private SlotCreationResult slotCreationInfo
-
hasInitedSlot
private boolean hasInitedSlot
-
-
Constructor Detail
-
PostgresReplicationConnection
private PostgresReplicationConnection(PostgresConnectorConfig config, String slotName, String publicationName, RelationalTableFilters tableFilter, PostgresConnectorConfig.AutoCreateMode publicationAutocreateMode, PostgresConnectorConfig.LogicalDecoder plugin, boolean dropSlotOnClose, boolean doSnapshot, Duration statusUpdateInterval, TypeRegistry typeRegistry, Properties streamParams, PostgresSchema schema)
Creates a new replication connection with the given params.- Parameters:
config- the JDBC configuration for the connection; may not be nullslotName- the name of the DB slot for logical replication; may not be nullpublicationName- the name of the DB publication for logical replication; may not be nulltableFilter- the tables to watch of the DB publication for logical replication; may not be nullpublicationAutocreateMode- the mode for publication autocreation; may not be nullplugin- decoder matching the server side plug-in used for streaming changes; may not be nulldropSlotOnClose- whether the replication slot should be dropped once the connection is closedstatusUpdateInterval- the interval at which the replication connection should periodically send statusdoSnapshot- whether the connector is doing snapshottypeRegistry- registry with PostgreSQL typesstreamParams- additional parameters to pass to the replication streamschema- the schema; must not be nullupdates to the server
-
-
Method Detail
-
getSlotInfo
private ServerInfo.ReplicationSlot getSlotInfo() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
initPublication
protected void initPublication()
-
determineCapturedTables
private Set<TableId> determineCapturedTables() throws Exception
- Throws:
Exception
-
initReplicationSlot
protected void initReplicationSlot() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
useTemporarySlot
private boolean useTemporarySlot() throws SQLException- Throws:
SQLException
-
startStreaming
public ReplicationStream startStreaming(WalPositionLocator walPosition) throws SQLException, InterruptedException
creating a replication connection and starting to stream involves a few steps: 1. we create the connection and ensure that a. the slot exists b. the slot isn't currently being used 2. we query to get our potential start position in the slot (lsn) 3. we try and start streaming, depending on our options (such as in wal2json) this may fail, which can result in the connection being killed and we need to start the process over if we are using a temporary slot 4. actually start the streamerThis method takes care of all of these and this method queries for a default starting position If you know where you are starting from you should call
startStreaming(Lsn, WalPositionLocator), this method delegates to that method- Specified by:
startStreamingin interfaceReplicationConnection- Returns:
- Throws:
SQLExceptionInterruptedException
-
startStreaming
public ReplicationStream startStreaming(Lsn offset, WalPositionLocator walPosition) throws SQLException, InterruptedException
Description copied from interface:ReplicationConnectionOpens a stream for reading logical replication changes from a given LSN position.Note that it is possible for a server to have recycled old WAL segments (see the
wal_keep_segmentssetting). If that is the case, then even though a LSN number may be valid, the server will not stream back any changes because they are not available.- Specified by:
startStreamingin interfaceReplicationConnection- Parameters:
offset- a value representing the WAL sequence number where replication should start from; if the value isnullor negative, this behaves exactly like#startStreaming().- Returns:
- a
PGReplicationStreamfrom which data is read; never null - Throws:
SQLException- if anything failsInterruptedException- See Also:
LogSequenceNumber
-
initConnection
public void initConnection() throws SQLException, InterruptedExceptionDescription copied from interface:ReplicationConnectionForces the connection to be created, is called by startStreaming, but can be called manually in cases where we want the connection but not to to start streaming yet. Can be called multiple times- Specified by:
initConnectionin interfaceReplicationConnection- Throws:
SQLExceptionInterruptedException
-
createReplicationSlot
public Optional<SlotCreationResult> createReplicationSlot() throws SQLException
Description copied from interface:ReplicationConnectionCreates a new replication slot with the given option and returns the result of the command, which may contain results (depending on postgres versions)- Specified by:
createReplicationSlotin interfaceReplicationConnection- Throws:
SQLException- if anything fails
-
pgConnection
protected org.postgresql.core.BaseConnection pgConnection() throws SQLException- Throws:
SQLException
-
parseSlotCreation
private SlotCreationResult parseSlotCreation(ResultSet rs)
-
createReplicationStream
private ReplicationStream createReplicationStream(Lsn startLsn, WalPositionLocator walPosition) throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
startPgReplicationStream
private org.postgresql.replication.PGReplicationStream startPgReplicationStream(Lsn lsn, Function<org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder,org.postgresql.replication.fluent.logical.ChainedLogicalStreamBuilder> configurator) throws SQLException
- Throws:
SQLException
-
close
public void close()
- Specified by:
closein interfaceAutoCloseable- Overrides:
closein classJdbcConnection
-
close
public void close(boolean dropSlot)
-
reconnect
public void reconnect() throws SQLException- Specified by:
reconnectin interfaceReplicationConnection- Throws:
SQLException
-
defaultSettings
protected static void defaultSettings(Configuration.Builder builder)
-
-