Class AbstractLogMinerEventProcessor<T extends AbstractTransaction>
- java.lang.Object
-
- io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor<T>
-
- All Implemented Interfaces:
LogMinerEventProcessor,AutoCloseable
- Direct Known Subclasses:
InfinispanLogMinerEventProcessor,MemoryLogMinerEventProcessor
public abstract class AbstractLogMinerEventProcessor<T extends AbstractTransaction> extends Object implements LogMinerEventProcessor
An abstract implementation ofLogMinerEventProcessorthat all processors should extend.- Author:
- Chris Cranford
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected classAbstractLogMinerEventProcessor.CountersWrapper for all counter variables
-
Field Summary
Fields Modifier and Type Field Description private OracleConnectorConfigconnectorConfigprivate ChangeEventSource.ChangeEventSourceContextcontextprotected AbstractLogMinerEventProcessor.Counterscountersprivate EventDispatcher<TableId>dispatcherprivate LogMinerDmlParserdmlParserprivate ScnlastProcessedScnprivate static org.slf4j.LoggerLOGGERprivate OracleStreamingChangeEventSourceMetricsmetricsprivate OracleOffsetContextoffsetContextprivate OraclePartitionpartitionprivate OracleDatabaseSchemaschemaprivate SelectLobParserselectLobParser
-
Constructor Summary
Constructors Constructor Description AbstractLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context, OracleConnectorConfig connectorConfig, OracleDatabaseSchema schema, OraclePartition partition, OracleOffsetContext offsetContext, EventDispatcher<TableId> dispatcher, OracleStreamingChangeEventSourceMetrics metrics)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected abstract voidaddToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)Add a transaction to the transaction map if allowed.protected abstract TcreateTransaction(LogMinerEventRow row)Creates a new transaction based on the suppliedSTARTevent.private TabledispatchSchemaChangeEventAndGetTableForNewCapturedTable(TableId tableId, OracleOffsetContext offsetContext, EventDispatcher<TableId> dispatcher)Dispatch a schema change event for a new table and get the newly created relational table model.protected OracleConnectorConfiggetConfig()protected ScngetLastProcessedScn()Return the last processed system change number handled by the processor.protected OracleDatabaseSchemagetSchema()private TablegetTableForDataEvent(LogMinerEventRow row)private StringgetTableMetadataDdl(TableId tableId)Get the specified table's create DDL statement.protected abstract TransactionCache<T,?>getTransactionCache()Returns theTransactionCacheimplementation.protected abstract intgetTransactionEventCount(T transaction)Returns the number of events associated with the specified transaction.protected abstract voidhandleCommit(LogMinerEventRow row)Handle processing a LogMinerEventRow for aCOMMITevent.protected voidhandleDataEvent(LogMinerEventRow row)Handle processing a LogMinerEventRow for aINSERT,UPDATE, orDELETEevent.private voidhandleLobErase(LogMinerEventRow row)Handle processing a LogMinerEventRow for aLOB_ERASEevent.protected voidhandleLobWrite(LogMinerEventRow row)Handle processing a LogMinerEventRow for aLOB_WRITEevent.protected voidhandleMissingScn(LogMinerEventRow row)Handle processing a LogMinerEventRow for aMISSING_SCNevent.protected voidhandleRollback(LogMinerEventRow row)Handle processing a LogMinerEventRow for aROLLBACKevent.protected voidhandleSchemaChange(LogMinerEventRow row)Handle processing a LogMinerEventRow for aDDLevent.protected voidhandleSelectLobLocator(LogMinerEventRow row)Handle processing a LogMinerEventRow for aSEL_LOB_LOCATORevent.protected voidhandleStart(LogMinerEventRow row)Handle processing a LogMinerEventRow for aSTARTevent.private booleanhasNextWithMetricsUpdate(ResultSet resultSet)Checks whether the result-set has any more data available.protected booleanhasSchemaChangeBeenSeen(LogMinerEventRow row)Checks whether the LogMinerEvent row for a schema change can be emitted.protected booleanisRecentlyCommitted(String transactionId)Check whether a transaction has been recently committed.protected booleanisTransactionIdAllowed(String transactionId)Return whether a give transaction can be added to the processor's buffer.protected booleanisTrxIdRawValue()private LogMinerDmlEntryparseDmlStatement(String redoSql, Table table, String transactionId)Parse a DML redo SQL statement.private StringparseLobWriteSql(String sql)Parses aLOB_WRITEoperation SQL fragment.protected voidprocessResults(ResultSet resultSet)Processes the LogMiner results.protected voidprocessRow(LogMinerEventRow row)Processes a single LogMinerEventRow.protected abstract voidremoveEventWithRowId(LogMinerEventRow row)Removes a specific transaction event by database row identifier.protected voidwarnPotentiallyStuckScn(Scn previousOffsetScn, Scn previousOffsetCommitScn)Checks to see whether the offset'sscnis remaining the same across multiple mining sessions while the offset'scommit_scnis changing between sessions.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface java.lang.AutoCloseable
close
-
Methods inherited from interface io.debezium.connector.oracle.logminer.processor.LogMinerEventProcessor
abandonTransactions, process
-
-
-
-
Field Detail
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
context
private final ChangeEventSource.ChangeEventSourceContext context
-
connectorConfig
private final OracleConnectorConfig connectorConfig
-
schema
private final OracleDatabaseSchema schema
-
partition
private final OraclePartition partition
-
offsetContext
private final OracleOffsetContext offsetContext
-
dispatcher
private final EventDispatcher<TableId> dispatcher
-
metrics
private final OracleStreamingChangeEventSourceMetrics metrics
-
dmlParser
private final LogMinerDmlParser dmlParser
-
selectLobParser
private final SelectLobParser selectLobParser
-
counters
protected final AbstractLogMinerEventProcessor.Counters counters
-
lastProcessedScn
private Scn lastProcessedScn
-
-
Constructor Detail
-
AbstractLogMinerEventProcessor
public AbstractLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context, OracleConnectorConfig connectorConfig, OracleDatabaseSchema schema, OraclePartition partition, OracleOffsetContext offsetContext, EventDispatcher<TableId> dispatcher, OracleStreamingChangeEventSourceMetrics metrics)
-
-
Method Detail
-
getConfig
protected OracleConnectorConfig getConfig()
-
getSchema
protected OracleDatabaseSchema getSchema()
-
isRecentlyCommitted
protected boolean isRecentlyCommitted(String transactionId)
Check whether a transaction has been recently committed. Any implementation that does not support recently-committed tracking should return false.- Parameters:
transactionId- the unique transaction id- Returns:
- true if the transaction has been recently committed, false otherwise
-
hasSchemaChangeBeenSeen
protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)
Checks whether the LogMinerEvent row for a schema change can be emitted.- Parameters:
row- the result set row- Returns:
- true if the schema change has been seen, false otherwise.
-
isTransactionIdAllowed
protected boolean isTransactionIdAllowed(String transactionId)
Return whether a give transaction can be added to the processor's buffer. The default implementation is to allow all transaction ids.- Parameters:
transactionId- the unique transaction id- Returns:
- whether a transaction id can be added to the processor's buffer
-
getLastProcessedScn
protected Scn getLastProcessedScn()
Return the last processed system change number handled by the processor.- Returns:
- the last processed system change number, never
null.
-
getTransactionCache
protected abstract TransactionCache<T,?> getTransactionCache()
Returns theTransactionCacheimplementation.- Returns:
- the transaction cache, never
null
-
createTransaction
protected abstract T createTransaction(LogMinerEventRow row)
Creates a new transaction based on the suppliedSTARTevent.- Parameters:
row- the event row, must not benull- Returns:
- the implementation-specific
Transactioninstance
-
removeEventWithRowId
protected abstract void removeEventWithRowId(LogMinerEventRow row)
Removes a specific transaction event by database row identifier.- Parameters:
row- the event row that contains the row identifier, must not benull
-
getTransactionEventCount
protected abstract int getTransactionEventCount(T transaction)
Returns the number of events associated with the specified transaction.- Parameters:
transaction- the transaction, must not benull- Returns:
- the number of events in the transaction
-
isTrxIdRawValue
protected boolean isTrxIdRawValue()
-
processResults
protected void processResults(ResultSet resultSet) throws SQLException, InterruptedException
Processes the LogMiner results.- Parameters:
resultSet- the result set from a LogMiner query- Throws:
SQLException- if a database exception occurredInterruptedException- if the dispatcher was interrupted sending an event
-
processRow
protected void processRow(LogMinerEventRow row) throws SQLException, InterruptedException
Processes a single LogMinerEventRow.- Parameters:
row- the event row, must not benull- Throws:
SQLException- if a database exception occurredInterruptedException- if the dispatcher was interrupted sending an event
-
handleMissingScn
protected void handleMissingScn(LogMinerEventRow row)
Handle processing a LogMinerEventRow for aMISSING_SCNevent.- Parameters:
row- the result set row
-
handleStart
protected void handleStart(LogMinerEventRow row)
Handle processing a LogMinerEventRow for aSTARTevent.- Parameters:
row- the result set row
-
handleCommit
protected abstract void handleCommit(LogMinerEventRow row) throws InterruptedException
Handle processing a LogMinerEventRow for aCOMMITevent.- Parameters:
row- the result set row- Throws:
InterruptedException- if the event dispatcher was interrupted sending events
-
handleRollback
protected void handleRollback(LogMinerEventRow row)
Handle processing a LogMinerEventRow for aROLLBACKevent. The default implementation is a no-op.- Parameters:
row- the result set row
-
handleSchemaChange
protected void handleSchemaChange(LogMinerEventRow row) throws InterruptedException
Handle processing a LogMinerEventRow for aDDLevent.- Parameters:
row- the result set row- Throws:
InterruptedException- if the event dispatcher is interrupted sending the event
-
handleSelectLobLocator
protected void handleSelectLobLocator(LogMinerEventRow row)
Handle processing a LogMinerEventRow for aSEL_LOB_LOCATORevent.- Parameters:
row- the result set row
-
handleLobWrite
protected void handleLobWrite(LogMinerEventRow row)
Handle processing a LogMinerEventRow for aLOB_WRITEevent.- Parameters:
row- the result set row
-
handleLobErase
private void handleLobErase(LogMinerEventRow row)
Handle processing a LogMinerEventRow for aLOB_ERASEevent.- Parameters:
row- the result set row
-
handleDataEvent
protected void handleDataEvent(LogMinerEventRow row) throws SQLException, InterruptedException
Handle processing a LogMinerEventRow for aINSERT,UPDATE, orDELETEevent.- Parameters:
row- the result set row- Throws:
SQLException- if a database exception occursInterruptedException- if the dispatch of an event is interrupted
-
warnPotentiallyStuckScn
protected void warnPotentiallyStuckScn(Scn previousOffsetScn, Scn previousOffsetCommitScn)
Checks to see whether the offset'sscnis remaining the same across multiple mining sessions while the offset'scommit_scnis changing between sessions.- Parameters:
previousOffsetScn- the previous offset system change numberpreviousOffsetCommitScn- the previous offset commit system change number
-
getTableForDataEvent
private Table getTableForDataEvent(LogMinerEventRow row) throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
hasNextWithMetricsUpdate
private boolean hasNextWithMetricsUpdate(ResultSet resultSet) throws SQLException
Checks whether the result-set has any more data available. When a new row is available, the streaming metrics is updated with the fetch timings.- Parameters:
resultSet- the result set to check if any more rows exist- Returns:
- true if another row exists, false otherwise
- Throws:
SQLException- if there was a database exception
-
addToTransaction
protected abstract void addToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)
Add a transaction to the transaction map if allowed.- Parameters:
transactionId- the unqiue transaction idrow- the LogMiner event roweventSupplier- the supplier of the event to create if the event is allowed to be added
-
dispatchSchemaChangeEventAndGetTableForNewCapturedTable
private Table dispatchSchemaChangeEventAndGetTableForNewCapturedTable(TableId tableId, OracleOffsetContext offsetContext, EventDispatcher<TableId> dispatcher) throws SQLException, InterruptedException
Dispatch a schema change event for a new table and get the newly created relational table model.- Parameters:
tableId- the unique table identifier, must not benulloffsetContext- the offset contextdispatcher- the event dispatcher- Returns:
- the relational table model
- Throws:
SQLException- if a database exception occurredInterruptedException- if the event dispatch was interrupted
-
getTableMetadataDdl
private String getTableMetadataDdl(TableId tableId) throws SQLException
Get the specified table's create DDL statement.- Parameters:
tableId- the table identifier, must not benull- Returns:
- the table's create DDL statement, never
null - Throws:
SQLException- if an exception occurred obtaining the DDL statement
-
parseDmlStatement
private LogMinerDmlEntry parseDmlStatement(String redoSql, Table table, String transactionId)
Parse a DML redo SQL statement.- Parameters:
redoSql- the redo SQL statementtable- the table the SQL statement is fortransactionId- the associated transaction id for the SQL statement- Returns:
- a parse object for the redo SQL statement
-
-