package cdc
Ordering
- Alphabetic
Visibility
- Public
- All
Type Members
- trait CDCReaderImpl extends DeltaLogging
Value Members
-
object
CDCReader extends CDCReaderImpl
The API that allows reading Change data between two versions of a table.
The API that allows reading Change data between two versions of a table.
The basic abstraction here is the CDC type column defined by CDCReader.CDC_TYPE_COLUMN_NAME. When CDC is enabled, our writer will treat this column as a special partition column even though it's not part of the table. Writers should generate a query that has two types of rows in it: the main data in partition CDC_TYPE_NOT_CDC and the CDC data with the appropriate CDC type value.
org.apache.spark.sql.delta.files.DelayedCommitProtocol does special handling for this column, dispatching the main data to its normal location while the CDC data is sent to AddCDCFile entries.