io.debezium.connector.postgresql

Class PostgresValueConverter

java.lang.Object

io.debezium.jdbc.JdbcValueConverters

io.debezium.connector.postgresql.PostgresValueConverter

All Implemented Interfaces:

ValueConverterProvider

public class PostgresValueConverter
extends JdbcValueConverters

A provider of ValueConverters and SchemaBuilders for various Postgres specific column types. In addition to handling data type conversion from values coming from JDBC, this is also expected to handle data type conversion for data types coming from the logical decoding plugin.

Author:

Horia Chiorean (hchiorea@redhat.com)

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.jdbc.JdbcValueConverters

JdbcValueConverters.BigIntUnsignedMode, JdbcValueConverters.DecimalMode

Field Summary

Fields

Modifier and Type	Field and Description
private Charset	databaseCharset The current database's character encoding.
private PostgresConnectorConfig.HStoreHandlingMode	hStoreMode
private boolean	includeUnknownDatatypes true if fields of data type not know should be handle as opaque binary; false if they should be omitted
private com.fasterxml.jackson.core.JsonFactory	jsonFactory
static String	N_A_N A string denoting not-a- number for FP and Numeric types
private static long	NANO_SECONDS_PER_DAY
static String	NEGATIVE_INFINITY A string denoting negative infinity for FP and Numeric types
private static Duration	ONE_DAY
static String	POSITIVE_INFINITY A string denoting positive infinity for FP and Numeric types
private static DateTimeFormatter	TIME_WITH_TIMEZONE_FORMATTER A formatter used to parse TIMETZ columns when provided as strings.
private byte[]	toastPlaceholderBinary
private String	toastPlaceholderString
private TypeRegistry	typeRegistry
private static int	VARIABLE_SCALE_DECIMAL_LENGTH Variable scale decimal/numeric is defined by metadata scale - 0 length - 131089

Fields inherited from class io.debezium.jdbc.JdbcValueConverters

adaptiveTimeMicrosecondsPrecisionMode, adaptiveTimePrecisionMode, bigIntUnsignedMode, decimalMode, logger

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	PostgresValueConverter(Charset databaseCharset, JdbcValueConverters.DecimalMode decimalMode, TemporalPrecisionMode temporalPrecisionMode, ZoneOffset defaultOffset, JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode, boolean includeUnknownDatatypes, TypeRegistry typeRegistry, PostgresConnectorConfig.HStoreHandlingMode hStoreMode, byte[] toastPlaceholder)

Method Summary

Modifier and Type	Method and Description
private String	asHstoreString(byte[] data) Returns an Hstore field as string in the form of "key 1"=>"value1", "key_2"=>"val 1"; i.e.
private String	changePlainStringRepresentationToJsonStringRepresentation(String text)
protected Object	convertArray(Column column, org.apache.kafka.connect.data.Field fieldDefn, ValueConverter elementConverter, Object data)
protected Object	convertBinary(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data) Extracts a value from a PGobject .
protected Object	convertBit(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertBits(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data, int numBytes)
protected Object	convertCitext(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertDecimal(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data, JdbcValueConverters.DecimalMode mode)
ValueConverter	converter(Column column, org.apache.kafka.connect.data.Field fieldDefn)
protected Object	convertGeography(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertGeometry(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertHStore(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data, PostgresConnectorConfig.HStoreHandlingMode mode)
private Object	convertHstoreToJsonString(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private Object	convertHstoreToMap(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertInterval(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private String	convertMapToJsonStringRepresentation(Map<String,String> map)
protected Object	convertMoney(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertPoint(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data) Converts a value representing a Postgres point for a column, to a Kafka Connect value.
protected Object	convertString(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data) Replaces toasted value with a placeholder
protected Object	convertTime(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertTimestampToLocalDateTime(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertTimestampWithZone(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertTimeWithZone(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private ValueConverter	createArrayConverter(Column column, org.apache.kafka.connect.data.Field fieldDefn)
protected int	getTimePrecision(Column column)
private org.apache.kafka.connect.data.SchemaBuilder	hstoreSchema()
private boolean	isVariableScaleDecimal(Column column)
private org.apache.kafka.connect.data.SchemaBuilder	numericSchema(Column column)
org.apache.kafka.connect.data.SchemaBuilder	schemaBuilder(Column column)
static Optional<SpecialValueDecimal>	toSpecialValue(String value)

Methods inherited from class io.debezium.jdbc.JdbcValueConverters

byteOrderOfBitType, convertBigInt, convertBits, convertBoolean, convertByteArray, convertDateToEpochDays, convertDateToEpochDaysAsDate, convertDecimal, convertDouble, convertFloat, convertInteger, convertNumeric, convertReal, convertRowId, convertSmallInt, convertTimestampToEpochMicros, convertTimestampToEpochMillis, convertTimestampToEpochMillisAsDate, convertTimestampToEpochNanos, convertTimeToMicrosPastMidnight, convertTimeToMillisPastMidnight, convertTimeToMillisPastMidnightAsDate, convertTimeToNanosPastMidnight, convertTinyInt, convertValue, handleUnknownData, padLittleEndian, toBigDecimal, unexpectedBinary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VARIABLE_SCALE_DECIMAL_LENGTH

private static final int VARIABLE_SCALE_DECIMAL_LENGTH

Variable scale decimal/numeric is defined by metadata scale - 0 length - 131089

See Also:

Constant Field Values

N_A_N

public static final String N_A_N

A string denoting not-a- number for FP and Numeric types

See Also:

Constant Field Values

POSITIVE_INFINITY

public static final String POSITIVE_INFINITY

A string denoting positive infinity for FP and Numeric types

See Also:

Constant Field Values

NEGATIVE_INFINITY

public static final String NEGATIVE_INFINITY

A string denoting negative infinity for FP and Numeric types

See Also:

Constant Field Values

TIME_WITH_TIMEZONE_FORMATTER

private static final DateTimeFormatter TIME_WITH_TIMEZONE_FORMATTER

A formatter used to parse TIMETZ columns when provided as strings.

ONE_DAY

private static final Duration ONE_DAY

NANO_SECONDS_PER_DAY

private static final long NANO_SECONDS_PER_DAY

includeUnknownDatatypes

private final boolean includeUnknownDatatypes

true if fields of data type not know should be handle as opaque binary; false if they should be omitted

typeRegistry

private final TypeRegistry typeRegistry

hStoreMode

private final PostgresConnectorConfig.HStoreHandlingMode hStoreMode

databaseCharset

private final Charset databaseCharset

The current database's character encoding.

jsonFactory

private final com.fasterxml.jackson.core.JsonFactory jsonFactory

toastPlaceholderString

private final String toastPlaceholderString

toastPlaceholderBinary

private final byte[] toastPlaceholderBinary

Constructor Detail

PostgresValueConverter

protected PostgresValueConverter(Charset databaseCharset,
                                 JdbcValueConverters.DecimalMode decimalMode,
                                 TemporalPrecisionMode temporalPrecisionMode,
                                 ZoneOffset defaultOffset,
                                 JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode,
                                 boolean includeUnknownDatatypes,
                                 TypeRegistry typeRegistry,
                                 PostgresConnectorConfig.HStoreHandlingMode hStoreMode,
                                 byte[] toastPlaceholder)

Method Detail

schemaBuilder

public org.apache.kafka.connect.data.SchemaBuilder schemaBuilder(Column column)

Specified by:

schemaBuilder in interface ValueConverterProvider

Overrides:

schemaBuilder in class JdbcValueConverters

numericSchema

private org.apache.kafka.connect.data.SchemaBuilder numericSchema(Column column)

hstoreSchema

private org.apache.kafka.connect.data.SchemaBuilder hstoreSchema()

converter

public ValueConverter converter(Column column,
                                org.apache.kafka.connect.data.Field fieldDefn)

Specified by:

converter in interface ValueConverterProvider

Overrides:

converter in class JdbcValueConverters

createArrayConverter

private ValueConverter createArrayConverter(Column column,
                                            org.apache.kafka.connect.data.Field fieldDefn)

convertTime

protected Object convertTime(Column column,
                             org.apache.kafka.connect.data.Field fieldDefn,
                             Object data)

Overrides:

convertTime in class JdbcValueConverters

convertDecimal

protected Object convertDecimal(Column column,
                                org.apache.kafka.connect.data.Field fieldDefn,
                                Object data,
                                JdbcValueConverters.DecimalMode mode)

convertHStore

protected Object convertHStore(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data,
                               PostgresConnectorConfig.HStoreHandlingMode mode)

convertHstoreToMap

private Object convertHstoreToMap(Column column,
                                  org.apache.kafka.connect.data.Field fieldDefn,
                                  Object data)

asHstoreString

private String asHstoreString(byte[] data)

Returns an Hstore field as string in the form of "key 1"=>"value1", "key_2"=>"val 1"; i.e. the given byte array is NOT the byte representation returned by HStoreConverter#toBytes(Map, org.postgresql.core.Encoding)), but the String based representation

convertHstoreToJsonString

private Object convertHstoreToJsonString(Column column,
                                         org.apache.kafka.connect.data.Field fieldDefn,
                                         Object data)

changePlainStringRepresentationToJsonStringRepresentation

private String changePlainStringRepresentationToJsonStringRepresentation(String text)

convertMapToJsonStringRepresentation

private String convertMapToJsonStringRepresentation(Map<String,String> map)

convertBit

protected Object convertBit(Column column,
                            org.apache.kafka.connect.data.Field fieldDefn,
                            Object data)

Overrides:

convertBit in class JdbcValueConverters

convertBits

protected Object convertBits(Column column,
                             org.apache.kafka.connect.data.Field fieldDefn,
                             Object data,
                             int numBytes)

Overrides:

convertBits in class JdbcValueConverters

convertMoney

protected Object convertMoney(Column column,
                              org.apache.kafka.connect.data.Field fieldDefn,
                              Object data)

convertInterval

protected Object convertInterval(Column column,
                                 org.apache.kafka.connect.data.Field fieldDefn,
                                 Object data)

convertTimestampWithZone

protected Object convertTimestampWithZone(Column column,
                                          org.apache.kafka.connect.data.Field fieldDefn,
                                          Object data)

Overrides:

convertTimestampWithZone in class JdbcValueConverters

convertTimeWithZone

protected Object convertTimeWithZone(Column column,
                                     org.apache.kafka.connect.data.Field fieldDefn,
                                     Object data)

Overrides:

convertTimeWithZone in class JdbcValueConverters

convertGeometry

protected Object convertGeometry(Column column,
                                 org.apache.kafka.connect.data.Field fieldDefn,
                                 Object data)

convertGeography

protected Object convertGeography(Column column,
                                  org.apache.kafka.connect.data.Field fieldDefn,
                                  Object data)

convertCitext

protected Object convertCitext(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data)

convertPoint

protected Object convertPoint(Column column,
                              org.apache.kafka.connect.data.Field fieldDefn,
                              Object data)

Converts a value representing a Postgres point for a column, to a Kafka Connect value.

Parameters:

column - the JDBC column; never null

fieldDefn - the Connect field definition for this column; never null

data - a data for the point column, either coming from the JDBC driver or logical decoding plugin

Returns:

a value which will be used by Connect to represent the actual point value

convertArray

protected Object convertArray(Column column,
                              org.apache.kafka.connect.data.Field fieldDefn,
                              ValueConverter elementConverter,
                              Object data)

isVariableScaleDecimal

private boolean isVariableScaleDecimal(Column column)

toSpecialValue

public static Optional<SpecialValueDecimal> toSpecialValue(String value)

convertTimestampToLocalDateTime

protected Object convertTimestampToLocalDateTime(Column column,
                                                 org.apache.kafka.connect.data.Field fieldDefn,
                                                 Object data)

getTimePrecision

protected int getTimePrecision(Column column)

Overrides:

getTimePrecision in class JdbcValueConverters

convertBinary

protected Object convertBinary(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data)

Extracts a value from a PGobject .

Overrides:

convertBinary in class JdbcValueConverters

Parameters:

column - the column definition describing the data value; never null

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect type

Returns:

the converted value, or null if the conversion could not be made and the column allows nulls

Throws:

IllegalArgumentException - if the value could not be converted but the column does not allow nulls

convertString

protected Object convertString(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data)

Replaces toasted value with a placeholder

Overrides:

convertString in class JdbcValueConverters

Parameters:

column - the column definition describing the data value; never null

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect type

Returns:

the converted value, or null if the conversion could not be made and the column allows nulls

Throws:

IllegalArgumentException - if the value could not be converted but the column does not allow nulls