Package io.debezium.connector.postgresql

Class PostgresValueConverter

java.lang.Object

io.debezium.jdbc.JdbcValueConverters

io.debezium.connector.postgresql.PostgresValueConverter

All Implemented Interfaces:

ValueConverterProvider

public class PostgresValueConverter
extends JdbcValueConverters

A provider of ValueConverters and SchemaBuilders for various Postgres specific column types. In addition to handling data type conversion from values coming from JDBC, this is also expected to handle data type conversion for data types coming from the logical decoding plugin.

Author:

Horia Chiorean (hchiorea@redhat.com)

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.jdbc.JdbcValueConverters

JdbcValueConverters.BigIntUnsignedMode, JdbcValueConverters.DecimalMode

Field Summary

Fields

Modifier and Type	Field	Description
private Charset	databaseCharset	The current database's character encoding.
private PostgresConnectorConfig.HStoreHandlingMode	hStoreMode
private boolean	includeUnknownDatatypes	true if fields of data type not know should be handle as opaque binary; false if they should be omitted
private PostgresConnectorConfig.IntervalHandlingMode	intervalMode
private com.fasterxml.jackson.core.JsonFactory	jsonFactory
private static BigDecimal	MICROSECONDS_PER_SECOND
static String	N_A_N	A string denoting not-a- number for FP and Numeric types
static String	NEGATIVE_INFINITY	A string denoting negative infinity for FP and Numeric types
static Instant	NEGATIVE_INFINITY_INSTANT
static LocalDateTime	NEGATIVE_INFINITY_LOCAL_DATE_TIME
static OffsetDateTime	NEGATIVE_INFINITY_OFFSET_DATE_TIME
static Timestamp	NEGATIVE_INFINITY_TIMESTAMP
static String	POSITIVE_INFINITY	A string denoting positive infinity for FP and Numeric types
static Instant	POSITIVE_INFINITY_INSTANT
static LocalDateTime	POSITIVE_INFINITY_LOCAL_DATE_TIME
static OffsetDateTime	POSITIVE_INFINITY_OFFSET_DATE_TIME
static Timestamp	POSITIVE_INFINITY_TIMESTAMP
private static DateTimeFormatter	TIME_WITH_TIMEZONE_FORMATTER	A formatter used to parse TIMETZ columns when provided as strings.
private byte[]	toastPlaceholderBinary
private String	toastPlaceholderString
private TypeRegistry	typeRegistry
private static int	VARIABLE_SCALE_DECIMAL_LENGTH	Variable scale decimal/numeric is defined by metadata scale - 0 length - 131089

Fields inherited from class io.debezium.jdbc.JdbcValueConverters

adaptiveTimeMicrosecondsPrecisionMode, adaptiveTimePrecisionMode, bigIntUnsignedMode, binaryMode, decimalMode, logger

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	PostgresValueConverter(Charset databaseCharset, JdbcValueConverters.DecimalMode decimalMode, TemporalPrecisionMode temporalPrecisionMode, ZoneOffset defaultOffset, JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode, boolean includeUnknownDatatypes, TypeRegistry typeRegistry, PostgresConnectorConfig.HStoreHandlingMode hStoreMode, CommonConnectorConfig.BinaryHandlingMode binaryMode, PostgresConnectorConfig.IntervalHandlingMode intervalMode, byte[] toastPlaceholder)

Method Summary

Modifier and Type	Method	Description
private String	asHstoreString(byte[] data)	Returns an Hstore field as string in the form of "key 1"=>"value1", "key_2"=>"val 1"; i.e.
private String	changePlainStringRepresentationToJsonStringRepresentation(String text)
protected Object	convertArray(Column column, org.apache.kafka.connect.data.Field fieldDefn, PostgresType elementType, ValueConverter elementConverter, Object data)
protected Object	convertBinaryToBase64(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertBinaryToBytes(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)	Extracts a value from a PGobject .
protected Object	convertBinaryToHex(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertBit(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertBits(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data, int numBytes)
protected Object	convertCitext(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertDecimal(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data, JdbcValueConverters.DecimalMode mode)
ValueConverter	converter(Column column, org.apache.kafka.connect.data.Field fieldDefn)
protected Object	convertGeography(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertGeometry(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertHStore(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data, PostgresConnectorConfig.HStoreHandlingMode mode)
private Object	convertHstoreToJsonString(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private Object	convertHstoreToMap(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertInterval(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private Object	convertLtree(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private Object	convertLtreeArray(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private String	convertMapToJsonStringRepresentation(Map<String,String> map)
protected Object	convertMoney(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertPoint(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)	Converts a value representing a Postgres point for a column, to a Kafka Connect value.
protected Object	convertString(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)	Replaces toasted value with a placeholder
protected Object	convertTime(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertTimestampToLocalDateTime(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertTimestampWithZone(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
protected Object	convertTimeWithZone(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private ValueConverter	createArrayConverter(Column column, org.apache.kafka.connect.data.Field fieldDefn)
protected int	getTimePrecision(Column column)
protected Object	handleUnknownData(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
private org.apache.kafka.connect.data.SchemaBuilder	hstoreSchema()
private boolean	isVariableScaleDecimal(Column column)
private org.apache.kafka.connect.data.SchemaBuilder	numericSchema(Column column)
static PostgresValueConverter	of(PostgresConnectorConfig connectorConfig, Charset databaseCharset, TypeRegistry typeRegistry)
private Object	resolveArrayValue(Object value, PostgresType elementType)
org.apache.kafka.connect.data.SchemaBuilder	schemaBuilder(Column column)
static Optional<SpecialValueDecimal>	toSpecialValue(String value)

Methods inherited from class io.debezium.jdbc.JdbcValueConverters

byteOrderOfBitType, convertBigInt, convertBinary, convertBits, convertBoolean, convertDateToEpochDays, convertDateToEpochDaysAsDate, convertDecimal, convertDouble, convertFloat, convertInteger, convertNumeric, convertReal, convertRowId, convertSmallInt, convertTimestampToEpochMicros, convertTimestampToEpochMillis, convertTimestampToEpochMillisAsDate, convertTimestampToEpochNanos, convertTimeToMicrosPastMidnight, convertTimeToMillisPastMidnight, convertTimeToMillisPastMidnightAsDate, convertTimeToNanosPastMidnight, convertTinyInt, convertValue, normalizeBinaryData, padLittleEndian, toBigDecimal, toByteBuffer, unexpectedBinary, withScaleAdjustedIfNeeded

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

POSITIVE_INFINITY_TIMESTAMP

public static final Timestamp POSITIVE_INFINITY_TIMESTAMP

POSITIVE_INFINITY_INSTANT

public static final Instant POSITIVE_INFINITY_INSTANT

POSITIVE_INFINITY_LOCAL_DATE_TIME

public static final LocalDateTime POSITIVE_INFINITY_LOCAL_DATE_TIME

POSITIVE_INFINITY_OFFSET_DATE_TIME

public static final OffsetDateTime POSITIVE_INFINITY_OFFSET_DATE_TIME

NEGATIVE_INFINITY_TIMESTAMP

public static final Timestamp NEGATIVE_INFINITY_TIMESTAMP

NEGATIVE_INFINITY_INSTANT

public static final Instant NEGATIVE_INFINITY_INSTANT

NEGATIVE_INFINITY_LOCAL_DATE_TIME

public static final LocalDateTime NEGATIVE_INFINITY_LOCAL_DATE_TIME

NEGATIVE_INFINITY_OFFSET_DATE_TIME

public static final OffsetDateTime NEGATIVE_INFINITY_OFFSET_DATE_TIME

VARIABLE_SCALE_DECIMAL_LENGTH

private static final int VARIABLE_SCALE_DECIMAL_LENGTH

Variable scale decimal/numeric is defined by metadata scale - 0 length - 131089

See Also:

Constant Field Values

N_A_N

public static final String N_A_N

A string denoting not-a- number for FP and Numeric types

See Also:

Constant Field Values

POSITIVE_INFINITY

public static final String POSITIVE_INFINITY

A string denoting positive infinity for FP and Numeric types

See Also:

Constant Field Values

NEGATIVE_INFINITY

public static final String NEGATIVE_INFINITY

A string denoting negative infinity for FP and Numeric types

See Also:

Constant Field Values

MICROSECONDS_PER_SECOND

private static final BigDecimal MICROSECONDS_PER_SECOND

TIME_WITH_TIMEZONE_FORMATTER

private static final DateTimeFormatter TIME_WITH_TIMEZONE_FORMATTER

A formatter used to parse TIMETZ columns when provided as strings.

includeUnknownDatatypes

private final boolean includeUnknownDatatypes

true if fields of data type not know should be handle as opaque binary; false if they should be omitted

typeRegistry

private final TypeRegistry typeRegistry

hStoreMode

private final PostgresConnectorConfig.HStoreHandlingMode hStoreMode

intervalMode

private final PostgresConnectorConfig.IntervalHandlingMode intervalMode

databaseCharset

private final Charset databaseCharset

The current database's character encoding.

jsonFactory

private final com.fasterxml.jackson.core.JsonFactory jsonFactory

toastPlaceholderString

private final String toastPlaceholderString

toastPlaceholderBinary

private final byte[] toastPlaceholderBinary

Constructor Detail

PostgresValueConverter

protected PostgresValueConverter(Charset databaseCharset,
                                 JdbcValueConverters.DecimalMode decimalMode,
                                 TemporalPrecisionMode temporalPrecisionMode,
                                 ZoneOffset defaultOffset,
                                 JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode,
                                 boolean includeUnknownDatatypes,
                                 TypeRegistry typeRegistry,
                                 PostgresConnectorConfig.HStoreHandlingMode hStoreMode,
                                 CommonConnectorConfig.BinaryHandlingMode binaryMode,
                                 PostgresConnectorConfig.IntervalHandlingMode intervalMode,
                                 byte[] toastPlaceholder)

Method Detail

of

public static PostgresValueConverter of(PostgresConnectorConfig connectorConfig,
                                        Charset databaseCharset,
                                        TypeRegistry typeRegistry)

schemaBuilder

public org.apache.kafka.connect.data.SchemaBuilder schemaBuilder(Column column)

Specified by:

schemaBuilder in interface ValueConverterProvider

Overrides:

schemaBuilder in class JdbcValueConverters

numericSchema

private org.apache.kafka.connect.data.SchemaBuilder numericSchema(Column column)

hstoreSchema

private org.apache.kafka.connect.data.SchemaBuilder hstoreSchema()

converter

public ValueConverter converter(Column column,
                                org.apache.kafka.connect.data.Field fieldDefn)

Specified by:

converter in interface ValueConverterProvider

Overrides:

converter in class JdbcValueConverters

createArrayConverter

private ValueConverter createArrayConverter(Column column,
                                            org.apache.kafka.connect.data.Field fieldDefn)

convertTime

protected Object convertTime(Column column,
                             org.apache.kafka.connect.data.Field fieldDefn,
                             Object data)

Overrides:

convertTime in class JdbcValueConverters

convertDecimal

protected Object convertDecimal(Column column,
                                org.apache.kafka.connect.data.Field fieldDefn,
                                Object data,
                                JdbcValueConverters.DecimalMode mode)

convertHStore

protected Object convertHStore(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data,
                               PostgresConnectorConfig.HStoreHandlingMode mode)

convertLtree

private Object convertLtree(Column column,
                            org.apache.kafka.connect.data.Field fieldDefn,
                            Object data)

convertLtreeArray

private Object convertLtreeArray(Column column,
                                 org.apache.kafka.connect.data.Field fieldDefn,
                                 Object data)

convertHstoreToMap

private Object convertHstoreToMap(Column column,
                                  org.apache.kafka.connect.data.Field fieldDefn,
                                  Object data)

asHstoreString

private String asHstoreString(byte[] data)

Returns an Hstore field as string in the form of "key 1"=>"value1", "key_2"=>"val 1"; i.e. the given byte array is NOT the byte representation returned by {@link HStoreConverter#toBytes(Map, org.postgresql.core.Encoding))}, but the String based representation

convertHstoreToJsonString

private Object convertHstoreToJsonString(Column column,
                                         org.apache.kafka.connect.data.Field fieldDefn,
                                         Object data)

changePlainStringRepresentationToJsonStringRepresentation

private String changePlainStringRepresentationToJsonStringRepresentation(String text)

convertMapToJsonStringRepresentation

private String convertMapToJsonStringRepresentation(Map<String,String> map)

convertBit

protected Object convertBit(Column column,
                            org.apache.kafka.connect.data.Field fieldDefn,
                            Object data)

Overrides:

convertBit in class JdbcValueConverters

convertBits

protected Object convertBits(Column column,
                             org.apache.kafka.connect.data.Field fieldDefn,
                             Object data,
                             int numBytes)

Overrides:

convertBits in class JdbcValueConverters

convertMoney

protected Object convertMoney(Column column,
                              org.apache.kafka.connect.data.Field fieldDefn,
                              Object data)

convertInterval

protected Object convertInterval(Column column,
                                 org.apache.kafka.connect.data.Field fieldDefn,
                                 Object data)

convertTimestampWithZone

protected Object convertTimestampWithZone(Column column,
                                          org.apache.kafka.connect.data.Field fieldDefn,
                                          Object data)

Overrides:

convertTimestampWithZone in class JdbcValueConverters

convertTimeWithZone

protected Object convertTimeWithZone(Column column,
                                     org.apache.kafka.connect.data.Field fieldDefn,
                                     Object data)

Overrides:

convertTimeWithZone in class JdbcValueConverters

convertGeometry

protected Object convertGeometry(Column column,
                                 org.apache.kafka.connect.data.Field fieldDefn,
                                 Object data)

convertGeography

protected Object convertGeography(Column column,
                                  org.apache.kafka.connect.data.Field fieldDefn,
                                  Object data)

convertCitext

protected Object convertCitext(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data)

convertPoint

protected Object convertPoint(Column column,
                              org.apache.kafka.connect.data.Field fieldDefn,
                              Object data)

Converts a value representing a Postgres point for a column, to a Kafka Connect value.

Parameters:

column - the JDBC column; never null

fieldDefn - the Connect field definition for this column; never null

data - a data for the point column, either coming from the JDBC driver or logical decoding plugin

Returns:

a value which will be used by Connect to represent the actual point value

convertArray

protected Object convertArray(Column column,
                              org.apache.kafka.connect.data.Field fieldDefn,
                              PostgresType elementType,
                              ValueConverter elementConverter,
                              Object data)

resolveArrayValue

private Object resolveArrayValue(Object value,
                                 PostgresType elementType)

isVariableScaleDecimal

private boolean isVariableScaleDecimal(Column column)

toSpecialValue

public static Optional<SpecialValueDecimal> toSpecialValue(String value)

convertTimestampToLocalDateTime

protected Object convertTimestampToLocalDateTime(Column column,
                                                 org.apache.kafka.connect.data.Field fieldDefn,
                                                 Object data)

getTimePrecision

protected int getTimePrecision(Column column)

Overrides:

getTimePrecision in class JdbcValueConverters

convertBinaryToBytes

protected Object convertBinaryToBytes(Column column,
                                      org.apache.kafka.connect.data.Field fieldDefn,
                                      Object data)

Extracts a value from a PGobject .

Overrides:

convertBinaryToBytes in class JdbcValueConverters

Parameters:

column - the column definition describing the data value; never null

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect type

Returns:

the converted value, or null if the conversion could not be made and the column allows nulls

Throws:

IllegalArgumentException - if the value could not be converted but the column does not allow nulls

convertBinaryToBase64

protected Object convertBinaryToBase64(Column column,
                                       org.apache.kafka.connect.data.Field fieldDefn,
                                       Object data)

Overrides:

convertBinaryToBase64 in class JdbcValueConverters

convertBinaryToHex

protected Object convertBinaryToHex(Column column,
                                    org.apache.kafka.connect.data.Field fieldDefn,
                                    Object data)

Overrides:

convertBinaryToHex in class JdbcValueConverters

convertString

protected Object convertString(Column column,
                               org.apache.kafka.connect.data.Field fieldDefn,
                               Object data)

Replaces toasted value with a placeholder

Overrides:

convertString in class JdbcValueConverters

Parameters:

column - the column definition describing the data value; never null

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect type

Returns:

the converted value, or null if the conversion could not be made and the column allows nulls

Throws:

IllegalArgumentException - if the value could not be converted but the column does not allow nulls

handleUnknownData

protected Object handleUnknownData(Column column,
                                   org.apache.kafka.connect.data.Field fieldDefn,
                                   Object data)

Overrides:

handleUnknownData in class JdbcValueConverters