net.snowflake.ingest.streaming

Interface SnowflakeStreamingIngestChannel

public interface SnowflakeStreamingIngestChannel

A logical partition that represents a connection to a single Snowflake table, data will be ingested into the channel, and then flushed to Snowflake table periodically in the background.

Channels are identified by their name and only one channel with the same name may ingest data at the same time. When a new channel is opened, all previously opened channels with the same name are invalidated (this applies for the table globally. not just in a single JVM). In order to ingest data from multiple threads/clients/applications, we recommend opening multiple channels, each with a different name. There is no limit on the number of channels that can be opened.

Thread safety note: Implementations of this interface are required to be thread safe.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Void>	close() Close the channel, this function will make sure all the data in this channel is committed
String	getDBName() Get the database name
String	getFullyQualifiedName() Get the fully qualified channel name
String	getFullyQualifiedTableName() Get the fully qualified table name that the channel belongs to
String	getLatestCommittedOffsetToken() Get the latest committed offset token from Snowflake
String	getName() Get the name of the channel
String	getSchemaName() Get the schema name
String	getTableName() Get the table name
InsertValidationResponse	insertRow(Map<String,Object> row, String offsetToken) Insert one row into the channel, the row is represented using Map where the key is column name and the value is a row of data.
InsertValidationResponse	insertRows(Iterable<Map<String,Object>> rows, String offsetToken) Insert a batch of rows into the channel, each row is represented using Map where the key is column name and the value is a row of data.
boolean	isClosed()
boolean	isValid()

Method Detail

getFullyQualifiedName

String getFullyQualifiedName()

Get the fully qualified channel name

Returns:

fully qualified name of the channel, in the format of dbName.schemaName.tableName.channelName

getName

String getName()

Get the name of the channel

Returns:

name of the channel

getDBName

String getDBName()

Get the database name

Returns:

name of the database

getSchemaName

String getSchemaName()

Get the schema name

Returns:

name of the schema

getTableName

String getTableName()

Get the table name

Returns:

name of the table

getFullyQualifiedTableName

String getFullyQualifiedTableName()

Get the fully qualified table name that the channel belongs to

Returns:

fully qualified table name, in the format of dbName.schemaName.tableName

isValid

boolean isValid()

Returns:

a boolean which indicates whether the channel is valid

isClosed

boolean isClosed()

Returns:

a boolean which indicates whether the channel is closed

close

CompletableFuture<Void> close()

Close the channel, this function will make sure all the data in this channel is committed

Returns:

a completable future which will be completed when the channel is closed

insertRow

InsertValidationResponse insertRow(Map<String,Object> row,
                                   @Nullable
                                   String offsetToken)

Insert one row into the channel, the row is represented using Map where the key is column name and the value is a row of data. The following table summarizes supported value types and their formats:

Snowflake Column Type	Allowed Java Data Type
CHAR, VARCHAR	String primitive data types (int, boolean, char, …)
BINARY	byte[] String (hex-encoded)
NUMBER, FLOAT	numeric types (BigInteger, BigDecimal, byte, int, double, …) String
BOOLEAN	boolean numeric types (BigInteger, BigDecimal, byte, int, double, …) String See boolean conversion details.
TIME	LocalTime OffsetTime String (in one of the following formats): DateTimeFormatter.ISO_LOCAL_TIME DateTimeFormatter.ISO_OFFSET_TIME Integer-stored time (see Snowflake Docs for more details)
DATE	LocalDate LocalDateTime OffsetDateTime ZonedDateTime Instant String (in one of the following formats): DateTimeFormatter.ISO_LOCAL_DATE DateTimeFormatter.ISO_LOCAL_DATE_TIME DateTimeFormatter.ISO_OFFSET_DATE_TIME DateTimeFormatter.ISO_ZONED_DATE_TIME Integer-stored date (see Snowflake Docs for more details)
TIMESTAMP_NTZ, TIMESTAMP_LTZ, TIMESTAMP_TZ	LocalDate LocalDateTime OffsetDateTime ZonedDateTime Instant String (in one of the following formats): DateTimeFormatter.ISO_LOCAL_DATE DateTimeFormatter.ISO_LOCAL_DATE_TIME DateTimeFormatter.ISO_OFFSET_DATE_TIME DateTimeFormatter.ISO_ZONED_DATE_TIME Integer-stored timestamp (see Snowflake Docs for more details) For TIMESTAMP_LTZ and TIMESTAMP_TZ, all input without timezone will be by default interpreted in the timezone "America/Los_Angeles". This can be changed by calling OpenChannelRequest.OpenChannelRequestBuilder.setDefaultTimezone(ZoneId).
VARIANT, ARRAY	String (must be a valid JSON value) primitive data types and their arrays BigInteger, BigDecimal LocalDate LocalDateTime OffsetDateTime ZonedDateTime Map<String, T> where T is a valid VARIANT type T[] where T is a valid VARIANT type List<T> where T is a valid VARIANT type
OBJECT	String (must be a valid JSON object) Map<String, T> where T is a valid variant type
GEOGRAPHY, GEOMETRY	Not supported

Parameters:

row - object data to write. For predictable results, we recommend not to concurrently modify the input row data.

offsetToken - offset of given row, used for replay in case of failures. It could be null if you don't plan on replaying or can't replay

Returns:

insert response that possibly contains errors because of insertion failures

insertRows

InsertValidationResponse insertRows(Iterable<Map<String,Object>> rows,
                                    @Nullable
                                    String offsetToken)

Insert a batch of rows into the channel, each row is represented using Map where the key is column name and the value is a row of data. See insertRow(Map, String) for more information about accepted values.

Parameters:

rows - object data to write

offsetToken - offset of last row in the row-set, used for replay in case of failures, It could be null if you don't plan on replaying or can't replay

Returns:

insert response that possibly contains errors because of insertion failures

getLatestCommittedOffsetToken

String getLatestCommittedOffsetToken()

Get the latest committed offset token from Snowflake

Returns:

the latest committed offset token