Package net.snowflake.ingest.streaming.internal

Class SnowflakeStreamingIngestClientInternal<T>

java.lang.Object

net.snowflake.ingest.streaming.internal.SnowflakeStreamingIngestClientInternal<T>

Type Parameters:

T - type of column data (ParquetChunkData)

All Implemented Interfaces:

AutoCloseable, SnowflakeStreamingIngestClient

public class SnowflakeStreamingIngestClientInternal<T>
extends Object
implements SnowflakeStreamingIngestClient

The first version of implementation for SnowflakeStreamingIngestClient. The client internally manages a few things:

the channel cache, which contains all the channels that belong to this account the flush service, which schedules and coordinates the flush to Snowflake tables

Constructor Summary

Constructors

Constructor	Description
SnowflakeStreamingIngestClientInternal(String name, SnowflakeURL accountURL, Properties prop, Map<String,Object> parameterOverrides, boolean isIcebergMode, boolean isTestMode)	Default Constructor

Method Summary

Modifier and Type	Method	Description
void	close()	Close the client, which will flush first and then release all the resources
void	dropChannel(DropChannelRequest request)	Drop the specified channel on the server using a DropChannelRequest
Map<String,String>	getLatestCommittedOffsetTokens(List<SnowflakeStreamingIngestChannel> channels)	Return the latest committed/persisted offset token for all channels
String	getName()	Get the client name
void	injectRequestBuilder(RequestBuilder requestBuilder)
boolean	isClosed()	Check whether the client is closed or not, if you want to make sure all data are committed before closing, please call AutoCloseable.close() before closing the entire client
net.snowflake.ingest.streaming.internal.SnowflakeStreamingIngestChannelInternal<?>	openChannel(OpenChannelRequest request)	Open a channel against a Snowflake table
void	setRefreshToken(String refreshToken)	Set refresh token, this method is for refresh token renewal without requiring to restart client.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnowflakeStreamingIngestClientInternal

public SnowflakeStreamingIngestClientInternal(String name,
                                              SnowflakeURL accountURL,
                                              Properties prop,
                                              Map<String,Object> parameterOverrides,
                                              boolean isIcebergMode,
                                              boolean isTestMode)

Default Constructor

Parameters:

name - the name of the client

accountURL - Snowflake account url

prop - connection properties

parameterOverrides - map of parameters to override for this client

isIcebergMode - whether we're streaming to Iceberg tables

isTestMode - indicates whether it's under test mode

Method Detail

injectRequestBuilder

public void injectRequestBuilder(RequestBuilder requestBuilder)

getName

public String getName()

Get the client name

Specified by:

getName in interface SnowflakeStreamingIngestClient

Returns:

the client name

isClosed

public boolean isClosed()

Description copied from interface: SnowflakeStreamingIngestClient

Check whether the client is closed or not, if you want to make sure all data are committed before closing, please call AutoCloseable.close() before closing the entire client

Specified by:

isClosed in interface SnowflakeStreamingIngestClient

Returns:

a boolean to indicate whether the client is closed or not

openChannel

public net.snowflake.ingest.streaming.internal.SnowflakeStreamingIngestChannelInternal<?> openChannel(OpenChannelRequest request)

Open a channel against a Snowflake table

Specified by:

openChannel in interface SnowflakeStreamingIngestClient

Parameters:

request - the open channel request

Returns:

a SnowflakeStreamingIngestChannel object

dropChannel

public void dropChannel(DropChannelRequest request)

Description copied from interface: SnowflakeStreamingIngestClient

Drop the specified channel on the server using a DropChannelRequest

Note that this call will blindly drop the latest version of the channel and any pending data will be lost. It will also delete Offset Token and other state from Snowflake servers. So only use it if you are completely done ingesting data for this channel. If you open a channel with the same name in the future, it will behave like a new channel.

Also see SnowflakeStreamingIngestChannel.close(boolean) to drop channels on close. That approach will drop the local version of the channel and if the channel has been concurrently reopened by another client, that version of the channel won't be affected.

Specified by:

dropChannel in interface SnowflakeStreamingIngestClient

Parameters:

request - the drop channel request

getLatestCommittedOffsetTokens

public Map<String,String> getLatestCommittedOffsetTokens(List<SnowflakeStreamingIngestChannel> channels)

Return the latest committed/persisted offset token for all channels

Specified by:

getLatestCommittedOffsetTokens in interface SnowflakeStreamingIngestClient

Returns:

map of channel to the latest persisted offset token

close

public void close()
           throws Exception

Close the client, which will flush first and then release all the resources

Specified by:

close in interface AutoCloseable

Throws:

Exception

setRefreshToken

public void setRefreshToken(String refreshToken)

Set refresh token, this method is for refresh token renewal without requiring to restart client. This method only works when the authorization type is OAuth

Specified by:

setRefreshToken in interface SnowflakeStreamingIngestClient

Parameters:

refreshToken - the new refresh token