net.snowflake.ingest.connection

Class RequestBuilder

java.lang.Object

net.snowflake.ingest.connection.RequestBuilder

public final class RequestBuilder
extends Object

This class handles constructing the URIs for our requests as well as putting together the payloads we'll be sending

Author:

obabarinsa

Field Summary

Fields

Modifier and Type	Field and Description
static String	CLIENT_NAME
static String	DEFAULT_HOST_SUFFIX
static String	DEFAULT_VERSION
static String	JAVA_USER_AGENT
static String	OS_INFO_USER_AGENT_FORMAT

Constructor Summary

Constructors

Constructor and Description
RequestBuilder(String accountName, String userName, KeyPair keyPair) RequestBuilder - general usage constructor
RequestBuilder(String accountName, String userName, KeyPair keyPair, String schemeName, String hostName, int portNum) Constructor to use if not intended to use userAgentSuffix.
RequestBuilder(String accountName, String userName, KeyPair keyPair, String schemeName, String hostName, int portNum, String userAgentSuffix) RequestBuilder - this constructor is for testing purposes only
RequestBuilder(String accountName, String userName, String hostName, KeyPair keyPair, String userAgentSuffix) RequestBuilder constructor which uses default schemes, host and port.

Method Summary

Modifier and Type	Method and Description
void	closeResources() Closes the resources being used by RequestBuilder object.
org.apache.http.client.methods.HttpGet	generateHistoryRangeRequest(UUID requestId, String pipe, String startTimeInclusive, String endTimeExclusive) generateHistoryRangeRequest - given a requestId and a pipe, get history for all ingests between time ranges start-end
org.apache.http.client.methods.HttpGet	generateHistoryRequest(UUID requestId, String pipe, Integer recentSeconds, String beginMark) generateHistoryRequest - given a requestId and a pipe, make a history request
org.apache.http.client.methods.HttpPost	generateInsertRequest(UUID requestId, String pipe, List<StagedFileWrapper> files, boolean showSkippedFiles) generateInsertRequest - given a table, stage and list of files, make a request for the insert endpoint

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_HOST_SUFFIX

public static final String DEFAULT_HOST_SUFFIX

See Also:

Constant Field Values

CLIENT_NAME

public static final String CLIENT_NAME

See Also:

Constant Field Values

DEFAULT_VERSION

public static final String DEFAULT_VERSION

See Also:

Constant Field Values

JAVA_USER_AGENT

public static final String JAVA_USER_AGENT

See Also:

Constant Field Values

OS_INFO_USER_AGENT_FORMAT

public static final String OS_INFO_USER_AGENT_FORMAT

See Also:

Constant Field Values

Constructor Detail

RequestBuilder

public RequestBuilder(String accountName,
                      String userName,
                      KeyPair keyPair)

RequestBuilder - general usage constructor

Parameters:

accountName - - the name of the Snowflake account to which we're connecting

userName - - the username of the entity loading files

keyPair - - the Public/Private key pair we'll use to authenticate

RequestBuilder

public RequestBuilder(String accountName,
                      String userName,
                      String hostName,
                      KeyPair keyPair,
                      String userAgentSuffix)

RequestBuilder constructor which uses default schemes, host and port.

Parameters:

accountName - - the name of the Snowflake account to which we're connecting

userName - - the username of the entity loading files

keyPair - - the Public/Private key pair we'll use to authenticate

RequestBuilder

public RequestBuilder(String accountName,
                      String userName,
                      KeyPair keyPair,
                      String schemeName,
                      String hostName,
                      int portNum)

Constructor to use if not intended to use userAgentSuffix. i.e. User-Agent HTTP header suffix part is null, (The default one is still valid, check out #defaultUserAgent)

Parameters:

accountName - - the account name to which we're connecting

userName - - for whom are we connecting?

keyPair - - our auth credentials

schemeName - - are we HTTP or HTTPS?

hostName - - the host for this snowflake instance

portNum - - the port number

RequestBuilder

public RequestBuilder(String accountName,
                      String userName,
                      KeyPair keyPair,
                      String schemeName,
                      String hostName,
                      int portNum,
                      String userAgentSuffix)

RequestBuilder - this constructor is for testing purposes only

Parameters:

accountName - - the account name to which we're connecting

userName - - for whom are we connecting?

keyPair - - our auth credentials

schemeName - - are we HTTP or HTTPS?

hostName - - the host for this snowflake instance

portNum - - the port number

userAgentSuffix - - The suffix part of HTTP Header User-Agent

Method Detail

generateInsertRequest

public org.apache.http.client.methods.HttpPost generateInsertRequest(UUID requestId,
                                                                     String pipe,
                                                                     List<StagedFileWrapper> files,
                                                                     boolean showSkippedFiles)
                                                              throws URISyntaxException

generateInsertRequest - given a table, stage and list of files, make a request for the insert endpoint

Parameters:

requestId - a UUID we will use to label this request

pipe - a fully qualified pipe name

files - a list of files

showSkippedFiles - a boolean which returns skipped files when set to true

Returns:

a post request with all the data we need

Throws:

URISyntaxException - if the URI components provided are improper

generateHistoryRequest

public org.apache.http.client.methods.HttpGet generateHistoryRequest(UUID requestId,
                                                                     String pipe,
                                                                     Integer recentSeconds,
                                                                     String beginMark)
                                                              throws URISyntaxException

generateHistoryRequest - given a requestId and a pipe, make a history request

Parameters:

requestId - a UUID we will use to label this request

pipe - a fully qualified pipe name

recentSeconds - history only for items in the recentSeconds window

beginMark - mark from which history should be fetched

Throws:

URISyntaxException - - If the URI components provided are improper

generateHistoryRangeRequest

public org.apache.http.client.methods.HttpGet generateHistoryRangeRequest(UUID requestId,
                                                                          String pipe,
                                                                          String startTimeInclusive,
                                                                          String endTimeExclusive)
                                                                   throws URISyntaxException

generateHistoryRangeRequest - given a requestId and a pipe, get history for all ingests between time ranges start-end

Parameters:

requestId - a UUID we will use to label this request

pipe - a fully qualified pipe name

startTimeInclusive - Start time inclusive of scan range, in ISO-8601 format. Missing millisecond part in string will lead to a zero milliseconds. This is a required query parameter, and a 400 will be returned if this query parameter is missing

endTimeExclusive - End time exclusive of scan range. If this query parameter is missing or user provided value is later than current millis, then current millis is used.

Returns:

URI for the insert request

Throws:

URISyntaxException

closeResources

public void closeResources()

Closes the resources being used by RequestBuilder object. SecurityManager is one such resource which uses a threadpool which needs to be shutdown once SimpleIngestManager is done interacting with Snowpipe Service (Rest APIs)

Throws:

Exception