Package org.apache.pinot.common.utils

Class FileUploadDownloadClient

java.lang.Object

org.apache.pinot.common.utils.FileUploadDownloadClient

All Implemented Interfaces:

AutoCloseable

public class FileUploadDownloadClient
extends Object
implements AutoCloseable

The FileUploadDownloadClient class provides methods to upload schema/segment, download segment or send segment completion protocol request through HTTP/HTTPS.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FileUploadDownloadClient.CustomHeaders
static class	FileUploadDownloadClient.FileUploadType
static class	FileUploadDownloadClient.QueryParameters

Constructor Summary

Constructors

Constructor	Description
FileUploadDownloadClient()
FileUploadDownloadClient(SSLContext sslContext)

Method Summary

Modifier and Type	Method	Description
SimpleHttpResponse	addSchema(URI uri, String schemaName, File schemaFile)	Deprecated.
SimpleHttpResponse	addSchema(URI uri, String schemaName, File schemaFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters)	Add schema.
void	close()
int	downloadFile(URI uri, int socketTimeoutMs, File dest)	Deprecated.
int	downloadFile(URI uri, File dest)	Deprecated.
int	downloadFile(URI uri, File dest, AuthProvider authProvider)	Download a file.
int	downloadFile(URI uri, File dest, AuthProvider authProvider, List<org.apache.http.Header> httpHeaders)	Download a file.
File	downloadUntarFileStreamed(URI uri, File dest, AuthProvider authProvider, List<org.apache.http.Header> httpHeaders, long maxStreamRateInByte)	Download and untar a file in a streamed way with rate limit
SimpleHttpResponse	endReplaceSegments(URI uri, int socketTimeoutMs, AuthProvider authProvider)	End replace segments with default settings.
static URI	getDeleteSchemaURI(String protocol, String host, int port, String schemaName)
static URI	getDeleteSegmentHttpUri(String host, int port, String rawTableName, String segmentName, String tableType)	Deprecated.
static URI	getDeleteTableURI(String protocol, String host, int port, String tableName, String type, String retention)
static URI	getEndReplaceSegmentsURI(URI controllerURI, String rawTableName, String tableType, String segmentLineageEntryId)
HttpClient	getHttpClient()
static URI	getOldUploadSegmentHttpsURI(String host, int port)	Deprecated.
static URI	getOldUploadSegmentHttpURI(String host, int port)	Deprecated.
static URI	getRetrieveAllSegmentWithTableTypeHttpUri(String host, int port, String rawTableName, String tableType)	Deprecated.
static URI	getRetrieveSchemaHttpURI(String host, int port, String schemaName)	Deprecated.
static URI	getRetrieveSchemaURI(String protocol, String host, int port, String schemaName)
static URI	getRetrieveTableConfigHttpURI(String host, int port, String rawTableName)	Deprecated.
static URI	getRetrieveTableConfigURI(String protocol, String host, int port, String rawTableName)
static URI	getRevertReplaceSegmentsURI(URI controllerURI, String rawTableName, String tableType, String segmentLineageEntryId, boolean forceRevert)
Map<String,List<String>>	getSegments(URI controllerUri, String rawTableName, TableType tableType, boolean excludeReplacedSegments)	Returns a map from a given tableType to a list of segments for that given tableType (OFFLINE or REALTIME) If tableType is left unspecified, both OFFLINE and REALTIME segments will be returned in the map.
static URI	getStartReplaceSegmentsURI(URI controllerURI, String rawTableName, String tableType, boolean forceCleanup)
static URI	getUploadSchemaHttpsURI(String host, int port)	Deprecated.
static URI	getUploadSchemaHttpURI(String host, int port)	Deprecated.
static URI	getUploadSchemaURI(String protocol, String host, int port)
static URI	getUploadSchemaURI(URI controllerURI)
static URI	getUploadSegmentHttpsURI(String host, int port)	Deprecated.
static URI	getUploadSegmentHttpURI(String host, int port)	Deprecated.
static URI	getUploadSegmentURI(String protocol, String host, int port)
static URI	getUploadSegmentURI(URI controllerURI)
static URI	getURI(String protocol, String host, int port, String path)
static URI	getURI(String protocol, String host, int port, String path, String query)
static List<org.apache.http.NameValuePair>	makeTableParam(String tableName)	Generate a param list with a table name attribute.
SimpleHttpResponse	revertReplaceSegments(URI uri)	Revert replace segments with default settings.
SimpleHttpResponse	sendSegmentCompletionProtocolRequest(URI uri, int socketTimeoutMs)	Deprecated.
SimpleHttpResponse	sendSegmentCompletionProtocolRequest(URI uri, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Send segment completion protocol request.
SimpleHttpResponse	sendSegmentJson(URI uri, String jsonString)	Deprecated.
SimpleHttpResponse	sendSegmentJson(URI uri, String jsonString, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Send segment json.
SimpleHttpResponse	sendSegmentUri(URI uri, String downloadUri, String rawTableName)	Deprecated.
SimpleHttpResponse	sendSegmentUri(URI uri, String downloadUri, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Send segment uri.
SimpleHttpResponse	startReplaceSegments(URI uri, StartReplaceSegmentsRequest startReplaceSegmentsRequest, AuthProvider authProvider)	Start replace segments with default settings.
SimpleHttpResponse	updateSchema(URI uri, String schemaName, File schemaFile)	Deprecated.
SimpleHttpResponse	updateSchema(URI uri, String schemaName, File schemaFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters)	Update schema.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, String tableName)	Deprecated.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, String tableName, TableType tableType)	Upload segment with segment file using default settings.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, String tableName, TableType tableType, boolean enableParallelPushProtection, boolean allowRefresh)	Upload segment with segment file using table name, type, enableParallelPushProtection and allowRefresh as request parameters.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Upload segment with segment file.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, InputStream inputStream, String rawTableName)	Upload segment with segment file input stream using default settings.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, InputStream inputStream, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Upload segment with segment file input stream.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, InputStream inputStream, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, String tableName, TableType tableType)	Upload segment with segment file input stream.
SimpleHttpResponse	uploadSegmentMetadata(URI uri, String segmentName, File segmentMetadataFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Upload segment by sending a zip of creation.meta and metadata.properties.
SimpleHttpResponse	uploadSegmentMetadataFiles(URI uri, Map<String,File> metadataFiles, int segmentUploadRequestTimeoutMs)	Deprecated.
SimpleHttpResponse	uploadSegmentMetadataFiles(URI uri, Map<String,File> metadataFiles, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int segmentUploadRequestTimeoutMs)
String	uploadToSegmentStore(String uri)	Used by controllers to send requests to servers: Controller periodic task uses this endpoint to ask servers to upload committed llc segment to segment store if missing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileUploadDownloadClient

public FileUploadDownloadClient()

FileUploadDownloadClient

public FileUploadDownloadClient(SSLContext sslContext)

Method Detail

getHttpClient

public HttpClient getHttpClient()

getURI

public static URI getURI(String protocol,
                         String host,
                         int port,
                         String path)
                  throws URISyntaxException

Throws:

URISyntaxException

getURI

public static URI getURI(String protocol,
                         String host,
                         int port,
                         String path,
                         String query)
                  throws URISyntaxException

Throws:

URISyntaxException

getRetrieveTableConfigHttpURI

@Deprecated
public static URI getRetrieveTableConfigHttpURI(String host,
                                                int port,
                                                String rawTableName)
                                         throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getRetrieveTableConfigURI(String, String, int, String)

getRetrieveTableConfigURI

public static URI getRetrieveTableConfigURI(String protocol,
                                            String host,
                                            int port,
                                            String rawTableName)
                                     throws URISyntaxException

Throws:

URISyntaxException

getDeleteSegmentHttpUri

@Deprecated
public static URI getDeleteSegmentHttpUri(String host,
                                          int port,
                                          String rawTableName,
                                          String segmentName,
                                          String tableType)
                                   throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled This method calls the old segment endpoint. We will deprecate this behavior soon.

Throws:

URISyntaxException

getRetrieveAllSegmentWithTableTypeHttpUri

@Deprecated
public static URI getRetrieveAllSegmentWithTableTypeHttpUri(String host,
                                                            int port,
                                                            String rawTableName,
                                                            String tableType)
                                                     throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled This method calls the old segment endpoint. We will deprecate this behavior soon.

Throws:

URISyntaxException

getRetrieveSchemaHttpURI

@Deprecated
public static URI getRetrieveSchemaHttpURI(String host,
                                           int port,
                                           String schemaName)
                                    throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getRetrieveSchemaURI(String, String, int, String)

getRetrieveSchemaURI

public static URI getRetrieveSchemaURI(String protocol,
                                       String host,
                                       int port,
                                       String schemaName)
                                throws URISyntaxException

Throws:

URISyntaxException

getUploadSchemaHttpURI

@Deprecated
public static URI getUploadSchemaHttpURI(String host,
                                         int port)
                                  throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSchemaURI(String, String, int)

getUploadSchemaHttpsURI

@Deprecated
public static URI getUploadSchemaHttpsURI(String host,
                                          int port)
                                   throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSchemaURI(String, String, int)

getUploadSchemaURI

public static URI getUploadSchemaURI(String protocol,
                                     String host,
                                     int port)
                              throws URISyntaxException

Throws:

URISyntaxException

getDeleteSchemaURI

public static URI getDeleteSchemaURI(String protocol,
                                     String host,
                                     int port,
                                     String schemaName)
                              throws URISyntaxException

Throws:

URISyntaxException

getDeleteTableURI

public static URI getDeleteTableURI(String protocol,
                                    String host,
                                    int port,
                                    String tableName,
                                    String type,
                                    String retention)
                             throws URISyntaxException

Throws:

URISyntaxException

getUploadSchemaURI

public static URI getUploadSchemaURI(URI controllerURI)
                              throws URISyntaxException

Throws:

URISyntaxException

getOldUploadSegmentHttpURI

@Deprecated
public static URI getOldUploadSegmentHttpURI(String host,
                                             int port)
                                      throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

This method calls the old segment upload endpoint. We will deprecate this behavior soon. Please call getUploadSegmentHttpURI to construct your request.

getOldUploadSegmentHttpsURI

@Deprecated
public static URI getOldUploadSegmentHttpsURI(String host,
                                              int port)
                                       throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

This method calls the old segment upload endpoint. We will deprecate this behavior soon. Please call getUploadSegmentHttpsURI to construct your request.

getUploadSegmentHttpURI

@Deprecated
public static URI getUploadSegmentHttpURI(String host,
                                          int port)
                                   throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSegmentURI(String, String, int)

getUploadSegmentHttpsURI

@Deprecated
public static URI getUploadSegmentHttpsURI(String host,
                                           int port)
                                    throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSegmentURI(String, String, int)

getUploadSegmentURI

public static URI getUploadSegmentURI(String protocol,
                                      String host,
                                      int port)
                               throws URISyntaxException

Throws:

URISyntaxException

getUploadSegmentURI

public static URI getUploadSegmentURI(URI controllerURI)
                               throws URISyntaxException

Throws:

URISyntaxException

getStartReplaceSegmentsURI

public static URI getStartReplaceSegmentsURI(URI controllerURI,
                                             String rawTableName,
                                             String tableType,
                                             boolean forceCleanup)
                                      throws URISyntaxException

Throws:

URISyntaxException

getEndReplaceSegmentsURI

public static URI getEndReplaceSegmentsURI(URI controllerURI,
                                           String rawTableName,
                                           String tableType,
                                           String segmentLineageEntryId)
                                    throws URISyntaxException

Throws:

URISyntaxException

getRevertReplaceSegmentsURI

public static URI getRevertReplaceSegmentsURI(URI controllerURI,
                                              String rawTableName,
                                              String tableType,
                                              String segmentLineageEntryId,
                                              boolean forceRevert)
                                       throws URISyntaxException

Throws:

URISyntaxException

addSchema

@Deprecated
public SimpleHttpResponse addSchema(URI uri,
                                    String schemaName,
                                    File schemaFile)
                             throws IOException,
                                    HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Add schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

addSchema(URI, String, File, List, List)

addSchema

public SimpleHttpResponse addSchema(URI uri,
                                    String schemaName,
                                    File schemaFile,
                                    @Nullable
                                    List<org.apache.http.Header> headers,
                                    @Nullable
                                    List<org.apache.http.NameValuePair> parameters)
                             throws IOException,
                                    HttpErrorStatusException

Add schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

headers - HTTP headers

parameters - HTTP parameters

Returns:

Response

Throws:

IOException

HttpErrorStatusException

updateSchema

@Deprecated
public SimpleHttpResponse updateSchema(URI uri,
                                       String schemaName,
                                       File schemaFile)
                                throws IOException,
                                       HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Update schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

updateSchema(URI, String, File, List, List)

updateSchema

public SimpleHttpResponse updateSchema(URI uri,
                                       String schemaName,
                                       File schemaFile,
                                       @Nullable
                                       List<org.apache.http.Header> headers,
                                       @Nullable
                                       List<org.apache.http.NameValuePair> parameters)
                                throws IOException,
                                       HttpErrorStatusException

Update schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

headers - HTTP headers

parameters - HTTP parameters

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegmentMetadata

public SimpleHttpResponse uploadSegmentMetadata(URI uri,
                                                String segmentName,
                                                File segmentMetadataFile,
                                                @Nullable
                                                List<org.apache.http.Header> headers,
                                                @Nullable
                                                List<org.apache.http.NameValuePair> parameters,
                                                int socketTimeoutMs)
                                         throws IOException,
                                                HttpErrorStatusException

Upload segment by sending a zip of creation.meta and metadata.properties.

Parameters:

uri - URI

segmentName - Segment name

segmentMetadataFile - Segment metadata file

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegmentMetadataFiles

@Deprecated
public SimpleHttpResponse uploadSegmentMetadataFiles(URI uri,
                                                     Map<String,File> metadataFiles,
                                                     int segmentUploadRequestTimeoutMs)
                                              throws IOException,
                                                     HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled

Throws:

IOException

HttpErrorStatusException

See Also:

uploadSegment(URI, String, InputStream, List, List, int)

uploadSegmentMetadataFiles

public SimpleHttpResponse uploadSegmentMetadataFiles(URI uri,
                                                     Map<String,File> metadataFiles,
                                                     @Nullable
                                                     List<org.apache.http.Header> headers,
                                                     @Nullable
                                                     List<org.apache.http.NameValuePair> parameters,
                                                     int segmentUploadRequestTimeoutMs)
                                              throws IOException,
                                                     HttpErrorStatusException

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        @Nullable
                                        List<org.apache.http.Header> headers,
                                        @Nullable
                                        List<org.apache.http.NameValuePair> parameters,
                                        int socketTimeoutMs)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file. Note: table name needs to be added as a parameter except for the case where this gets called during realtime segment commit protocol. TODO: fix the realtime segment commit protocol to add table name as a parameter.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

@Deprecated
public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        String tableName)
                                 throws IOException,
                                        HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Upload segment with segment file using default settings. Include table name as a request parameter.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

tableName - Table name with or without type suffix

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

uploadSegment(URI, String, InputStream, List, List, int)

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        String tableName,
                                        TableType tableType)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file using default settings. Include table name and type as a request parameters.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

tableName - Table name with or without type suffix

tableType - Table type

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        String tableName,
                                        TableType tableType,
                                        boolean enableParallelPushProtection,
                                        boolean allowRefresh)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file using table name, type, enableParallelPushProtection and allowRefresh as request parameters.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

tableName - Table name with or without type suffix

tableType - Table type

enableParallelPushProtection - enable protection against concurrent segment uploads for the same segment

allowRefresh - whether to refresh a segment if it already exists

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        InputStream inputStream,
                                        @Nullable
                                        List<org.apache.http.Header> headers,
                                        @Nullable
                                        List<org.apache.http.NameValuePair> parameters,
                                        String tableName,
                                        TableType tableType)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file input stream. Note: table name has to be set as a parameter.

Parameters:

uri - URI

segmentName - Segment name

inputStream - Segment file input stream

headers - Optional http headers

parameters - Optional query parameters

tableName - Table name with or without type suffix

tableType - Table type

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        InputStream inputStream,
                                        @Nullable
                                        List<org.apache.http.Header> headers,
                                        @Nullable
                                        List<org.apache.http.NameValuePair> parameters,
                                        int socketTimeoutMs)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file input stream. Note: table name has to be set as a parameter.

Parameters:

uri - URI

segmentName - Segment name

inputStream - Segment file input stream

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        InputStream inputStream,
                                        String rawTableName)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file input stream using default settings. Include table name as a request parameter.

Parameters:

uri - URI

segmentName - Segment name

inputStream - Segment file input stream

rawTableName - Raw table name

Returns:

Response

Throws:

IOException

HttpErrorStatusException

getSegments

public Map<String,List<String>> getSegments(URI controllerUri,
                                                  String rawTableName,
                                                  @Nullable
                                                  TableType tableType,
                                                  boolean excludeReplacedSegments)
                                           throws Exception

Returns a map from a given tableType to a list of segments for that given tableType (OFFLINE or REALTIME) If tableType is left unspecified, both OFFLINE and REALTIME segments will be returned in the map.

Throws:

Exception

uploadToSegmentStore

public String uploadToSegmentStore(String uri)
                            throws URISyntaxException,
                                   IOException,
                                   HttpErrorStatusException

Used by controllers to send requests to servers: Controller periodic task uses this endpoint to ask servers to upload committed llc segment to segment store if missing.

Parameters:

uri - The uri to ask servers to upload segment to segment store

Returns:

the uploaded segment download url from segment store

Throws:

URISyntaxException

IOException

HttpErrorStatusException - TODO: migrate this method to another class

sendSegmentUri

public SimpleHttpResponse sendSegmentUri(URI uri,
                                         String downloadUri,
                                         @Nullable
                                         List<org.apache.http.Header> headers,
                                         @Nullable
                                         List<org.apache.http.NameValuePair> parameters,
                                         int socketTimeoutMs)
                                  throws IOException,
                                         HttpErrorStatusException

Send segment uri. Note: table name has to be set as a parameter.

Parameters:

uri - URI

downloadUri - Segment download uri

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

sendSegmentUri

@Deprecated
public SimpleHttpResponse sendSegmentUri(URI uri,
                                         String downloadUri,
                                         String rawTableName)
                                  throws IOException,
                                         HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Send segment uri using default settings. Include table name as a request parameter.

Parameters:

uri - URI

downloadUri - Segment download uri

rawTableName - Raw table name

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

sendSegmentUri(URI, String, List, List, int)

sendSegmentJson

public SimpleHttpResponse sendSegmentJson(URI uri,
                                          String jsonString,
                                          @Nullable
                                          List<org.apache.http.Header> headers,
                                          @Nullable
                                          List<org.apache.http.NameValuePair> parameters,
                                          int socketTimeoutMs)
                                   throws IOException,
                                          HttpErrorStatusException

Send segment json.

Parameters:

uri - URI

jsonString - Segment json string

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

sendSegmentJson

@Deprecated
public SimpleHttpResponse sendSegmentJson(URI uri,
                                          String jsonString)
                                   throws IOException,
                                          HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Send segment json using default settings.

Parameters:

uri - URI

jsonString - Segment json string

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

sendSegmentJson(URI, String, List, List, int)

startReplaceSegments

public SimpleHttpResponse startReplaceSegments(URI uri,
                                               StartReplaceSegmentsRequest startReplaceSegmentsRequest,
                                               @Nullable
                                               AuthProvider authProvider)
                                        throws IOException,
                                               HttpErrorStatusException

Start replace segments with default settings.

Parameters:

uri - URI

startReplaceSegmentsRequest - request

authProvider - auth provider

Returns:

Response

Throws:

IOException

HttpErrorStatusException

endReplaceSegments

public SimpleHttpResponse endReplaceSegments(URI uri,
                                             int socketTimeoutMs,
                                             @Nullable
                                             AuthProvider authProvider)
                                      throws IOException,
                                             HttpErrorStatusException

End replace segments with default settings.

Parameters:

uri - URI

authProvider - auth provider

Returns:

Response

Throws:

IOException

HttpErrorStatusException

revertReplaceSegments

public SimpleHttpResponse revertReplaceSegments(URI uri)
                                         throws IOException,
                                                HttpErrorStatusException

Revert replace segments with default settings.

Parameters:

uri - URI

Returns:

Response

Throws:

IOException

HttpErrorStatusException

sendSegmentCompletionProtocolRequest

@Deprecated
public SimpleHttpResponse sendSegmentCompletionProtocolRequest(URI uri,
                                                               int socketTimeoutMs)
                                                        throws IOException,
                                                               HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Send segment completion protocol request.

Parameters:

uri - URI

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

sendSegmentCompletionProtocolRequest(URI, List, List, int)

sendSegmentCompletionProtocolRequest

public SimpleHttpResponse sendSegmentCompletionProtocolRequest(URI uri,
                                                               @Nullable
                                                               List<org.apache.http.Header> headers,
                                                               @Nullable
                                                               List<org.apache.http.NameValuePair> parameters,
                                                               int socketTimeoutMs)
                                                        throws IOException,
                                                               HttpErrorStatusException

Send segment completion protocol request.

Parameters:

uri - URI

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

downloadFile

@Deprecated
public int downloadFile(URI uri,
                        int socketTimeoutMs,
                        File dest)
                 throws IOException,
                        HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Download a file using default settings

Parameters:

uri - URI

socketTimeoutMs - Socket timeout in milliseconds

dest - File destination

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

See Also:

HttpClient.downloadFile(URI, int, File, AuthProvider, List)

downloadFile

@Deprecated
public int downloadFile(URI uri,
                        File dest)
                 throws IOException,
                        HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Download a file.

Parameters:

uri - URI

dest - File destination

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

See Also:

downloadFile(URI, File, AuthProvider)

downloadFile

public int downloadFile(URI uri,
                        File dest,
                        AuthProvider authProvider)
                 throws IOException,
                        HttpErrorStatusException

Download a file.

Parameters:

uri - URI

dest - File destination

authProvider - auth provider

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

downloadFile

public int downloadFile(URI uri,
                        File dest,
                        AuthProvider authProvider,
                        List<org.apache.http.Header> httpHeaders)
                 throws IOException,
                        HttpErrorStatusException

Download a file.

Parameters:

uri - URI

dest - File destination

authProvider - auth provider

httpHeaders - http headers

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

downloadUntarFileStreamed

public File downloadUntarFileStreamed(URI uri,
                                      File dest,
                                      AuthProvider authProvider,
                                      List<org.apache.http.Header> httpHeaders,
                                      long maxStreamRateInByte)
                               throws IOException,
                                      HttpErrorStatusException

Download and untar a file in a streamed way with rate limit

Parameters:

uri - URI

dest - File destination

authProvider - auth token

httpHeaders - http headers

maxStreamRateInByte - limit the rate to write download-untar stream to disk, in bytes -1 for no disk write limit, 0 for limit the writing to min(untar, download) rate

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

makeTableParam

public static List<org.apache.http.NameValuePair> makeTableParam(String tableName)

Generate a param list with a table name attribute.

Parameters:

tableName - table name

Returns:

param list

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Throws:

IOException