Package org.apache.pinot.common.utils.fetcher

Class HttpSegmentFetcher

java.lang.Object

org.apache.pinot.common.utils.fetcher.BaseSegmentFetcher

org.apache.pinot.common.utils.fetcher.HttpSegmentFetcher

All Implemented Interfaces:

SegmentFetcher

Direct Known Subclasses:

HttpsSegmentFetcher

public class HttpSegmentFetcher
extends BaseSegmentFetcher

Field Summary

Fields

Modifier and Type	Field	Description
protected FileUploadDownloadClient	_httpClient

Fields inherited from class org.apache.pinot.common.utils.fetcher.BaseSegmentFetcher

_authProvider, _logger, _retryCount, _retryDelayScaleFactor, _retryWaitMs, DEFAULT_RETRY_COUNT, DEFAULT_RETRY_DELAY_SCALE_FACTOR, DEFAULT_RETRY_WAIT_MS, RETRY_COUNT_CONFIG_KEY, RETRY_DELAY_SCALE_FACTOR_CONFIG_KEY, RETRY_WAIT_MS_CONFIG_KEY

Constructor Summary

Constructors

Constructor	Description
HttpSegmentFetcher()

Method Summary

Modifier and Type	Method	Description
protected void	doInit(PinotConfiguration config)	Override this for custom initialization.
void	fetchSegmentToLocal(URI downloadURI, File dest)	Fetches a segment from URI location to local.
void	fetchSegmentToLocalWithoutRetry(URI uri, File dest)	Fetches a segment from URI location to local without retry.
File	fetchUntarSegmentToLocalStreamed(URI downloadURI, File dest, long maxStreamRateInByte)	Fetches a segment from URI location and untar to local in a streamed manner

Methods inherited from class org.apache.pinot.common.utils.fetcher.BaseSegmentFetcher

fetchSegmentToLocal, init

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_httpClient

protected FileUploadDownloadClient _httpClient

Constructor Detail

HttpSegmentFetcher

public HttpSegmentFetcher()

Method Detail

doInit

protected void doInit(PinotConfiguration config)

Description copied from class: BaseSegmentFetcher

Override this for custom initialization.

Overrides:

doInit in class BaseSegmentFetcher

fetchSegmentToLocal

public void fetchSegmentToLocal(URI downloadURI,
                                File dest)
                         throws Exception

Description copied from interface: SegmentFetcher

Fetches a segment from URI location to local.

Specified by:

fetchSegmentToLocal in interface SegmentFetcher

Overrides:

fetchSegmentToLocal in class BaseSegmentFetcher

Throws:

Exception

fetchUntarSegmentToLocalStreamed

public File fetchUntarSegmentToLocalStreamed(URI downloadURI,
                                             File dest,
                                             long maxStreamRateInByte)
                                      throws Exception

Description copied from interface: SegmentFetcher

Fetches a segment from URI location and untar to local in a streamed manner

Specified by:

fetchUntarSegmentToLocalStreamed in interface SegmentFetcher

Overrides:

fetchUntarSegmentToLocalStreamed in class BaseSegmentFetcher

Throws:

Exception

fetchSegmentToLocalWithoutRetry

public void fetchSegmentToLocalWithoutRetry(URI uri,
                                            File dest)
                                     throws Exception

Description copied from class: BaseSegmentFetcher

Fetches a segment from URI location to local without retry. Sub-class should override this or BaseSegmentFetcher.fetchSegmentToLocal(URI, File).

Overrides:

fetchSegmentToLocalWithoutRetry in class BaseSegmentFetcher

Throws:

Exception