org.apache.hadoop.fs.s3a.impl

Class UploadContentProviders.BaseContentProvider<T extends InputStream>

java.lang.Object

org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider<T>

Type Parameters:

T - type of stream created.

All Implemented Interfaces:

Closeable, AutoCloseable, software.amazon.awssdk.http.ContentStreamProvider

Enclosing class:

UploadContentProviders

@VisibleForTesting
public abstract static class UploadContentProviders.BaseContentProvider<T extends InputStream>
extends Object
implements software.amazon.awssdk.http.ContentStreamProvider, Closeable

Base class for content providers; tracks the number of times a stream has been opened.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseContentProvider(int size) Constructor.
protected	BaseContentProvider(int size, Supplier<Boolean> isOpen) Constructor.

Method Summary

Modifier and Type	Method and Description
void	close() Close the current stream.
protected abstract T	createNewStream() Override point for subclasses to create their new streams.
protected T	getCurrentStream() Current stream.
int	getSize() Size as set by constructor parameter.
LocalDateTime	getStartTime() When did this upload start?
int	getStreamCreationCount() How many times has a stream been created?
InputStream	newStream() Create a new stream.
protected T	setCurrentStream(T stream) Set the current stream.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BaseContentProvider

protected BaseContentProvider(int size)

Constructor.

Parameters:

size - size of the data. Must be non-negative.

BaseContentProvider

protected BaseContentProvider(int size,
                              @Nullable
                              Supplier<Boolean> isOpen)

Constructor.

Parameters:

size - size of the data. Must be non-negative.

isOpen - optional predicate to check if the stream is open.

Method Detail

close

public void close()

Close the current stream.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

newStream

public final InputStream newStream()

Create a new stream.

Calls close() to ensure that any existing stream is closed, then checkOpen() to verify that the data source is still open. Logs if this is a subsequent event as it implies a failure of the first attempt.

Specified by:

newStream in interface software.amazon.awssdk.http.ContentStreamProvider

Returns:

the new stream

createNewStream

protected abstract T createNewStream()

Override point for subclasses to create their new streams.

Returns:

a stream

getStreamCreationCount

public int getStreamCreationCount()

How many times has a stream been created?

Returns:

stream creation count

getSize

public int getSize()

Size as set by constructor parameter.

Returns:

size of the data

getStartTime

public LocalDateTime getStartTime()

When did this upload start?

Returns:

start time

getCurrentStream

protected T getCurrentStream()

Current stream. When newStream() is called, this is set to the new value, after closing the previous one.

Why? The AWS SDK implementations do this, so there is an implication that it is needed to avoid keeping streams open on retries.

Returns:

the current stream, or null if none is open.

setCurrentStream

protected T setCurrentStream(T stream)

Set the current stream.

Parameters:

stream - the new stream

Returns:

the current stream.

toString

public String toString()

Overrides:

toString in class Object