org.apache.hadoop.fs.s3a.statistics.impl

Class BondedS3AStatisticsContext

java.lang.Object

org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext

All Implemented Interfaces:

CountersAndGauges, S3AStatisticsContext, org.apache.hadoop.fs.statistics.DurationTrackerFactory

public class BondedS3AStatisticsContext
extends Object
implements S3AStatisticsContext

An S3A statistics context which is bonded to a S3AInstrumentation instance -inevitably that of an S3AFileSystem instance.

An interface is used to bind to the relevant fields, rather than have them passed in the constructor because some production code, specifically, DelegateToFileSystem, patches the protected field after initialization.

All operations are passed through directly to that class.

If an instance of FileSystem.Statistics is passed in, it will be used whenever input stream statistics are created - However, Internally always increments the statistics in the current thread.

As a result, cross-thread IO will under-report.

This is addressed through the stream statistics classes only updating the stats in the close() call. Provided they are closed in the worker thread, all stats collected in helper threads will be included.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	BondedS3AStatisticsContext.S3AFSStatisticsSource This is the interface which an integration source must implement for the integration.

Constructor Summary

Constructors

Constructor and Description
BondedS3AStatisticsContext(BondedS3AStatisticsContext.S3AFSStatisticsSource statisticsSource) Instantiate.

Method Summary

Modifier and Type	Method and Description
void	addValueToQuantiles(Statistic op, long value) Add a value to a quantiles statistic.
S3AMultipartUploaderStatistics	createMultipartUploaderStatistics() Creaet a multipart statistics collector.
void	decrementGauge(Statistic op, long count) Decrement a specific gauge.
void	incrementCounter(Statistic op, long count) Increment a specific counter.
void	incrementGauge(Statistic op, long count) Increment a specific gauge.
CommitterStatistics	newCommitterStatistics() Create a new instance of the committer statistics.
DelegationTokenStatistics	newDelegationTokenStatistics() Create a delegation token statistics instance.
S3AInputStreamStatistics	newInputStreamStatistics() Create a stream input statistics instance.
BlockOutputStreamStatistics	newOutputStreamStatistics() Create a stream output statistics instance.
StatisticsFromAwsSdk	newStatisticsFromAwsSdk() Create a StatisticsFromAwsSdk instance.
void	recordDuration(Statistic op, boolean success, Duration duration) Record a duration.
org.apache.hadoop.fs.statistics.DurationTracker	trackDuration(String key, long count)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.fs.statistics.DurationTrackerFactory

trackDuration

Constructor Detail

BondedS3AStatisticsContext

public BondedS3AStatisticsContext(BondedS3AStatisticsContext.S3AFSStatisticsSource statisticsSource)

Instantiate.

Parameters:

statisticsSource - integration binding

Method Detail

newInputStreamStatistics

public S3AInputStreamStatistics newInputStreamStatistics()

Create a stream input statistics instance. The FileSystem.Statistics instance of the statisticsSource is used as the reference to FileSystem statistics to update

Specified by:

newInputStreamStatistics in interface S3AStatisticsContext

Returns:

the new instance

newCommitterStatistics

public CommitterStatistics newCommitterStatistics()

Create a new instance of the committer statistics.

Specified by:

newCommitterStatistics in interface S3AStatisticsContext

Returns:

a new committer statistics instance

newOutputStreamStatistics

public BlockOutputStreamStatistics newOutputStreamStatistics()

Create a stream output statistics instance.

Specified by:

newOutputStreamStatistics in interface S3AStatisticsContext

Returns:

the new instance

incrementCounter

public void incrementCounter(Statistic op,
                             long count)

Increment a specific counter.

No-op if not defined.

Specified by:

incrementCounter in interface CountersAndGauges

Parameters:

op - operation

count - increment value

incrementGauge

public void incrementGauge(Statistic op,
                           long count)

Increment a specific gauge.

No-op if not defined.

Specified by:

incrementGauge in interface CountersAndGauges

Parameters:

op - operation

count - increment value

Throws:

ClassCastException - if the metric is of the wrong type

decrementGauge

public void decrementGauge(Statistic op,
                           long count)

Decrement a specific gauge.

No-op if not defined.

Specified by:

decrementGauge in interface CountersAndGauges

Parameters:

op - operation

count - increment value

Throws:

ClassCastException - if the metric is of the wrong type

addValueToQuantiles

public void addValueToQuantiles(Statistic op,
                                long value)

Add a value to a quantiles statistic. No-op if the quantile isn't found.

Specified by:

addValueToQuantiles in interface CountersAndGauges

Parameters:

op - operation to look up.

value - value to add.

Throws:

ClassCastException - if the metric is not a Quantiles.

recordDuration

public void recordDuration(Statistic op,
                           boolean success,
                           Duration duration)

Description copied from interface: CountersAndGauges

Record a duration.

Specified by:

recordDuration in interface CountersAndGauges

Parameters:

op - operation

success - was the operation a success?

duration - how long did it take

newDelegationTokenStatistics

public DelegationTokenStatistics newDelegationTokenStatistics()

Create a delegation token statistics instance.

Specified by:

newDelegationTokenStatistics in interface S3AStatisticsContext

Returns:

an instance of delegation token statistics

newStatisticsFromAwsSdk

public StatisticsFromAwsSdk newStatisticsFromAwsSdk()

Description copied from interface: S3AStatisticsContext

Create a StatisticsFromAwsSdk instance.

Specified by:

newStatisticsFromAwsSdk in interface S3AStatisticsContext

Returns:

an instance of StatisticsFromAwsSdk

createMultipartUploaderStatistics

public S3AMultipartUploaderStatistics createMultipartUploaderStatistics()

Description copied from interface: S3AStatisticsContext

Creaet a multipart statistics collector.

Specified by:

createMultipartUploaderStatistics in interface S3AStatisticsContext

Returns:

an instance

trackDuration

public org.apache.hadoop.fs.statistics.DurationTracker trackDuration(String key,
                                                                     long count)

Specified by:

trackDuration in interface org.apache.hadoop.fs.statistics.DurationTrackerFactory