org.apache.hadoop.fs.s3a.impl

Class GetContentSummaryOperation

java.lang.Object

org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation

org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation<org.apache.hadoop.fs.ContentSummary>

org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation

All Implemented Interfaces:

org.apache.hadoop.fs.statistics.IOStatisticsSource, org.apache.hadoop.util.functional.CallableRaisingIOE<org.apache.hadoop.fs.ContentSummary>

public class GetContentSummaryOperation
extends ExecutingStoreOperation<org.apache.hadoop.fs.ContentSummary>
implements org.apache.hadoop.fs.statistics.IOStatisticsSource

GetContentSummary operation. This is based on FileSystem.get#getContentSummary; its still doing sequential treewalk with the efficiency issues. Changes: 1. On the recursive calls there is no probe to see if the path is a file: we know the recursion only happens with a dir. 2. If a subdirectory is not found during the walk, that does not trigger an error. The directory is clearly not part of the content any more. The Operation serves up IOStatistics; this counts the cost of all the list operations, but not the initial HEAD probe to see if the path is a file.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	GetContentSummaryOperation.GetContentSummaryCallbacks Callbacks used by the operation.

Constructor Summary

Constructors

Constructor and Description
GetContentSummaryOperation(StoreContext storeContext, org.apache.hadoop.fs.Path path, GetContentSummaryOperation.GetContentSummaryCallbacks callbacks) Constructor.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.ContentSummary	execute() Return the ContentSummary of a given path.
org.apache.hadoop.fs.ContentSummary	getDirSummary(org.apache.hadoop.fs.Path dir) Return the ContentSummary of a given directory.
org.apache.hadoop.fs.statistics.IOStatistics	getIOStatistics()

Methods inherited from class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation

apply, executeOnlyOnce

Methods inherited from class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation

activateAuditSpan, getAuditSpan, getStoreContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GetContentSummaryOperation

public GetContentSummaryOperation(StoreContext storeContext,
                                  org.apache.hadoop.fs.Path path,
                                  GetContentSummaryOperation.GetContentSummaryCallbacks callbacks)

Constructor.

Parameters:

storeContext - context.

path - path to summarize

callbacks - callbacks for S3 access.

Method Detail

getIOStatistics

public org.apache.hadoop.fs.statistics.IOStatistics getIOStatistics()

Specified by:

getIOStatistics in interface org.apache.hadoop.fs.statistics.IOStatisticsSource

execute

@Retries.RetryTranslated
public org.apache.hadoop.fs.ContentSummary execute()
                                                                     throws IOException

Return the ContentSummary of a given path.

Specified by:

execute in class ExecutingStoreOperation<org.apache.hadoop.fs.ContentSummary>

Returns:

the summary.

Throws:

FileNotFoundException - if the path does not resolve

IOException - failure

getDirSummary

public org.apache.hadoop.fs.ContentSummary getDirSummary(org.apache.hadoop.fs.Path dir)
                                                  throws IOException

Return the ContentSummary of a given directory. This is a recursive operation (as the original is); it'd be more efficient of stack and heap if it managed its own stack.

Parameters:

dir - dir to scan

Returns:

the content summary

Throws:

FileNotFoundException - if the path does not resolve

IOException - IO failure

FileNotFoundException - the path does not exist

IOException - failure