org.apache.hadoop.fs.s3a.impl

Class ChangeDetectionPolicy

java.lang.Object

org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public abstract class ChangeDetectionPolicy
extends Object

Object change detection policy. Determines which attribute is used to detect change and what to do when change is detected.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ChangeDetectionPolicy.Mode What to do when change is detected.
static class	ChangeDetectionPolicy.Source The S3 object attribute used to detect change.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CHANGE_DETECTED

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ChangeDetectionPolicy(ChangeDetectionPolicy.Mode mode, boolean requireVersion)

Method Summary

Modifier and Type	Method and Description
abstract void	applyRevisionConstraint(software.amazon.awssdk.services.s3.model.CopyObjectRequest.Builder requestBuilder, String revisionId) Applies the given revisionId as a server-side qualification on the CopyObjectRequest.
abstract void	applyRevisionConstraint(software.amazon.awssdk.services.s3.model.GetObjectRequest.Builder request, String revisionId) Applies the given revisionId as a server-side qualification on the GetObjectRequest.Builder.
abstract void	applyRevisionConstraint(software.amazon.awssdk.services.s3.model.HeadObjectRequest.Builder requestBuilder, String revisionId) Applies the given revisionId as a server-side qualification on the GetObjectMetadataRequest.
static ChangeDetectionPolicy	createPolicy(ChangeDetectionPolicy.Mode mode, ChangeDetectionPolicy.Source source, boolean requireVersion) Create a policy.
org.apache.hadoop.fs.store.LogExactlyOnce	getLogNoVersionSupport()
ChangeDetectionPolicy.Mode	getMode()
static ChangeDetectionPolicy	getPolicy(org.apache.hadoop.conf.Configuration configuration) Reads the change detection policy from Configuration.
abstract String	getRevisionId(software.amazon.awssdk.services.s3.model.CopyObjectResponse copyObjectResponse) Like {getRevisionId(HeadObjectResponse, String)}, but retrieves the revision identifier from CopyObjectResponse.
abstract String	getRevisionId(software.amazon.awssdk.services.s3.model.GetObjectResponse getObjectResponse, String uri) Like {getRevisionId(HeadObjectResponse, String)}, but retrieves the * revision identifier from GetObjectResponse.
abstract String	getRevisionId(software.amazon.awssdk.services.s3.model.HeadObjectResponse objectMetadata, String uri) Pulls the attribute this policy uses to detect change out of the S3 object metadata.
abstract String	getRevisionId(S3ObjectAttributes s3Attributes) Like {getRevisionId(HeadObjectResponse, String)}, but retrieves the revision identifier from S3ObjectAttributes.
abstract ChangeDetectionPolicy.Source	getSource()
boolean	isRequireVersion()
org.apache.commons.lang3.tuple.ImmutablePair<Boolean,RemoteFileChangedException>	onChangeDetected(String revisionId, String newRevisionId, String uri, long position, String operation, long timesAlreadyDetected) Takes appropriate action based on mode when a change has been detected.
String	toString() String value for logging.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CHANGE_DETECTED

@VisibleForTesting
public static final String CHANGE_DETECTED

See Also:

Constant Field Values

Constructor Detail

ChangeDetectionPolicy

protected ChangeDetectionPolicy(ChangeDetectionPolicy.Mode mode,
                                boolean requireVersion)

Method Detail

getMode

public ChangeDetectionPolicy.Mode getMode()

getSource

public abstract ChangeDetectionPolicy.Source getSource()

isRequireVersion

public boolean isRequireVersion()

getLogNoVersionSupport

public org.apache.hadoop.fs.store.LogExactlyOnce getLogNoVersionSupport()

getPolicy

public static ChangeDetectionPolicy getPolicy(org.apache.hadoop.conf.Configuration configuration)

Reads the change detection policy from Configuration.

Parameters:

configuration - the configuration

Returns:

the policy

createPolicy

@VisibleForTesting
public static ChangeDetectionPolicy createPolicy(ChangeDetectionPolicy.Mode mode,
                                                                    ChangeDetectionPolicy.Source source,
                                                                    boolean requireVersion)

Create a policy.

Parameters:

mode - mode pf checks

source - source of change

requireVersion - throw exception when no version available?

Returns:

the policy

toString

public String toString()

String value for logging.

Overrides:

toString in class Object

Returns:

source and mode.

getRevisionId

public abstract String getRevisionId(software.amazon.awssdk.services.s3.model.HeadObjectResponse objectMetadata,
                                     String uri)

Pulls the attribute this policy uses to detect change out of the S3 object metadata. The policy generically refers to this attribute as revisionId.

Parameters:

objectMetadata - the s3 object metadata

uri - the URI of the object

Returns:

the revisionId string as interpreted by this policy, or potentially null if the attribute is unavailable (such as when the policy says to use versionId but object versioning is not enabled for the bucket).

getRevisionId

public abstract String getRevisionId(software.amazon.awssdk.services.s3.model.GetObjectResponse getObjectResponse,
                                     String uri)

Like {getRevisionId(HeadObjectResponse, String)}, but retrieves the * revision identifier from GetObjectResponse.

Parameters:

getObjectResponse - the response instance

uri - the URI of the object

Returns:

the revisionId string as interpreted by this policy, or potentially null if the attribute is unavailable (such as when the policy says to use versionId but object versioning is not enabled for the bucket).

getRevisionId

public abstract String getRevisionId(S3ObjectAttributes s3Attributes)

Like {getRevisionId(HeadObjectResponse, String)}, but retrieves the revision identifier from S3ObjectAttributes.

Parameters:

s3Attributes - the object attributes

Returns:

the revisionId string as interpreted by this policy, or potentially null if the attribute is unavailable (such as when the policy says to use versionId but object versioning is not enabled for the bucket).

getRevisionId

public abstract String getRevisionId(software.amazon.awssdk.services.s3.model.CopyObjectResponse copyObjectResponse)

Like {getRevisionId(HeadObjectResponse, String)}, but retrieves the revision identifier from CopyObjectResponse.

Parameters:

copyObjectResponse - the copy response

Returns:

the revisionId string as interpreted by this policy, or potentially null if the attribute is unavailable (such as when the policy says to use versionId but object versioning is not enabled for the bucket).

applyRevisionConstraint

public abstract void applyRevisionConstraint(software.amazon.awssdk.services.s3.model.GetObjectRequest.Builder request,
                                             String revisionId)

Applies the given revisionId as a server-side qualification on the GetObjectRequest.Builder.

Parameters:

request - the request

revisionId - the revision id

applyRevisionConstraint

public abstract void applyRevisionConstraint(software.amazon.awssdk.services.s3.model.CopyObjectRequest.Builder requestBuilder,
                                             String revisionId)

Applies the given revisionId as a server-side qualification on the CopyObjectRequest.

Parameters:

requestBuilder - the copy object request builder

revisionId - the revision id

applyRevisionConstraint

public abstract void applyRevisionConstraint(software.amazon.awssdk.services.s3.model.HeadObjectRequest.Builder requestBuilder,
                                             String revisionId)

Applies the given revisionId as a server-side qualification on the GetObjectMetadataRequest.

Parameters:

requestBuilder - the head object request builder

revisionId - the revision id

onChangeDetected

public org.apache.commons.lang3.tuple.ImmutablePair<Boolean,RemoteFileChangedException> onChangeDetected(String revisionId,
                                                                                                         String newRevisionId,
                                                                                                         String uri,
                                                                                                         long position,
                                                                                                         String operation,
                                                                                                         long timesAlreadyDetected)

Takes appropriate action based on mode when a change has been detected.

Parameters:

revisionId - the expected revision id

newRevisionId - the detected revision id

uri - the URI of the object being accessed

position - the position being read in the object

operation - the operation being performed on the object (e.g. open or re-open) that triggered the change detection

timesAlreadyDetected - number of times a change has already been detected on the current stream

Returns:

a pair of: was a change detected, and any exception to throw. If the change was detected, this updates a counter in the stream statistics; If an exception was returned it is thrown after the counter update.