org.apache.hadoop.fs.s3a.auth.delegation

Class AbstractDelegationTokenBinding

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService

org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service

Direct Known Subclasses:

FullCredentialsTokenBinding, SessionTokenBinding

public abstract class AbstractDelegationTokenBinding
extends AbstractDTService

An AbstractDelegationTokenBinding implementation is a class which handles the binding of its underlying authentication mechanism to the Hadoop Delegation token mechanism. See also org.apache.hadoop.fs.azure.security.WasbDelegationTokenManager but note that it assumes Kerberos tokens for which the renewal mechanism is the sole plugin point. This class is designed to be more generic. Lifecycle It is a Hadoop Service, so has a standard lifecycle: once started its lifecycle will follow that of the S3ADelegationTokens instance which created it --which itself follows the lifecycle of the FS. One big difference is that AbstractDTService.bindToFileSystem(URI, org.apache.hadoop.fs.s3a.impl.StoreContext, DelegationOperations) will be called before the AbstractService.init(Configuration) operation, this is where the owning FS is passed in. Implementations are free to start background operations in their serviceStart() method, provided they are safely stopped in serviceStop(). When to check for the ability to issue tokens Implementations MUST start up without actually holding the secrets needed to issue tokens (config options, credentials to talk to STS etc) as in server-side deployments they are not expected to have these. Retry Policy All methods which talk to AWS services are expected to do translation, with retries as they see fit.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	AbstractDelegationTokenBinding.TokenSecretManager The secret manager always uses the same secret; the factory for new identifiers is that of the token manager.

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractDelegationTokenBinding(String name, org.apache.hadoop.io.Text kind) Constructor.

Method Summary

Modifier and Type	Method and Description
AWSCredentialProviderList	bindToTokenIdentifier(AbstractS3ATokenIdentifier retrievedIdentifier) Bind to the token identifier, returning the credential providers to use for the owner to talk to S3 and related AWS Services.
protected <T extends AbstractS3ATokenIdentifier> T	convertTokenIdentifier(AbstractS3ATokenIdentifier identifier, Class<T> expectedClass) Verify that a token identifier is of a specific class.
org.apache.hadoop.security.token.Token<AbstractS3ATokenIdentifier>	createDelegationToken(Optional<RoleModel.Policy> policy, EncryptionSecrets encryptionSecrets, org.apache.hadoop.io.Text renewer) Create a delegation token for the user.
abstract AbstractS3ATokenIdentifier	createEmptyIdentifier() Create a new subclass of AbstractS3ATokenIdentifier.
protected org.apache.hadoop.security.token.SecretManager<AbstractS3ATokenIdentifier>	createSecretMananger() Create a secret manager.
abstract AbstractS3ATokenIdentifier	createTokenIdentifier(Optional<RoleModel.Policy> policy, EncryptionSecrets encryptionSecrets, org.apache.hadoop.io.Text renewer) Create a token identifier with all the information needed to be included in a delegation token.
DelegationBindingInfo	deploy(AbstractS3ATokenIdentifier retrievedIdentifier) Deploy, returning the binding information.
AWSCredentialProviderList	deployUnbonded() Perform any actions when deploying unbonded, and return a list of credential providers.
String	getDescription() Return a description.
org.apache.hadoop.io.Text	getKind() Get the kind of the tokens managed here.
org.apache.hadoop.io.Text	getOwnerText() Return the name of the owner to be used in tokens.
protected static byte[]	getSecretManagerPasssword() Get the password to use in secret managers.
S3ADelegationTokens.TokenIssuingPolicy	getTokenIssuingPolicy() Predicate: will this binding issue a DT? That is: should the filesystem declare that it is issuing delegation tokens? If true
String	getUserAgentField() Return a string for use in building up the User-Agent field, so get into the S3 access logs.
protected void	serviceStart() Service startup: create the secret manager.
String	toString()

Methods inherited from class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService

bindToFileSystem, getCanonicalUri, getOwner, getPolicyProvider, getStoreContext, requireServiceStarted, requireServiceState, serviceInit

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStop, setConfig, start, stop, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AbstractDelegationTokenBinding

protected AbstractDelegationTokenBinding(String name,
                                         org.apache.hadoop.io.Text kind)

Constructor.

Parameters:

name - as passed to superclass for use in log messages.

kind - token kind.

Method Detail

getKind

public org.apache.hadoop.io.Text getKind()

Get the kind of the tokens managed here.

Returns:

the token kind.

getOwnerText

public org.apache.hadoop.io.Text getOwnerText()

Return the name of the owner to be used in tokens. This may be that of the UGI owner, or it could be related to the AWS login.

Returns:

a text name of the owner.

getTokenIssuingPolicy

public S3ADelegationTokens.TokenIssuingPolicy getTokenIssuingPolicy()

Predicate: will this binding issue a DT? That is: should the filesystem declare that it is issuing delegation tokens? If true

Returns:

a declaration of what will happen when asked for a token.

createDelegationToken

public org.apache.hadoop.security.token.Token<AbstractS3ATokenIdentifier> createDelegationToken(Optional<RoleModel.Policy> policy,
                                                                                                EncryptionSecrets encryptionSecrets,
                                                                                                org.apache.hadoop.io.Text renewer)
                                                                                         throws IOException

Create a delegation token for the user. This will only be called if a new DT is needed, that is: the filesystem has been deployed unbonded.

Parameters:

policy - minimum policy to use, if known.

encryptionSecrets - encryption secrets for the token.

renewer - the principal permitted to renew the token.

Returns:

the token or null if the back end does not want to issue one.

Throws:

IOException - if one cannot be created

createTokenIdentifier

public abstract AbstractS3ATokenIdentifier createTokenIdentifier(Optional<RoleModel.Policy> policy,
                                                                 EncryptionSecrets encryptionSecrets,
                                                                 org.apache.hadoop.io.Text renewer)
                                                          throws IOException

Create a token identifier with all the information needed to be included in a delegation token. This is where session credentials need to be extracted, etc. This will only be called if a new DT is needed, that is: the filesystem has been deployed unbonded. If createDelegationToken(Optional, EncryptionSecrets, Text) is overridden, this method can be replaced with a stub.

Parameters:

policy - minimum policy to use, if known.

encryptionSecrets - encryption secrets for the token.

renewer - the principal permitted to renew the token.

Returns:

the token data to include in the token identifier.

Throws:

IOException - failure creating the token data.

convertTokenIdentifier

protected <T extends AbstractS3ATokenIdentifier> T convertTokenIdentifier(AbstractS3ATokenIdentifier identifier,
                                                                          Class<T> expectedClass)
                                                                   throws DelegationTokenIOException

Verify that a token identifier is of a specific class. This will reject subclasses (i.e. it is stricter than instanceof, then cast it to that type.

Type Parameters:

T - type of S3A delegation ttoken identifier.

Parameters:

identifier - identifier to validate

expectedClass - class of the expected token identifier.

Returns:

token identifier.

Throws:

DelegationTokenIOException - If the wrong class was found.

deploy

public DelegationBindingInfo deploy(AbstractS3ATokenIdentifier retrievedIdentifier)
                             throws IOException

Deploy, returning the binding information. The base implementation calls

Parameters:

retrievedIdentifier - any identifier -null if deployed unbonded.

Returns:

binding information

Throws:

IOException - any failure.

deployUnbonded

public AWSCredentialProviderList deployUnbonded()
                                         throws IOException

Perform any actions when deploying unbonded, and return a list of credential providers.

Returns:

non-empty list of AWS credential providers to use for authenticating this client with AWS services.

Throws:

IOException - any failure.

UnsupportedOperationException - in the base implementation.

bindToTokenIdentifier

public AWSCredentialProviderList bindToTokenIdentifier(AbstractS3ATokenIdentifier retrievedIdentifier)
                                                throws IOException

Bind to the token identifier, returning the credential providers to use for the owner to talk to S3 and related AWS Services.

Parameters:

retrievedIdentifier - the unmarshalled data

Returns:

non-empty list of AWS credential providers to use for authenticating this client with AWS services.

Throws:

IOException - any failure

UnsupportedOperationException - in the base implementation.

createEmptyIdentifier

public abstract AbstractS3ATokenIdentifier createEmptyIdentifier()

Create a new subclass of AbstractS3ATokenIdentifier. This is used in the secret manager.

Returns:

an empty identifier.

toString

public String toString()

Overrides:

toString in class org.apache.hadoop.service.AbstractService

serviceStart

protected void serviceStart()
                     throws Exception

Service startup: create the secret manager.

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

Exception - failure.

getDescription

public String getDescription()

Return a description. This is logged during after service start and binding: it should be as informative as possible.

Returns:

a description to log.

createSecretMananger

protected org.apache.hadoop.security.token.SecretManager<AbstractS3ATokenIdentifier> createSecretMananger()
                                                                                                   throws IOException

Create a secret manager.

Returns:

a secret manager.

Throws:

IOException - on failure

getUserAgentField

public String getUserAgentField()

Return a string for use in building up the User-Agent field, so get into the S3 access logs. Useful for diagnostics.

Returns:

a string for the S3 logs or "" for "nothing to add"

getSecretManagerPasssword

protected static byte[] getSecretManagerPasssword()

Get the password to use in secret managers. This is a constant; its just recalculated every time to stop findbugs highlighting security risks of shared mutable byte arrays.

Returns:

a password.