org.apache.hadoop.fs.azurebfs.services

Class AbfsClient

java.lang.Object

org.apache.hadoop.fs.azurebfs.services.AbfsClient

All Implemented Interfaces:

Closeable, AutoCloseable

public class AbfsClient
extends Object
implements Closeable

AbfsClient.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
AbfsClient(URL baseUrl, SharedKeyCredentials sharedKeyCredentials, AbfsConfiguration abfsConfiguration, AccessTokenProvider tokenProvider, AbfsClientContext abfsClientContext)
AbfsClient(URL baseUrl, SharedKeyCredentials sharedKeyCredentials, AbfsConfiguration abfsConfiguration, SASTokenProvider sasTokenProvider, AbfsClientContext abfsClientContext)

Method Summary

Modifier and Type	Method and Description
AbfsRestOperation	acquireLease(String path, int duration, TracingContext tracingContext)
<V> void	addCallback(org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListenableFuture<V> future, org.apache.hadoop.thirdparty.com.google.common.util.concurrent.FutureCallback<V> callback)
AbfsRestOperation	append(String path, byte[] buffer, AppendRequestParameters reqParams, String cachedSasToken, TracingContext tracingContext)
boolean	appendSuccessCheckOp(AbfsRestOperation op, String path, long length, TracingContext tracingContext)
AbfsRestOperation	breakLease(String path, TracingContext tracingContext)
AbfsRestOperation	checkAccess(String path, String rwx, TracingContext tracingContext) Talks to the server to check whether the permission specified in the rwx parameter is present for the path specified in the path parameter.
void	close()
AbfsRestOperation	createFilesystem(TracingContext tracingContext)
AbfsRestOperation	createPath(String path, boolean isFile, boolean overwrite, String permission, String umask, boolean isAppendBlob, String eTag, TracingContext tracingContext)
protected URL	createRequestUrl(String path, String query)
AbfsRestOperation	deleteFilesystem(TracingContext tracingContext)
AbfsRestOperation	deleteIdempotencyCheckOp(AbfsRestOperation op) Check if the delete request failure is post a retry and if delete failure qualifies to be a success response assuming idempotency.
AbfsRestOperation	deletePath(String path, boolean recursive, String continuation, TracingContext tracingContext)
AbfsRestOperation	flush(String path, long position, boolean retainUncommittedData, boolean isClose, String cachedSasToken, String leaseId, TracingContext tracingContext)
protected AbfsCounters	getAbfsCounters() Getter for abfsCounters from AbfsClient.
protected AbfsPerfTracker	getAbfsPerfTracker()
String	getAccessToken()
AbfsRestOperation	getAclStatus(String path, boolean useUPN, TracingContext tracingContext)
AbfsRestOperation	getAclStatus(String path, TracingContext tracingContext)
AuthType	getAuthType()
static String	getDirectoryQueryParameter(String path) Get the directory query parameter used by the List Paths REST API and used as the path in the continuation token.
String	getFileSystem()
AbfsRestOperation	getFilesystemProperties(TracingContext tracingContext)
int	getNumLeaseThreads()
AbfsRestOperation	getPathStatus(String path, boolean includeProperties, TracingContext tracingContext)
SASTokenProvider	getSasTokenProvider()
AbfsRestOperation	listPath(String relativePath, boolean recursive, int listMaxResults, String continuation, TracingContext tracingContext)
AbfsRestOperation	read(String path, long position, byte[] buffer, int bufferOffset, int bufferLength, String eTag, String cachedSasToken, TracingContext tracingContext)
AbfsRestOperation	releaseLease(String path, String leaseId, TracingContext tracingContext)
AbfsRestOperation	renamePath(String source, String destination, String continuation, TracingContext tracingContext)
AbfsRestOperation	renewLease(String path, String leaseId, TracingContext tracingContext)
<V> org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListenableScheduledFuture<V>	schedule(Callable<V> callable, long delay, TimeUnit timeUnit)
AbfsRestOperation	setAcl(String path, String aclSpecString, String eTag, TracingContext tracingContext)
AbfsRestOperation	setAcl(String path, String aclSpecString, TracingContext tracingContext)
AbfsRestOperation	setFilesystemProperties(String properties, TracingContext tracingContext)
AbfsRestOperation	setOwner(String path, String owner, String group, TracingContext tracingContext)
AbfsRestOperation	setPathProperties(String path, String properties, TracingContext tracingContext)
AbfsRestOperation	setPermission(String path, String permission, TracingContext tracingContext)
org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListenableFuture<?>	submit(Runnable runnable)
static String	urlEncode(String value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

AbfsClient

public AbfsClient(URL baseUrl,
                  SharedKeyCredentials sharedKeyCredentials,
                  AbfsConfiguration abfsConfiguration,
                  AccessTokenProvider tokenProvider,
                  AbfsClientContext abfsClientContext)
           throws IOException

Throws:

IOException

AbfsClient

public AbfsClient(URL baseUrl,
                  SharedKeyCredentials sharedKeyCredentials,
                  AbfsConfiguration abfsConfiguration,
                  SASTokenProvider sasTokenProvider,
                  AbfsClientContext abfsClientContext)
           throws IOException

Throws:

IOException

Method Detail

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

getFileSystem

public String getFileSystem()

getAbfsPerfTracker

protected AbfsPerfTracker getAbfsPerfTracker()

createFilesystem

public AbfsRestOperation createFilesystem(TracingContext tracingContext)
                                   throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

setFilesystemProperties

public AbfsRestOperation setFilesystemProperties(String properties,
                                                 TracingContext tracingContext)
                                          throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

listPath

public AbfsRestOperation listPath(String relativePath,
                                  boolean recursive,
                                  int listMaxResults,
                                  String continuation,
                                  TracingContext tracingContext)
                           throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

getFilesystemProperties

public AbfsRestOperation getFilesystemProperties(TracingContext tracingContext)
                                          throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

deleteFilesystem

public AbfsRestOperation deleteFilesystem(TracingContext tracingContext)
                                   throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

createPath

public AbfsRestOperation createPath(String path,
                                    boolean isFile,
                                    boolean overwrite,
                                    String permission,
                                    String umask,
                                    boolean isAppendBlob,
                                    String eTag,
                                    TracingContext tracingContext)
                             throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

acquireLease

public AbfsRestOperation acquireLease(String path,
                                      int duration,
                                      TracingContext tracingContext)
                               throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

renewLease

public AbfsRestOperation renewLease(String path,
                                    String leaseId,
                                    TracingContext tracingContext)
                             throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

releaseLease

public AbfsRestOperation releaseLease(String path,
                                      String leaseId,
                                      TracingContext tracingContext)
                               throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

breakLease

public AbfsRestOperation breakLease(String path,
                                    TracingContext tracingContext)
                             throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

renamePath

public AbfsRestOperation renamePath(String source,
                                    String destination,
                                    String continuation,
                                    TracingContext tracingContext)
                             throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

append

public AbfsRestOperation append(String path,
                                byte[] buffer,
                                AppendRequestParameters reqParams,
                                String cachedSasToken,
                                TracingContext tracingContext)
                         throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

appendSuccessCheckOp

public boolean appendSuccessCheckOp(AbfsRestOperation op,
                                    String path,
                                    long length,
                                    TracingContext tracingContext)
                             throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

flush

public AbfsRestOperation flush(String path,
                               long position,
                               boolean retainUncommittedData,
                               boolean isClose,
                               String cachedSasToken,
                               String leaseId,
                               TracingContext tracingContext)
                        throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

setPathProperties

public AbfsRestOperation setPathProperties(String path,
                                           String properties,
                                           TracingContext tracingContext)
                                    throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

getPathStatus

public AbfsRestOperation getPathStatus(String path,
                                       boolean includeProperties,
                                       TracingContext tracingContext)
                                throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

read

public AbfsRestOperation read(String path,
                              long position,
                              byte[] buffer,
                              int bufferOffset,
                              int bufferLength,
                              String eTag,
                              String cachedSasToken,
                              TracingContext tracingContext)
                       throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

deletePath

public AbfsRestOperation deletePath(String path,
                                    boolean recursive,
                                    String continuation,
                                    TracingContext tracingContext)
                             throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

deleteIdempotencyCheckOp

public AbfsRestOperation deleteIdempotencyCheckOp(AbfsRestOperation op)

Check if the delete request failure is post a retry and if delete failure qualifies to be a success response assuming idempotency. There are below scenarios where delete could be incorrectly deducted as success post request retry: 1. Target was originally not existing and initial delete request had to be re-tried. 2. Parallel delete issued from any other store interface rather than delete issued from this filesystem instance. These are few corner cases and usually returning a success at this stage should help the job to continue.

Parameters:

op - Delete request REST operation response with non-null HTTP response

Returns:

REST operation response post idempotency check

setOwner

public AbfsRestOperation setOwner(String path,
                                  String owner,
                                  String group,
                                  TracingContext tracingContext)
                           throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

setPermission

public AbfsRestOperation setPermission(String path,
                                       String permission,
                                       TracingContext tracingContext)
                                throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

setAcl

public AbfsRestOperation setAcl(String path,
                                String aclSpecString,
                                TracingContext tracingContext)
                         throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

setAcl

public AbfsRestOperation setAcl(String path,
                                String aclSpecString,
                                String eTag,
                                TracingContext tracingContext)
                         throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

getAclStatus

public AbfsRestOperation getAclStatus(String path,
                                      TracingContext tracingContext)
                               throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

getAclStatus

public AbfsRestOperation getAclStatus(String path,
                                      boolean useUPN,
                                      TracingContext tracingContext)
                               throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

checkAccess

public AbfsRestOperation checkAccess(String path,
                                     String rwx,
                                     TracingContext tracingContext)
                              throws AzureBlobFileSystemException

Talks to the server to check whether the permission specified in the rwx parameter is present for the path specified in the path parameter.

Parameters:

path - Path for which access check needs to be performed

rwx - The permission to be checked on the path

tracingContext - Tracks identifiers for request header

Returns:

The AbfsRestOperation object for the operation

Throws:

AzureBlobFileSystemException - in case of bad requests

getDirectoryQueryParameter

public static String getDirectoryQueryParameter(String path)

Get the directory query parameter used by the List Paths REST API and used as the path in the continuation token. If the input path is null or the root path "/", empty string is returned. If the input path begins with '/', the return value is the substring beginning at offset 1. Otherwise, the input path is returned.

Parameters:

path - the path to be listed.

Returns:

the value of the directory query parameter

createRequestUrl

protected URL createRequestUrl(String path,
                               String query)
                        throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

urlEncode

public static String urlEncode(String value)
                        throws AzureBlobFileSystemException

Throws:

AzureBlobFileSystemException

getAccessToken

public String getAccessToken()
                      throws IOException

Throws:

IOException

getAuthType

public AuthType getAuthType()

getSasTokenProvider

public SASTokenProvider getSasTokenProvider()

getAbfsCounters

protected AbfsCounters getAbfsCounters()

Getter for abfsCounters from AbfsClient.

Returns:

AbfsCounters instance.

getNumLeaseThreads

public int getNumLeaseThreads()

schedule

public <V> org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListenableScheduledFuture<V> schedule(Callable<V> callable,
                                                                                                                long delay,
                                                                                                                TimeUnit timeUnit)

submit

public org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListenableFuture<?> submit(Runnable runnable)

addCallback

public <V> void addCallback(org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListenableFuture<V> future,
                            org.apache.hadoop.thirdparty.com.google.common.util.concurrent.FutureCallback<V> callback)