public abstract class UnderFileSystem extends Object
| Modifier and Type | Class and Description |
|---|---|
static class |
UnderFileSystem.SpaceType |
static class |
UnderFileSystem.UnderFSType
Type of under filesystem, to be used by
getUnderFSType() to determine which concrete
under filesystem implementation is being used. |
| Modifier and Type | Field and Description |
|---|---|
protected TachyonConf |
mTachyonConf |
| Modifier | Constructor and Description |
|---|---|
protected |
UnderFileSystem(TachyonConf tachyonConf) |
| Modifier and Type | Method and Description |
|---|---|
abstract void |
close()
Closes this under file system.
|
abstract void |
connectFromMaster(TachyonConf conf,
String hostname)
Takes any necessary actions required to establish a connection to the under file system from
the given master host e.g.
|
abstract void |
connectFromWorker(TachyonConf conf,
String hostname)
Takes any necessary actions required to establish a connection to the under file system from
the given worker host e.g.
|
abstract OutputStream |
create(String path)
Creates a file in the under file system with the indicated name.
|
abstract OutputStream |
create(String path,
int blockSizeByte)
Creates a file in the under file system with the indicated name and block size.
|
abstract OutputStream |
create(String path,
short replication,
int blockSizeByte)
Creates a file in the under file system with the indicated name, replication number and block
size.
|
abstract boolean |
delete(String path,
boolean recursive)
Deletes a file or folder from the under file system with the indicated name.
|
abstract boolean |
exists(String path)
Checks if a file or folder exists in under file system.
|
static UnderFileSystem |
get(String path,
Object ufsConf,
TachyonConf tachyonConf)
Gets the UnderFileSystem instance according to its scheme and configuration.
|
static UnderFileSystem |
get(String path,
TachyonConf tachyonConf)
Gets the UnderFileSystem instance according to its schema.
|
abstract long |
getBlockSizeByte(String path)
Gets the block size of a file in under file system, in bytes.
|
abstract Object |
getConf()
Gets the configuration object for UnderFileSystem.
|
abstract List<String> |
getFileLocations(String path)
Gets the list of locations of the indicated path.
|
abstract List<String> |
getFileLocations(String path,
long offset)
Gets the list of locations of the indicated path given its offset.
|
abstract long |
getFileSize(String path)
Gets the file size in bytes.
|
abstract long |
getModificationTimeMs(String path)
Gets the UTC time of when the indicated path was modified recently in ms.
|
abstract long |
getSpace(String path,
UnderFileSystem.SpaceType type)
Queries the under file system about the space of the indicated path (e.g., space left, space
used and etc).
|
abstract UnderFileSystem.UnderFSType |
getUnderFSType() |
abstract boolean |
isFile(String path)
Checks if the indicated path is a file or not.
|
static boolean |
isHadoopUnderFS(String path,
TachyonConf tachyonConf)
Determines if the given path is on a Hadoop under file system
To decide if a path should use the hadoop implementation, we check
String.startsWith(String) to see if the configured schemas are found. |
abstract String[] |
list(String path)
Returns an array of strings naming the files and directories in the directory denoted by this
abstract pathname.
|
abstract boolean |
mkdirs(String path,
boolean createParent)
Creates the directory named by this abstract pathname.
|
abstract InputStream |
open(String path)
Opens an
InputStream at the indicated path. |
static Pair<String,String> |
parse(TachyonURI path,
TachyonConf tachyonConf)
Transform an input string like hdfs://host:port/dir, hdfs://host:port, file:///dir, /dir into a
pair of address and path.
|
boolean |
providesStorage()
Checks whether the underFS provides storage
|
abstract boolean |
rename(String src,
String dst)
Renames a file or folder from
src to dst in under file system |
abstract void |
setConf(Object conf)
Sets the configuration object for UnderFileSystem.
|
abstract void |
setPermission(String path,
String posixPerm)
Changes posix file permission
|
protected final TachyonConf mTachyonConf
protected UnderFileSystem(TachyonConf tachyonConf)
public static UnderFileSystem get(String path, TachyonConf tachyonConf)
path - file path storing over the ufs.tachyonConf - the TachyonConf instance.IllegalArgumentException - for unknown schemepublic static UnderFileSystem get(String path, Object ufsConf, TachyonConf tachyonConf)
path - file path storing over the ufsufsConf - the configuration object for ufs onlytachyonConf - the TachyonConf instance.IllegalArgumentException - for unknown schemepublic abstract UnderFileSystem.UnderFSType getUnderFSType()
public static boolean isHadoopUnderFS(String path, TachyonConf tachyonConf)
String.startsWith(String) to see if the configured schemas are found.public boolean providesStorage()
public static Pair<String,String> parse(TachyonURI path, TachyonConf tachyonConf)
path - the input path stringpublic abstract void connectFromMaster(TachyonConf conf, String hostname) throws IOException
Depending on the implementation this may be a no-op
conf - Tachyon configurationhostname - The host that wants to connect to the under file systemIOExceptionpublic abstract void connectFromWorker(TachyonConf conf, String hostname) throws IOException
Depending on the implementation this may be a no-op
conf - Tachyon configurationhostname - The host that wants to connect to the under file systemIOExceptionpublic abstract void close()
throws IOException
IOExceptionpublic abstract OutputStream create(String path) throws IOException
path - The file nameOutputStream objectIOExceptionpublic abstract OutputStream create(String path, int blockSizeByte) throws IOException
path - The file nameblockSizeByte - The block size in bytesOutputStream objectIOExceptionpublic abstract OutputStream create(String path, short replication, int blockSizeByte) throws IOException
path - The file namereplication - The number of replications for each blockblockSizeByte - The block size in bytesOutputStream objectIOExceptionpublic abstract boolean delete(String path, boolean recursive) throws IOException
path - The file or folder namerecursive - Whether we delete folder and its childrenIOExceptionpublic abstract boolean exists(String path) throws IOException
path - The file nameIOExceptionpublic abstract long getBlockSizeByte(String path) throws IOException
path - The file nameIOExceptionpublic abstract Object getConf()
public abstract List<String> getFileLocations(String path) throws IOException
path - The file nameIOExceptionpublic abstract List<String> getFileLocations(String path, long offset) throws IOException
path - The file nameoffset - Offset in bytesIOExceptionpublic abstract long getFileSize(String path) throws IOException
path - The file nameIOExceptionpublic abstract long getModificationTimeMs(String path) throws IOException
path - The file or folder nameIOExceptionpublic abstract long getSpace(String path, UnderFileSystem.SpaceType type) throws IOException
path - The path to querytype - The type of queriesIOExceptionpublic abstract boolean isFile(String path) throws IOException
path - The path nameIOExceptionpublic abstract String[] list(String path) throws IOException
If this abstract pathname does not denote a directory, then this method returns null.
Otherwise an array of strings is returned, one for each file or directory in the directory.
Names denoting the directory itself and the directory's parent directory are not included in
the result. Each string is a file name rather than a complete path.
There is no guarantee that the name strings in the resulting array will appear in any specific order; they are not, in particular, guaranteed to appear in alphabetical order.
path - the path to list.null if this abstract pathname does not denote a directory, or if an I/O error
occurs.IOExceptionpublic abstract boolean mkdirs(String path, boolean createParent) throws IOException
path - the folder to createcreateParent - If true, the method creates any necessary but nonexistent parent
directories. Otherwise, the method does not create nonexistent parent directories.true if and only if the directory was created; false
otherwiseIOExceptionpublic abstract InputStream open(String path) throws IOException
InputStream at the indicated path.path - The file nameInputStream objectIOExceptionpublic abstract boolean rename(String src, String dst) throws IOException
src to dst in under file systemsrc - The source file or folder namedst - The destination file or folder nameIOExceptionpublic abstract void setConf(Object conf)
conf - The configuration object accepted by ufs.public abstract void setPermission(String path, String posixPerm) throws IOException
path - path of the fileposixPerm - standard posix permission like "777", "775", etc.IOExceptionCopyright © 2015. All Rights Reserved.