org.apache.hadoop.hbase.io

类 FileLink

java.lang.Object

org.apache.hadoop.hbase.io.FileLink

直接已知子类:

HFileLink, WALLink

@InterfaceAudience.Private
public class FileLink
extends Object

The FileLink is a sort of hardlink, that allows access to a file given a set of locations.

The Problem:

• HDFS doesn't have support for hardlinks, and this make impossible to referencing the same data blocks using different names.
• HBase store files in one location (e.g. table/region/family/) and when the file is not needed anymore (e.g. compaction, region deletion, ...) moves it to an archive directory.

If we want to create a reference to a file, we need to remember that it can be in its original location or in the archive folder. The FileLink class tries to abstract this concept and given a set of locations it is able to switch between them making this operation transparent for the user. HFileLink is a more concrete implementation of the FileLink.

Back-references: To help the CleanerChore to keep track of the links to a particular file, during the FileLink creation, a new file is placed inside a back-reference directory. There's one back-reference directory for each file that has links, and in the directory there's one file per link.

HFileLink Example

• /hbase/table/region-x/cf/file-k (Original File)
• /hbase/table-cloned/region-y/cf/file-k.region-x.table (HFileLink to the original file)
• /hbase/table-2nd-cloned/region-z/cf/file-k.region-x.table (HFileLink to the original file)
• /hbase/.archive/table/region-x/.links-file-k/region-y.table-cloned (Back-reference to the link in table-cloned)
• /hbase/.archive/table/region-x/.links-file-k/region-z.table-2nd-cloned (Back-reference to the link in table-2nd-cloned)

字段概要

字段

限定符和类型	字段和说明
static String	BACK_REFERENCES_DIRECTORY_PREFIX Define the Back-reference directory name prefix: .links-<hfile>/

构造器概要

构造器

限定符	构造器和说明
protected	FileLink()
	FileLink(Collection<org.apache.hadoop.fs.Path> locations)
	FileLink(org.apache.hadoop.fs.Path originPath, org.apache.hadoop.fs.Path... alternativePaths)

方法概要

方法

限定符和类型	方法和说明
boolean	equals(Object obj)
boolean	exists(org.apache.hadoop.fs.FileSystem fs)
org.apache.hadoop.fs.Path	getAvailablePath(org.apache.hadoop.fs.FileSystem fs)
static String	getBackReferenceFileName(org.apache.hadoop.fs.Path dirPath) Get the referenced file name from the reference link directory path.
static org.apache.hadoop.fs.Path	getBackReferencesDir(org.apache.hadoop.fs.Path storeDir, String fileName) Get the directory to store the link back references To simplify the reference count process, during the FileLink creation a back-reference is added to the back-reference directory of the specified file.
org.apache.hadoop.fs.FileStatus	getFileStatus(org.apache.hadoop.fs.FileSystem fs) Get the FileStatus of the referenced file.
org.apache.hadoop.fs.Path[]	getLocations()
int	hashCode()
static boolean	isBackReferencesDir(org.apache.hadoop.fs.Path dirPath) Checks if the specified directory path is a back reference links folder.
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.FileSystem fs) Open the FileLink for read.
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.FileSystem fs, int bufferSize) Open the FileLink for read.
protected void	setLocations(org.apache.hadoop.fs.Path originPath, org.apache.hadoop.fs.Path... alternativePaths) NOTE: This method must be used only in the constructor!
String	toString()

从类继承的方法 java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

字段详细资料

BACK_REFERENCES_DIRECTORY_PREFIX

public static final String BACK_REFERENCES_DIRECTORY_PREFIX

Define the Back-reference directory name prefix: .links-<hfile>/

另请参阅:

常量字段值

构造器详细资料

FileLink

protected FileLink()

FileLink

public FileLink(org.apache.hadoop.fs.Path originPath,
        org.apache.hadoop.fs.Path... alternativePaths)

参数:

originPath - Original location of the file to link

alternativePaths - Alternative locations to look for the linked file

FileLink

public FileLink(Collection<org.apache.hadoop.fs.Path> locations)

参数:

locations - locations to look for the linked file

方法详细资料

getLocations

public org.apache.hadoop.fs.Path[] getLocations()

返回:

the locations to look for the linked file.

toString

public String toString()

覆盖:

toString 在类中 Object

exists

public boolean exists(org.apache.hadoop.fs.FileSystem fs)
               throws IOException

返回:

true if the file pointed by the link exists

抛出:

IOException

getAvailablePath

public org.apache.hadoop.fs.Path getAvailablePath(org.apache.hadoop.fs.FileSystem fs)
                                           throws IOException

返回:

the path of the first available link.

抛出:

IOException

getFileStatus

public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.FileSystem fs)
                                              throws IOException

Get the FileStatus of the referenced file.

参数:

fs - FileSystem on which to get the file status

返回:

InputStream for the hfile link.

抛出:

IOException - on unexpected error.

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.FileSystem fs)
                                            throws IOException

Open the FileLink for read.

It uses a wrapper of FSDataInputStream that is agnostic to the location of the file, even if the file switches between locations.

参数:

fs - FileSystem on which to open the FileLink

返回:

InputStream for reading the file link.

抛出:

IOException - on unexpected error.

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.FileSystem fs,
                                          int bufferSize)
                                            throws IOException

Open the FileLink for read.

It uses a wrapper of FSDataInputStream that is agnostic to the location of the file, even if the file switches between locations.

参数:

fs - FileSystem on which to open the FileLink

bufferSize - the size of the buffer to be used.

返回:

InputStream for reading the file link.

抛出:

IOException - on unexpected error.

setLocations

protected void setLocations(org.apache.hadoop.fs.Path originPath,
                org.apache.hadoop.fs.Path... alternativePaths)

NOTE: This method must be used only in the constructor! It creates a List with the specified locations for the link.

getBackReferencesDir

public static org.apache.hadoop.fs.Path getBackReferencesDir(org.apache.hadoop.fs.Path storeDir,
                                             String fileName)

Get the directory to store the link back references

To simplify the reference count process, during the FileLink creation a back-reference is added to the back-reference directory of the specified file.

参数:

storeDir - Root directory for the link reference folder

fileName - File Name with links

返回:

Path for the link back references.

getBackReferenceFileName

public static String getBackReferenceFileName(org.apache.hadoop.fs.Path dirPath)

Get the referenced file name from the reference link directory path.

参数:

dirPath - Link references directory path

返回:

Name of the file referenced

isBackReferencesDir

public static boolean isBackReferencesDir(org.apache.hadoop.fs.Path dirPath)

Checks if the specified directory path is a back reference links folder.

参数:

dirPath - Directory path to verify

返回:

True if the specified directory is a link references folder

equals

public boolean equals(Object obj)

覆盖:

equals 在类中 Object

hashCode

public int hashCode()

覆盖:

hashCode 在类中 Object