org.apache.hadoop.hbase.util

类 HBaseFsck

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.util.HBaseFsck

所有已实现的接口:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable

已过时。

For removal in hbase-4.0.0. Use HBCK2 instead.

@Deprecated
 @InterfaceAudience.LimitedPrivate(value="Tools")
 @InterfaceStability.Evolving
public class HBaseFsck
extends org.apache.hadoop.conf.Configured
implements Closeable

HBaseFsck (hbck) is a tool for checking and repairing region consistency and table integrity problems in a corrupted HBase. This tool was written for hbase-1.x. It does not work with hbase-2.x; it can read state but is not allowed to change state; i.e. effect 'repair'. Even though it can 'read' state, given how so much has changed in how hbase1 and hbase2 operate, it will often misread. See hbck2 (HBASE-19121) for a hbck tool for hbase2. This class is deprecated.

Region consistency checks verify that hbase:meta, region deployment on region servers and the state of data in HDFS (.regioninfo files) all are in accordance.

Table integrity checks verify that all possible row keys resolve to exactly one region of a table. This means there are no individual degenerate or backwards regions; no holes between regions; and that there are no overlapping regions.

The general repair strategy works in two phases:

1. Repair Table Integrity on HDFS. (merge or fabricate regions)
2. Repair Region Consistency with hbase:meta and assignments

For table integrity repairs, the tables' region directories are scanned for .regioninfo files. Each table's integrity is then verified. If there are any orphan regions (regions with no .regioninfo files) or holes, new regions are fabricated. Backwards regions are sidelined as well as empty degenerate (endkey==startkey) regions. If there are any overlapping regions, a new region is created and all data is merged into the new region.

Table integrity repairs deal solely with HDFS and could potentially be done offline -- the hbase region servers or master do not need to be running. This phase can eventually be used to completely reconstruct the hbase:meta table in an offline fashion.

Region consistency requires three conditions -- 1) valid .regioninfo file present in an HDFS region dir, 2) valid row with .regioninfo data in META, and 3) a region is deployed only at the regionserver that was assigned to with proper state in the master.

Region consistency repairs require hbase to be online so that hbck can contact the HBase master and region servers. The hbck#connect() method must first be called successfully. Much of the region consistency information is transient and less risky to repair.

If hbck is run from the command line, there are a handful of arguments that can be used to limit the kinds of repairs hbck will do. See the code in printUsageAndExit() for more details.

字段概要

字段

限定符和类型	字段和说明
static long	DEFAULT_SLEEP_BEFORE_RERUN 已过时。
static long	DEFAULT_TIME_LAG 已过时。
protected ExecutorService	executor 已过时。
static String	HBCK_LOCK_FILE 已过时。 Here is where hbase-1.x used to default the lock for hbck1.

构造器概要

构造器

构造器和说明
HBaseFsck(org.apache.hadoop.conf.Configuration conf) 已过时。 Constructor
HBaseFsck(org.apache.hadoop.conf.Configuration conf, ExecutorService exec) 已过时。 Constructor

方法概要

限定符和类型	方法和说明
static Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.FSDataOutputStream>	checkAndMarkRunningHbck(org.apache.hadoop.conf.Configuration conf, RetryCounter retryCounter) 已过时。 This method maintains a lock using a file.
void	checkRegionBoundaries() 已过时。
void	cleanReplicationBarrier() 已过时。
void	close() 已过时。
void	connect() 已过时。 To repair region consistency, one must call connect() in order to repair online state.
protected HFileCorruptionChecker	createHFileCorruptionChecker(boolean sidelineCorruptHFiles) 已过时。
static RetryCounterFactory	createLockRetryCounterFactory(org.apache.hadoop.conf.Configuration conf) 已过时。
static void	debugLsr(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path p) 已过时。 ls -r for debugging purposes
static void	debugLsr(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path p, HbckErrorReporter errors) 已过时。 ls -r for debugging purposes
HBaseFsck	exec(ExecutorService exec, String[] args) 已过时。
void	fixEmptyMetaCells() 已过时。 To fix the empty REGIONINFO_QUALIFIER rows from hbase:meta
void	fixOrphanTables() 已过时。 To fix orphan table by creating a .tableinfo file under tableDir 1. if TableInfo is cached, to recover the .tableinfo accordingly 2. else create a default .tableinfo file with following items 2.1 the correct tablename 2.2 the correct colfamily list 2.3 the default properties for both TableDescriptor and ColumnFamilyDescriptor
HbckErrorReporter	getErrors() 已过时。
HFileCorruptionChecker	getHFilecorruptionChecker() 已过时。
int	getMaxMerge() 已过时。
int	getMaxOverlapsToSideline() 已过时。
int	getRetCode() 已过时。
static org.apache.hadoop.fs.Path	getTmpDir(org.apache.hadoop.conf.Configuration conf) 已过时。
void	includeTable(TableName table) 已过时。
boolean	isExclusive() 已过时。 Only one instance of hbck can modify HBase at a time.
static byte[]	keyOnly(byte[] b) 已过时。
void	loadHdfsRegionDirs() 已过时。 Scan HDFS for all regions, recording their information into regionInfoMap
static void	main(String[] args) 已过时。 Main program
int	mergeRegionDirs(org.apache.hadoop.fs.Path targetRegionDir, HbckRegionInfo contained) 已过时。 Merge hdfs data by moving from contained HbckRegionInfo into targetRegionDir.
void	offlineHdfsIntegrityRepair() 已过时。 This repair method analyzes hbase data in hdfs and repairs it to satisfy the table integrity rules.
int	onlineConsistencyRepair() 已过时。 This repair method requires the cluster to be online since it contacts region servers and the masters.
int	onlineHbck() 已过时。 Contacts the master and prints out cluster-wide information
protected HBaseFsck	printUsageAndExit() 已过时。
void	setCheckHdfs(boolean checking) 已过时。
void	setCleanReplicationBarrier(boolean shouldClean) 已过时。
void	setCleanReplicationBarrierTable(String cleanReplicationBarrierTable) 已过时。
static void	setDisplayFullReport() 已过时。 Display the full report from fsck.
void	setFixAssignments(boolean shouldFix) 已过时。 Fix inconsistencies found by fsck.
void	setFixEmptyMetaCells(boolean shouldFix) 已过时。
void	setFixHdfsHoles(boolean shouldFix) 已过时。
void	setFixHdfsOrphans(boolean shouldFix) 已过时。
void	setFixHdfsOverlaps(boolean shouldFix) 已过时。
void	setFixHFileLinks(boolean shouldFix) 已过时。
void	setFixMeta(boolean shouldFix) 已过时。
void	setFixReferenceFiles(boolean shouldFix) 已过时。
void	setFixReplication(boolean shouldFix) 已过时。 Set replication fix mode.
void	setFixSplitParents(boolean shouldFix) 已过时。
void	setFixTableOrphans(boolean shouldFix) 已过时。
void	setFixVersionFile(boolean shouldFix) 已过时。
static void	setForceExclusive() 已过时。 Set exclusive mode.
void	setHFileCorruptionChecker(HFileCorruptionChecker hfcc) 已过时。
void	setIgnorePreCheckPermission(boolean ignorePreCheckPermission) 已过时。
void	setMaxMerge(int mm) 已过时。
void	setMaxOverlapsToSideline(int mo) 已过时。
void	setRemoveParents(boolean shouldFix) 已过时。
void	setRetCode(int code) 已过时。
void	setSidelineBigOverlaps(boolean sbo) 已过时。
void	setSidelineDir(String sidelineDir) 已过时。
void	setTimeLag(long seconds) 已过时。 We are interested in only those tables that have not changed their state in hbase:meta during the last few seconds specified by hbase.admin.fsck.timelag
static boolean	shouldDisplayFullReport() 已过时。
boolean	shouldFixVersionFile() 已过时。
boolean	shouldIgnorePreCheckPermission() 已过时。
boolean	shouldRerun() 已过时。
boolean	shouldSidelineBigOverlaps() 已过时。

从类继承的方法 org.apache.hadoop.conf.Configured

getConf, setConf

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

DEFAULT_TIME_LAG

public static final long DEFAULT_TIME_LAG

已过时。

另请参阅:

常量字段值

DEFAULT_SLEEP_BEFORE_RERUN

public static final long DEFAULT_SLEEP_BEFORE_RERUN

已过时。

另请参阅:

常量字段值

HBCK_LOCK_FILE

public static final String HBCK_LOCK_FILE

已过时。

Here is where hbase-1.x used to default the lock for hbck1. It puts in place a lock when it goes to write/make changes.

另请参阅:

常量字段值

executor

protected ExecutorService executor

已过时。

构造器详细资料

HBaseFsck

public HBaseFsck(org.apache.hadoop.conf.Configuration conf)
          throws IOException,
                 ClassNotFoundException

已过时。

Constructor

参数:

conf - Configuration object

抛出:

MasterNotRunningException - if the master is not running

ZooKeeperConnectionException - if unable to connect to ZooKeeper

IOException

ClassNotFoundException

HBaseFsck

public HBaseFsck(org.apache.hadoop.conf.Configuration conf,
                 ExecutorService exec)
          throws MasterNotRunningException,
                 ZooKeeperConnectionException,
                 IOException,
                 ClassNotFoundException

已过时。

Constructor

参数:

conf - Configuration object

抛出:

MasterNotRunningException - if the master is not running

ZooKeeperConnectionException - if unable to connect to ZooKeeper

IOException

ClassNotFoundException

方法详细资料

createLockRetryCounterFactory

public static RetryCounterFactory createLockRetryCounterFactory(org.apache.hadoop.conf.Configuration conf)

已过时。

返回:

A retry counter factory configured for retrying lock file creation.

getTmpDir

public static org.apache.hadoop.fs.Path getTmpDir(org.apache.hadoop.conf.Configuration conf)
                                           throws IOException

已过时。

返回:

Return the tmp dir this tool writes too.

抛出:

IOException

checkAndMarkRunningHbck

public static Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.FSDataOutputStream> checkAndMarkRunningHbck(org.apache.hadoop.conf.Configuration conf,
                                                                                                              RetryCounter retryCounter)
                                                                                                       throws IOException

已过时。

This method maintains a lock using a file. If the creation fails we return null

返回:

FSDataOutputStream object corresponding to the newly opened lock file

抛出:

IOException - if IO failure occurs

connect

public void connect()
             throws IOException

已过时。

To repair region consistency, one must call connect() in order to repair online state.

抛出:

IOException

offlineHdfsIntegrityRepair

public void offlineHdfsIntegrityRepair()
                                throws IOException,
                                       InterruptedException

已过时。

This repair method analyzes hbase data in hdfs and repairs it to satisfy the table integrity rules. HBase doesn't need to be online for this operation to work.

抛出:

IOException

InterruptedException

onlineConsistencyRepair

public int onlineConsistencyRepair()
                            throws IOException,
                                   org.apache.zookeeper.KeeperException,
                                   InterruptedException

已过时。

This repair method requires the cluster to be online since it contacts region servers and the masters. It makes each region's state in HDFS, in hbase:meta, and deployments consistent.

返回:

If > 0 , number of errors detected, if < 0 there was an unrecoverable error. If 0, we have a clean hbase.

抛出:

IOException

org.apache.zookeeper.KeeperException

InterruptedException

onlineHbck

public int onlineHbck()
               throws IOException,
                      org.apache.zookeeper.KeeperException,
                      InterruptedException,
                      ReplicationException

已过时。

Contacts the master and prints out cluster-wide information

返回:

0 on success, non-zero on failure

抛出:

IOException

org.apache.zookeeper.KeeperException

InterruptedException

ReplicationException

keyOnly

public static byte[] keyOnly(byte[] b)

已过时。

close

public void close()
           throws IOException

已过时。

指定者:

close 在接口中 Closeable

指定者:

close 在接口中 AutoCloseable

抛出:

IOException

checkRegionBoundaries

public void checkRegionBoundaries()

已过时。

getErrors

public HbckErrorReporter getErrors()

已过时。

fixEmptyMetaCells

public void fixEmptyMetaCells()
                       throws IOException

已过时。

To fix the empty REGIONINFO_QUALIFIER rows from hbase:meta

抛出:

IOException

fixOrphanTables

public void fixOrphanTables()
                     throws IOException

已过时。

To fix orphan table by creating a .tableinfo file under tableDir
1. if TableInfo is cached, to recover the .tableinfo accordingly
2. else create a default .tableinfo file with following items
2.1 the correct tablename
2.2 the correct colfamily list
2.3 the default properties for both TableDescriptor and ColumnFamilyDescriptor

抛出:

IOException

loadHdfsRegionDirs

public void loadHdfsRegionDirs()
                        throws IOException,
                               InterruptedException

已过时。

Scan HDFS for all regions, recording their information into regionInfoMap

抛出:

IOException

InterruptedException

mergeRegionDirs

public int mergeRegionDirs(org.apache.hadoop.fs.Path targetRegionDir,
                           HbckRegionInfo contained)
                    throws IOException

已过时。

Merge hdfs data by moving from contained HbckRegionInfo into targetRegionDir.

返回:

number of file move fixes done to merge regions.

抛出:

IOException

setDisplayFullReport

public static void setDisplayFullReport()

已过时。

Display the full report from fsck. This displays all live and dead region servers, and all known regions.

shouldDisplayFullReport

public static boolean shouldDisplayFullReport()

已过时。

setForceExclusive

public static void setForceExclusive()

已过时。

Set exclusive mode.

isExclusive

public boolean isExclusive()

已过时。

Only one instance of hbck can modify HBase at a time.

setFixReplication

public void setFixReplication(boolean shouldFix)

已过时。

Set replication fix mode.

setCleanReplicationBarrier

public void setCleanReplicationBarrier(boolean shouldClean)

已过时。

shouldRerun

public boolean shouldRerun()

已过时。

setFixAssignments

public void setFixAssignments(boolean shouldFix)

已过时。

Fix inconsistencies found by fsck. This should try to fix errors (if any) found by fsck utility.

setFixMeta

public void setFixMeta(boolean shouldFix)

已过时。

setFixEmptyMetaCells

public void setFixEmptyMetaCells(boolean shouldFix)

已过时。

setCheckHdfs

public void setCheckHdfs(boolean checking)

已过时。

setFixHdfsHoles

public void setFixHdfsHoles(boolean shouldFix)

已过时。

setFixTableOrphans

public void setFixTableOrphans(boolean shouldFix)

已过时。

setFixHdfsOverlaps

public void setFixHdfsOverlaps(boolean shouldFix)

已过时。

setFixHdfsOrphans

public void setFixHdfsOrphans(boolean shouldFix)

已过时。

setFixVersionFile

public void setFixVersionFile(boolean shouldFix)

已过时。

shouldFixVersionFile

public boolean shouldFixVersionFile()

已过时。

setSidelineBigOverlaps

public void setSidelineBigOverlaps(boolean sbo)

已过时。

shouldSidelineBigOverlaps

public boolean shouldSidelineBigOverlaps()

已过时。

setFixSplitParents

public void setFixSplitParents(boolean shouldFix)

已过时。

setRemoveParents

public void setRemoveParents(boolean shouldFix)

已过时。

setFixReferenceFiles

public void setFixReferenceFiles(boolean shouldFix)

已过时。

setFixHFileLinks

public void setFixHFileLinks(boolean shouldFix)

已过时。

shouldIgnorePreCheckPermission

public boolean shouldIgnorePreCheckPermission()

已过时。

setIgnorePreCheckPermission

public void setIgnorePreCheckPermission(boolean ignorePreCheckPermission)

已过时。

setMaxMerge

public void setMaxMerge(int mm)

已过时。

参数:

mm - maximum number of regions to merge into a single region.

getMaxMerge

public int getMaxMerge()

已过时。

setMaxOverlapsToSideline

public void setMaxOverlapsToSideline(int mo)

已过时。

getMaxOverlapsToSideline

public int getMaxOverlapsToSideline()

已过时。

includeTable

public void includeTable(TableName table)

已过时。

setTimeLag

public void setTimeLag(long seconds)

已过时。

We are interested in only those tables that have not changed their state in hbase:meta during the last few seconds specified by hbase.admin.fsck.timelag

参数:

seconds - - the time in seconds

setSidelineDir

public void setSidelineDir(String sidelineDir)

已过时。

参数:

sidelineDir - - HDFS path to sideline data

createHFileCorruptionChecker

protected HFileCorruptionChecker createHFileCorruptionChecker(boolean sidelineCorruptHFiles)
                                                       throws IOException

已过时。

抛出:

IOException

getHFilecorruptionChecker

public HFileCorruptionChecker getHFilecorruptionChecker()

已过时。

setHFileCorruptionChecker

public void setHFileCorruptionChecker(HFileCorruptionChecker hfcc)

已过时。

setRetCode

public void setRetCode(int code)

已过时。

getRetCode

public int getRetCode()

已过时。

printUsageAndExit

protected HBaseFsck printUsageAndExit()

已过时。

main

public static void main(String[] args)
                 throws Exception

已过时。

Main program

参数:

args -

抛出:

Exception

exec

public HBaseFsck exec(ExecutorService exec,
                      String[] args)
               throws org.apache.zookeeper.KeeperException,
                      IOException,
                      InterruptedException,
                      ReplicationException

已过时。

抛出:

org.apache.zookeeper.KeeperException

IOException

InterruptedException

ReplicationException

setCleanReplicationBarrierTable

public void setCleanReplicationBarrierTable(String cleanReplicationBarrierTable)

已过时。

cleanReplicationBarrier

public void cleanReplicationBarrier()
                             throws IOException

已过时。

抛出:

IOException

debugLsr

public static void debugLsr(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.Path p)
                     throws IOException

已过时。

ls -r for debugging purposes

抛出:

IOException

debugLsr

public static void debugLsr(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.Path p,
                            HbckErrorReporter errors)
                     throws IOException

已过时。

ls -r for debugging purposes

抛出:

IOException