HBase自带的很多工具可用于管理、分析、修复和调试,这些工具一部分的入口是hbase shell 客户端,另一部分是在hbase的Jar包中。
目录:
hbckhfile数据备份与恢复 SnapshotsReplicationExportCopyTableHTable APIOffline backup of HDFS datahbck:
hbase hbck -help Usage: fsck [opts] {only tables} where [opts] are: -help Display help options (this) -details Display full report of all regions. -timelag <timeInSeconds> Process only regions that have not experienced any metadata updates in the last <timeInSeconds> seconds. -sleepBeforeRerun <timeInSeconds> Sleep this many seconds before checking if the fix worked if run with -fix -summary Print only summary of the tables and status. -metaonly Only check the state of the hbase:meta table. -sidelineDir <hdfs://> HDFS path to backup existing meta. -boundaries Verify that regions boundaries are the same between META and store files. -exclusive Abort if another hbck is exclusive or fixing. -disableBalancer Disable the load balancer.
Metadata Repair options: (expert features, use with caution!) -fix Try to fix region assignments. This is for backwards compatiblity -fixAssignments Try to fix region assignments. Replaces the old -fix -fixMeta Try to fix meta problems. This assumes HDFS region info is good. -noHdfsChecking Don't load/check region info from HDFS. Assumes hbase:meta region info is good. Won't check/fix any HDFS issue, e.g. hole, orphan, or overlap -fixHdfsHoles Try to fix region holes in hdfs. -fixHdfsOrphans Try to fix region dirs with no .regioninfo file in hdfs -fixTableOrphans Try to fix table dirs with no .tableinfo file in hdfs (online mode only) -fixHdfsOverlaps Try to fix region overlaps in hdfs. -fixVersionFile Try to fix missing hbase.version file in hdfs. -maxMerge <n> When fixing region overlaps, allow at most <n> regions to merge. (n=5 by default) -sidelineBigOverlaps When fixing region overlaps, allow to sideline big overlaps -maxOverlapsToSideline <n> When fixing region overlaps, allow at most <n> regions to sideline per group. (n=2 by default) -fixSplitParents Try to force offline split parents to be online. -ignorePreCheckPermission ignore filesystem permission pre-check -fixReferenceFiles Try to offline lingering reference store files -fixEmptyMetaCells Try to fix hbase:meta entries not referencing any region (empty REGIONINFO_QUALIFIER rows)
Datafile Repair options: (expert features, use with caution!) -checkCorruptHFiles Check all Hfiles by opening them to make sure they are valid -sidelineCorruptHFiles Quarantine corrupted HFiles. implies -checkCorruptHFiles
Metadata Repair shortcuts -repair Shortcut for -fixAssignments -fixMeta -fixHdfsHoles -fixHdfsOrphans -fixHdfsOverlaps -fixVersionFile -sidelineBigOverlaps -fixReferenceFiles -fixTableLocks -fixOrphanedTableZnodes -repairHoles Shortcut for -fixAssignments -fixMeta -fixHdfsHoles
Table lock options -fixTableLocks Deletes table locks held for a long time (hbase.table.lock.expire.ms, 10min by default)
Table Znode options -fixOrphanedTableZnodes Set table state in ZNode to disabled if table does not exists
命令: hbase hbck -details 显示所有Region的完整报告命令: hbase hbck -metaonly 只检测元数据表的状态,如下图:快捷修复命令:命令:hbase hbck -repair -ignorePreCheckPermission 命令:hbase hbck -repairHoles -ignorePreCheckPermission 应用示例,参见:HBase(三): Azure HDInsigt HBase表数据导入本地HBasehfile:
hbase hfile usage: HFile [-a] [-b] [-e] [-f <arg> | -r <arg>] [-h] [-k] [-m] [-p] [-s] [-v] [-w <arg>] -a,--checkfamily Enable family check -b,--printblocks Print block index meta data -e,--printkey Print keys -f,--file <arg> File to scan. Pass full-path; e.g. hdfs://a:9000/hbase/hbase:meta/12/34 -h,--printblockheaders Print block headers for each block. -k,--checkrow Enable row order check; looks for out-of-order keys -m,--printmeta Print meta data of file -p,--printkv Print key/value pairs -r,--region <arg> Region to scan. Pass region name; e.g. 'hbase:meta,,1' -s,--stats Print statistics -v,--verbose Verbose output; emits file and meta data delimiters -w,--seekToRow <arg> Seek to this row and print all the kvs for this row only
命令:hbase hfile -p -f /apps/hbase/data/data/default/PerTest/7685e6c39d1394d94e26cf5ddafb7f9f/d/3ef195ca65044eca93cfa147414b56c2效果如下图:数据备份与恢复:
snapshots:
命令:clone_snapshot 'snapPerTest','PerTest1' 根据快照clone新表(注:clone出来的新表不带数据副本)
ExportSnapshot tool 快照导出工具命令: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot 'snapPerTest' -copy-to /apps/hbase/data/zhu注意: 如果能访问到另一集群,则后面的地址可直接改为另一集群hdfs目录导出的文件结构如下Replication:
Export:
copyTable:
Offline backup of HDFS data:
参考:https://www.cnblogs.com/tgzhu/archive/2016/09/11/5836231.html