Resolve “Orphan region in HDFS: Unable to load .regioninfo from table” in Hbase

“Orphan region in HDFS: Unable to load .regioninfo from table” usually happens, When “.regioninfo” is unavailable under the HDFS table location. This could be due to manual deletion or corruption

Symptoms:

“Orphan region in HDFS: Unable to load .regioninfo from table” might cause Hbase job failures, Where the Client unable to connect and access this particular region and its data

You can confirm the issue by spotting the below error message in the “hbase hbck” output:

ERROR: Orphan region in HDFS: Unable to load .regioninfo from table ns:testing_table in hdfs dir hdfs://nameservice/hbase/data/ns/testing_table/df96hksd388udb231605690827273c1  It may be an invalid format or version file.  Treating as an orphaned regiondir."

The command used to get the “hbase hbck” output to validate inconsistencies

hbase hbck -details 2>&1 | tee /tmp/hbase-hbck.txt

Resolution:

To fix the above inconsistencies in Hbase, We need to do the below steps

Sideline this orphan dir to HDFS /tmp location

• mkdir a backup dir on HDFS  by running “hdfs dfs -mkdir /tmp/backup_tbl”
• then run “hdfs dfs -mv hdfs://nameservice/hbase/data/ns/testing_table/df96hksd3882udb231605690827273c1 /tmp/backup_tbl”

Once we have the backup in the temp location, We can replay the WAL edits manually to load data into the table.

Hbase WALPlayer

Hbase has an internal utility called “WALPlayer” Which has multiple uses, But in this case, We are using this utility to replay the content of a Regions “recovered.edits” directory

NOTE: Files under the “recovered.edits” will be in the same format as of WAL, 

Now, We are going to use the backup data dir (it will have recovered.edits), and with the help of the backup will able to recover the region data for this table

Example Syntax:

#hbase org.apache.hadoop.hbase.mapreduce.WALPlayer /tmp/backup_tbl ns:testing_table

The above command will trigger a Map Reduce job to complete the task, Post that you can able to see the table data in the hdfs location with “.regioninfo” file

hdfs://nameservice/hbase/data/ns/testing_table/df96hksd3882udb231605690827273c1

To confirm, Run the “hbase hbck” command and check if the inconsistency has been resolved 🙂

Good Luck with your Learning !!

Related Topics:

“hole in the region chain” issue in Hbase

“org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Unable to accept edit”

“Regions not on HDFS or in hbase:meta but deployed on” issue in Hbase

“org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under” in Hbase