Saturday, June 23, 2012

HBase fixing region missing Region //regionname// on HDFS but not listed in .META or deployed on any server

If you see this error on HBase

Region //regionname// on HDFS but not listed in .META or deployed on any server

then use the add_regions.rb script to have this fixed. 

Normally hbase errors can be fixed by hbase hbck -fix
But this approach works mostly in cases where the region is listed in .META but not assigned etc.

In such cases, the script add_regions.rb comes to our rescue. It can be used as follows:

hbase org.jruby.Main add_regions.rb //full-path-region-on-HDFS/.regioninfo

This will add the region to the .META. table. Next assign the region to a region-server. To do this , launch hbase shell and issue the command

assign 'full-region-name'

on the hbase shell.

The full region-name includes the standard format tablename.startrowkey.timestamp.md5sum.

The dot at the end is important and should be included while passing to the assign command.

Now run hbase hbck again and look at status 'simple' on the shell. Your region numbers should have increased according to the regions you have added now. The errors related to not listed in .META should be gone now.

Tuesday, June 12, 2012

Python copy absolute value of array elements into another array


In the array below, some values are positive, while others are negative.

>>> x = [-538,-181,-145,552,-847,6,141,-58,-122,314,-816,245,594,-613,-287,1232,-1479,-326,-197,715,4,-677,95,308,-1224,953,-81,-189,341,-654,242,-948,1088,-533,-328,123,552,-855,49,-443,-37,-57,199,56,-459,-47,-167,13,-521,476,-161,440,-540,180,43,-57,-236,-29,-830,265,-2,-379,-9,198,12,79,-257,113]
>>>


I want a sum total of the array after conversion to their absolute values.

>>> values = [abs(x[n]) for n in range(1,len(x))]
>>> values
[181, 145, 552, 847, 6, 141, 58, 122, 314, 816, 245, 594, 613, 287, 1232, 1479, 326, 197, 715, 4, 677, 95, 308, 1224, 953, 81, 189, 341, 654, 242, 948, 1088, 533, 328, 123, 552, 855, 49, 443, 37, 57, 199, 56, 459, 47, 167, 13, 521, 476, 161, 440, 540, 180, 43, 57, 236, 29, 830, 265, 2, 379, 9, 198, 12, 79, 257, 113]
>>> print sum (values)
24419


Thursday, June 07, 2012

Extract each archive into its own directory


Assuming the archives are all bzipped archives with .tbz2

for i in `ls`; do a=`basename $i .tbz2`; mkdir $a; cd $a;tar -xvf ../$i;cd ..; done