hbase的四种压缩格式-创新互联
Hbase支持的压缩格式:
在彭水苗族土家族等地区,都构建了全面的区域性战略布局,加强发展的系统性、市场前瞻性、产品创新能力,以专注、极致的服务理念,为客户提供网站建设、成都网站建设 网站设计制作按需开发网站,公司网站建设,企业网站建设,成都品牌网站建设,网络营销推广,成都外贸网站建设公司,彭水苗族土家族网站建设费用合理。hbase支持的压缩格式:GZ(GZIP),LZ0,LZ4,Snappy
GZ:用于冷数据压缩,与Snappy和LZ0相比,GZIP的压缩率更高,但是更消耗CPU,解压/压缩速度更慢。
Snappy和LZ0:用于热数据压缩,占用CPU少,解压/压缩速度比GZ快,但是压缩率不如GZ高。
Snappy与LZ0相比,Snappy整体性能优于LZ0,Snappy压缩率比LZ0更低,但是解压/压缩速度更快。
LZ4与LZ0相比,LZ4的压缩率和LZ0的压缩率相差不多,但是LZ4的解压/压缩速度更快。
多数情况下,选择Snppy或LZ0是比较好的选择,因为它们的压缩开销底,能节省空间。
建表时指定压缩格式
hbase(main):013:0> create 'test3',{NAME=>'f1'},{NAME=>'f2',COMPRESSION=>'Snappy'} 0 row(s) in 1.2740 seconds => Hbase::Table - test3 hbase(main):014:0> desc 'test3' Table test3 is ENABLED test3 COLUMN FAMILIES DESCRIPTION {NAME => 'f1', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'NONE', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} {NAME => 'f2', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'SNAPPY', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} 2 row(s) in 0.0300 seconds hbase(main):002:0> create 'test4' ,{NAME=>'f1'},{NAME=>'f2',COMPRESSION=>'GZ'} 0 row(s) in 1.4900 seconds => Hbase::Table - test4 hbase(main):003:0> desc 'test4' Table test4 is ENABLED test4 COLUMN FAMILIES DESCRIPTION {NAME => 'f1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} {NAME => 'f2', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'GZ', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2 row(s) in 0.1290 seconds
建表后修改columnfamily压缩格式
正确做法是先disable表,再修改列族压缩格式,enbale表后做major_compact操作。
如下:
hbase(main):004:0> desc 'test1' Table test1 is ENABLED test1 COLUMN FAMILIES DESCRIPTION {NAME => 'f1', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'NONE', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} {NAME => 'f2', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'NONE', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} 2 row(s) in 0.0230 seconds hbase(main):005:0> disable 'test1' 0 row(s) in 2.2870 seconds hbase(main):006:0> alter 'test1' ,{NAME=>'f1',COMPRESSION=>'Snappy'} Updating all regions with the new schema... 1/1 regions updated. Done. 0 row(s) in 1.9510 seconds hbase(main):007:0> enable 'test1' 0 row(s) in 1.2820 seconds hbase(main):008:0> desc 'test1' Table test1 is ENABLED test1 COLUMN FAMILIES DESCRIPTION {NAME => 'f1', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'SNAPPY', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} {NAME => 'f2', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'NONE', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} 2 row(s) in 0.0310 seconds hbase(main):009:0> major_compact 'test1' 0 row(s) in 0.1380 seconds hbase(main):010:0> desc 'test1' Table test1 is ENABLED test1 COLUMN FAMILIES DESCRIPTION {NAME => 'f1', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'SNAPPY', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} {NAME => 'f2', BLOOMFILTER => 'ROW',VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE',DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESS ION => 'NONE', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} 2 row(s) in 0.0260 seconds
但是没有disable表,也不做major_compact,列族压缩格式也修改成功了(暂时不知道原因)。
hbase(main):001:0> desc 'test' Table test is ENABLED test COLUMN FAMILIES DESCRIPTION {NAME => 'fam1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRE SSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 1 row(s) in 0.3680 seconds hbase(main):002:0> alter 'test',{NAME=>'fam1',COMPRESSION=>'LZ4'} Updating all regions with the new schema... 1/1 regions updated. Done. 0 row(s) in 2.0460 seconds hbase(main):003:0> desc 'test' Table test is ENABLED test COLUMN FAMILIES DESCRIPTION {NAME => 'fam1', BLOOMFILTER =>'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS =>'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRE SSION => 'LZ4', MIN_VERSIONS => '0',BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>'0'} 1 row(s) in 0.0280 seconds
另外有需要云服务器可以了解下创新互联scvps.cn,海内外云服务器15元起步,三天无理由+7*72小时售后在线,公司持有idc许可证,提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案,具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势,专为企业上云打造定制,能够满足用户丰富、多元化的应用场景需求。
文章名称:hbase的四种压缩格式-创新互联
转载注明:http://scyanting.com/article/jcici.html