我应该在HBase中使用prefixfilter还是rowkey范围扫描 | 在HBase中使用prefixfilter还是rowkey范围扫

本文介绍了我应该在HBase中使用prefixfilter还是rowkey范围扫描的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我不知道为什么如果我使用prefixfilter查询，速度很慢。有人可以解释哪个是查询HBase的最佳方法，谢谢。

  hbase（main）：002：0>扫描'userlib'，{FILTER => org.apache.hadoop.hbase.filter.PrefixFilter.new（org.apache.hadoop.hbase.util.Bytes.toBytes（'0000115831F8'））} 
 ROW COLUMN + CELL 
 0000115831F8001 column = track：aid，timestamp = 1339121507633，value = aaa 
 1行（s）在41.0700秒
 
 hbase（main）：002：0> scan'userlib'，{STARTROW =>'0000115831F8'，ENDROW =>'0000115831F9'} 
 ROW COLUMN + CELL 
 0000115831F8001 column = track：aid，timestamp = 1339121507633，value = aaa 
 1行（0.1100秒）

解决方案

HBase过滤器 - 甚至行过滤器 - 真的很慢，因为在大多数情况下，这些过滤器会执行完整的表扫描，然后过滤这些结果。看看这个讨论：

行键范围扫描然而，确实快得多 - 他们做了相当于过滤的表扫描。这是因为行键以排序顺序存储（这是HBase的基本保证之一，这是一个类似于BigTable的解决方案），所以行键范围扫描速度非常快。更多解释在这里：

$ b [UPDATE 1] PrefixFilter的确如此做一个全表扫描，直到它传递过滤器中使用的前缀（如果它找到它）。使用PrefixFilter的快速性能建议似乎是为PrefixFilter另外指定一个start_row参数。请参阅。

[更新2，来自@ aaa90210]关于上述更新，现在有一个比PrefixFilter快得多的高效行前缀过滤器，请参阅此答案：

I don't know why it's very slow if I use prefixfilter to query. Can someone explain which is the best way to query HBase, thanks.
hbase(main):002:0> scan 'userlib',{FILTER=>org.apache.hadoop.hbase.filter.PrefixFilter.new(org.apache.hadoop.hbase.util.Bytes.toBytes('0000115831F8'))} ROW COLUMN+CELL 0000115831F8001 column=track:aid, timestamp=1339121507633, value=aaa 1 row(s) in 41.0700 seconds hbase(main):002:0> scan 'userlib',{STARTROW=>'0000115831F8',ENDROW=>'0000115831F9'} ROW COLUMN+CELL 0000115831F8001 column=track:aid, timestamp=1339121507633, value=aaa 1 row(s) in 0.1100 seconds
解决方案
HBase filters - even row filters - are really slow, since in most cases these do a complete table scan, and then filter on those results. Have a look at this discussion: http://grokbase.com/p/hbase/user/115cg0d7jh/very-slow-scan-performance-using-filters
Row key range scans however, are indeed much faster - they do the equivalent of a filtered table scan. This is because the row keys are stored in sorted order (this is one of the basic guarantees of HBase, which is a BigTable-like solution), so the range scans on row keys are very fast. More explanation here: http://www.quora.com/How-feasible-is-real-time-querying-on-HBase-Can-it-be-achieved-through-a-programming-language-such-as-Python-PHP-or-JSP
[UPDATE 1] turns out that PrefixFilter does do a full table scan until it passes the prefix used in the filter (if it finds it). The recommendation for fast performance using a PrefixFilter seems to be to specify a start_row parameter in addition to the PrefixFilter. See related 2013 discussion on the hbase-user mailing list.
[UPDATE 2, from @aaa90210] In regards to above update, there is now an efficient row prefix filter that is much faster than PrefixFilter, see this answer: https://stackoverflow.com/a/38632100/150050

这篇关于我应该在HBase中使用prefixfilter还是rowkey范围扫描的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！