问题描述
我需要一个磁盘支持的Map结构来在Java应用程序中使用。它必须具有以下标准:
I need a disk backed Map structure to use in a Java app. It must have the following criteria:
- 能够存储数百万条记录(甚至数十亿)
- 快速查找 - 地图上的大部分操作将简单地查看密钥是否已经存在。这和上面的1是最重要的标准。应该有一个有效的内存缓存机制,用于经常使用的密钥。
- 持久但不需要事务性,可能会遇到一些失败。即可快速与磁盘同步,不需要进行事务处理。
- 能够存储简单的基本类型,但不需要存储序列化对象。
- 它不需要分发,即将在一台机器上运行。
- 简单的设置&
- 不需要关系查询
- Capable of storing millions of records (even billions)
- Fast lookup - the majority of operations on the Map will simply to see if a key already exists. This, and 1 above are the most important criteria. There should be an effective in memory caching mechanism for frequently used keys.
- Persistent, but does not need to be transactional, can live with some failure. i.e. happy to synch with disk periodically, and does not need to be transactional.
- Capable of storing simple primitive types - but I don't need to store serialised objects.
- It does not need to be distributed, i.e. will run all on one machine.
- Simple to set up & free to use.
- No relational queries required
记录键将是字符串或长整型。如上所述,读取将比写入更频繁,并且大多数读取将仅仅是检查密钥是否存在(即不需要读取密钥关联数据)。每个记录只会更新一次,记录不会被删除。
Records keys will be strings or longs. As described above reads will be much more frequent than writes, and the majority of reads will simply be to check if a key exists (i.e. will not need to read the keys associated data). Each record will be updated once only and records are not deleted.
我目前使用Bdb JE,但正在寻找其他选项。
I currently use Bdb JE but am seeking other options.
更新
自从我改进了现有BDB设置的查询性能后,对次级键的依赖。一些查询需要两个辅助键上的连接,并将它们组合成一个复合键,我删除了查找中的间接级别,可以很好地加快速度。
Have since improved query performance on my existing BDB setup by reducing the dependency on secondary keys. Some queries required a join on two secondary keys and by combining them into a composite key I removed a level of indirection in the lookup which speeds things up nicely.
推荐答案
我可能会使用本地数据库。就像或。我可以问这种方法有什么问题吗?你必须有一些理由来寻找替代方案。
I'd likely use a local database. Like say Bdb JE or HSQLDB. May I ask what is wrong with this approach? You must have some reason to be looking for alternatives.
回应评论:
作为问题的性能,我猜你已经在使用JDBC来处理这个可能值得尝试HSQLB并阅读有关。
这篇关于推荐快速可扩展持久化Map - Java的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!