我用整数键和字符串值测试了mapdb,以便在其中插入10000000个元素。我看到的是:
Processed 1.0E-5 percent of the data / time so far = 0 seconds
Processed 1.00001 percent of the data / time so far = 7 seconds
Processed 2.00001 percent of the data / time so far = 14 seconds
Processed 3.00001 percent of the data / time so far = 20 seconds
Processed 4.00001 percent of the data / time so far = 26 seconds
Processed 5.00001 percent of the data / time so far = 33 seconds
Processed 6.00001 percent of the data / time so far = 39 seconds
Processed 7.00001 percent of the data / time so far = 45 seconds
Processed 8.00001 percent of the data / time so far = 53 seconds
Processed 9.00001 percent of the data / time so far = 60 seconds
Processed 10.00001 percent of the data / time so far = 66 seconds
Processed 11.00001 percent of the data / time so far = 73 seconds
Processed 12.00001 percent of the data / time so far = 80 seconds
Processed 13.00001 percent of the data / time so far = 88 seconds
Processed 14.00001 percent of the data / time so far = 96 seconds
Processed 15.00001 percent of the data / time so far = 102 seconds
Processed 16.00001 percent of the data / time so far = 110 seconds
Processed 17.00001 percent of the data / time so far = 119 seconds
Processed 18.00001 percent of the data / time so far = 127 seconds
Processed 19.00001 percent of the data / time so far = 134 seconds
Processed 20.00001 percent of the data / time so far = 141 seconds
Processed 21.00001 percent of the data / time so far = 149 seconds
Processed 22.00001 percent of the data / time so far = 157 seconds
Processed 23.00001 percent of the data / time so far = 164 seconds
Processed 24.00001 percent of the data / time so far = 171 seconds
Processed 25.00001 percent of the data / time so far = 178 seconds
....
大约250万个实例在178秒内被放入地图。1000万美元大约是12分钟。
然后我切换到更复杂的值,速度急剧下降(将整个10000000个实例添加到映射中需要3-4天)。有人建议加快mapdb的插入速度吗?以前是否有与mabdb速度相关的经验/问题?
这里还有一个评估:http://kotek.net/blog/3G_map
更新:我使用了创建地图的通用过程。这是一个伪代码:
DB db = DBMaker.newFileDB()....;
... map = db.getHashMap(...);
loop (...) {
map.put(...);
}
db.commit();
最佳答案
这里是mapdb作者。
对于开始使用专用序列化程序,它们会更快:
map m=dbmaker.createHashMap(“a”).keyserizer(serializer.long).valueserializer(serializer.long).makeOrGet()
接下来对于导入,我建议使用带有treemap的数据泵。举个例子:
https://github.com/jankotek/MapDB/blob/master/src/test/java/examples/Huge_Insert.java