问题描述
我们使用具有8核32GB RAM的mongodb 3.4.14.我正在用Jmeter执行负载测试，有70个线程，我有可接受的输出.但是，随着负载的增加，SLA呈指数增长，吞吐量急剧下降.我尝试增加ulimit，并且下一步是分片，除此之外，还有其他我可以做的性能优化吗?
We use mongodb 3.4.14 with 8 core, 32GB RAM. I was performing the load test with Jmeter, with 70 threads I have acceptable output. But as the load increases SLA is exponentially increasing and throughput reduces drastically. I tried increasing the ulimit and sharding is the next step, apart from that is there any other performance optimization that I can do ?
@Jetet，这是发现的地方:
@Jeet, here are the findings :
- 是否有很多聚合查询?您拥有哪种收集结构，即
负载测试在单个聚合查询上运行，并且文档的结构也具有相同的字段集.固定文档大小会有所帮助吗?我该怎么办?
The load test is run on a single aggregation query and the structure of the document is also having same set of fields. Fixing the document size would help ? how can I do it?
- 是否有很多嵌套数组?
答案:没有嵌套查询.
- 是单个实例还是副本集?尝试将副本集放入具有不同节点的读写权限.
当前，我们只希望在单个节点上运行.
Currently we want to run only on single node.
- 查询是否从多个集合中返回数据?
否，只有1个收藏集.
- 检查您的实例是页面错误的操作百分比是多少?
在有500个用户的情况下，我看不到太多的页面错误，只有2位数字.
With a load of 500 users I don't see much page faults, only 2 digit numbers.
- 在高锁定/队列期间检查日志中是否具有高nscanned或scanAndOrder的操作，并相应地建立索引.
如何检查?
- 检查查询中是否有CPU密集型运算符，例如$ all，$ push/$ pop/$ addToSet，以及对大型文档的更新，尤其是对具有大型数组(或大型子文档数组)的文档的更新.
是，在上述负载下，CPU已满，并且响应被延迟.我们正在进行groupBy，然后进行限制排序.
Yes, with the above load CPU is full and responses are delayed. We are doing a groupBy and then sorting with limit.
- 如果您的数据库是大量写入操作，请记住，每个数据库一次只能写入一个CPU(由于该线程持有写入锁定).考虑将部分数据移至其自己的数据库中.
我们的数据库通常读得很重，该馆藏每天都会填充一次.
Our database is mostly read heavy, the collection will be populated once a day.
除此之外，我还尝试通过将以下代码放入for循环中来做一个简单的测试:
Apart from this I tried to do a simple test by putting the below code in a for loop :
Document findQuery = new Document("userId", "Sham");
FindIterable<Document> find = collection.find(findQuery);
MongoCursor<Document> iterator = find.iterator();
使用执行程序启动该过程:
Used executor to start the process:
ExecutorService executorService = Executors.newFixedThreadPool(100);
即使执行此操作，它的返回速度也很慢，大约需要900毫秒.
even with this the performance is slow its taking like 900ms to return.
1个请求=每个请求150ms
1 request = 150ms per request
100个请求=每个请求900ms
100 request = 900ms per request
当我看到500个用户的统计信息如下时:
when I see the stats its as below for 500 users:
insert query update delete getmore command dirty used flushes vsize res qrw arw net_in net_out conn time
*0 *0 *0 *0 0 1|0 0.0% 0.0% 0 317M 28.0M 0|0 0|0 156b 45.1k 3 Oct 12 15:31:19.644
*0 *0 *0 *0 0 1|0 0.0% 0.0% 0 317M 28.0M 0|0 0|0 156b 45.1k 3 Oct 12 15:31:20.650
*0 *0 *0 *0 0 3|0 0.0% 0.0% 0 317M 28.0M 0|0 0|0 218b 46.1k 3 Oct 12 15:31:21.638
*0 *0 *0 *0 0 2|0 0.0% 0.0% 0 317M 28.0M 0|0 0|0 158b 45.4k 3 Oct 12 15:31:22.638
*0 *0 *0 *0 0 1|0 0.0% 0.0% 0 317M 28.0M 0|0 0|0 157b 45.4k 3 Oct 12 15:31:23.638
*0 376 *0 *0 0 112|0 0.0% 0.0% 0 340M 30.0M 0|0 0|0 64.9k 23.6m 26 Oct 12 15:31:24.724
*0 98 *0 *0 0 531|0 0.0% 0.0% 0 317M 27.0M 0|0 0|0 109k 6.38m 3 Oct 12 15:31:25.646
*0 *0 *0 *0 0 2|0 0.0% 0.0% 0 317M 27.0M 0|0 0|0 215b 45.6k 3 Oct 12 15:31:26.646
*0 *0 *0 *0 0 1|0 0.0% 0.0% 0 317M 27.0M 0|0 0|0 157b 45.1k 3 Oct 12 15:31:27.651
*0 *0 *0 *0 0 2|0 0.0% 0.0% 0 317M 27.0M 0|0 0|0 159b 45.8k 3 Oct 12 15:31:28.642
推荐答案
这也取决于您要触发的查询类型，请检查是否存在以下提到的要点-
This also depends on the kind of queries you are firing, Please check if below mentioned points are there -
- 是否有很多聚合查询?什么样的收藏你有没有结构
- 是否有很多嵌套数组?
- 是不是单一实例或副本集?尝试将副本集与read一起放置并写入不同的节点.
- 查询是否从中返回数据多个收藏?
- 检查您的实例是页面错误的操作百分比是多少?
- 在高锁定/队列期间检查日志中是否具有高nscanned或scanAndOrder的操作，并相应地建立索引.
- 检查查询中是否有CPU密集型运算符，例如$ all，$ push/$ pop/$ addToSet，以及对大型文档的更新，尤其是对具有大型数组(或大型子文档数组)的文档的更新.
- 如果您的数据库是大量写入操作，请记住，每个数据库一次只能写入一个CPU(由于该线程持有写入锁定).考虑将部分数据移到其自己的数据库中.
- is there lot of aggregation queries? What kind of collectionstructure do you have i.e
- is there a lot of nested arrays?
- Is it asingle instance or replica set? Try putting a replica set with readand write to different node.
- Are the queries returning data frommultiple collections?
- Check your instance is page-faulting for how much % of operations?
- Check your logs for operations with high nscanned or scanAndOrder during periods of high lock/queue, and index accordingly.
- Check your queries for CPU-intensive operators like $all, $push/$pop/$addToSet, as well as updates to large documents, and especially updates to documents with large arrays (or large subdocument arrays).
- if your database is write-heavy, keep in mind that only one CPU per database can write at a time (owing to that thread holding the write lock). Consider moving part of that data into its own database.
这是随着时间的推移会降低性能的几件事.我在这里介绍了最常见的用例，但是请查看此帖子以了解更多信息见解.
This are few things which decreases performance over the time. I have covered most common use cases here however please check this post for some more insights.
这篇关于MongoDB在负载下执行速度很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！