无法在不阻塞DigitalOcean的情况下运行多个节点子进程

问题描述

我很努力地在DigitalOcean上运行多个Puppeteer实例，但运气不好.我可以使用 puppeteer-cluster 之类的工具同时运行〜5几乎没有任何有用的消息传递，整个过程令人窒息.因此，我切换为在没有任何其他库的情况下生成〜5个子进程-仅是Puppeteer本身.同样的问题.窒息而没有有用的错误.

I've been struggling to run multiple instances of Puppeteer on DigitalOcean for quite some time with little luck. I'm able to run ~5 concurrently using tools like puppeteer-cluster, but for some reason the whole thing just chokes with little helpful messaging. So, I switched to spawning ~5 child processes without any additional library -- just Puppeteer itself. Same issue. Chokes with no helpful errors.

我能够在本地很好地运行所有这些作业，但是在部署之后，我碰壁了.因此，我的直觉是这是资源/性能问题，但我不能肯定地说.

I'm able to run all of these jobs just fine locally, but after I deploy, I hit these walls. So, my hunch is that it's a resource/performance issue, but I can't say for sure.

我正在Digital Ocean上运行具有1GB和3CPU的Droplet.

I'm running a droplet with 1GB and 3CPUs on Digital Ocean.

基本上，我只是在寻找开始对此类问题进行故障排除的方法.有什么方法可以确定我是否遇到了资源短缺问题?我已经尝试过pm2和DO仪表盘图，但是我觉得这些都遗漏了很多信息，否则我就完全没了其他东西.

Basically, I'm just looking for ways to start troubleshooting something like this. is there a way I can know for sure that I'm hitting resource walls? I've tried pm2 and the DO dashboard graphs, but I feel like those are all leaving a lot of information out, or else I'm missing something else altogether.

推荐答案

此处是puppeteer-cluster的作者.没错，除了操作系统甚至其他后台任务之外，1 GB的内存可能不足以运行5个浏览器窗口(或选项卡).

Author of puppeteer-cluster here. You are right, 1 GB of memory is likely not enough for running 5 browser windows (or tabs) in addition to your operating system and maybe even other background tasks.

以下是您应检查的资源列表:

Here is a list of resources you should check:

• 内存:使用类似 htop 检查应用程序运行时的内存使用情况.
• CPU :同样，您可以使用htop，三个vCPU足以容纳5个窗口.
• 磁盘空间:使用df之类的工具检查磁盘上是否有足够的空间.我知道在很多情况下磁盘上没有足够的空间(例如一些旧的内核填充了磁盘)，而Chrome需要至少 some 个空间才能运行.
• 网络吞吐量:问题很少，但有时网络只是没有带宽来支持许多打开的浏览器.使用nload之类的工具来检查网络吞吐量.

• Memory: Use a tool like htop to check your memory usage while your application is running.
• CPU: Again, you can use htop for that, 3 vCPUs should be more than enough for 5 windows.
• Disk space: Use a tool like df to check if there is enough space on the disk. I know of multiple cases in which there was not enough space on the disk (like some old kernels filling the disk), and Chrome needs at least some space to run.
• Network throughput: Rarely the problem, but sometimes the network just does not have the bandwidth to support many open browser. Use a tool like nload to check the network throughput.

要使用htop或nload，请在后台启动脚本(node script.js &)或使用终端多路复用器(如 tmux ).这样一来，资源问题就应该容易发现.

To use htop or nload, you start your script in the background (node script.js &) or use a terminal multiplexer (like tmux). Resource problems should then be easy to spot.

这篇关于无法在不阻塞DigitalOcean的情况下运行多个节点子进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！