问题描述
我在一个相当繁忙的网站工作，该网站经常会出现非常大的流量高峰.在这些峰值期间，每秒请求数百个页面，这会产生随机的 502 网关错误.
I work for a rather busy internet site that is often gets very large spikes of traffic. During these spikes hundreds of pages per second are requested and this produces random 502 gateway errors.
现在我们在配备 4 个 SAS 15k 驱动器 (raid10)、16 核 CPU 和 24GB DDR3 内存的机器上运行 Nginx (1.0.10) 和 PHP-FPM.我们还使用了最新的 Xcache 版本.DB在另一台机器上，但是这台机器的负载很低，没有问题.
Now we run Nginx (1.0.10) and PHP-FPM on a machine with 4x SAS 15k drives (raid10) with a 16 core CPU and 24GB of DDR3 ram. Also we make use of the latest Xcache version. The DB is located on another machine, but this machine's load is very low, and has no issues.
在正常负载下一切运行完美，系统负载低于 1，PHP-FPM 状态报告一次从未真正显示超过 10 个活动进程.始终有大约 10GB 的内存可用.在正常负载下，机器每秒处理大约 100 次网页浏览.
Under normal load everything runs perfect, system load is below 1, and PHP-FPM status report never really shows more than 10 active processes at one time. There is always about 10GB of ram still available. Under normal load the machine handles about 100 pageviews per second.
当巨大的流量高峰到来时，就会出现问题，并且从机器请求每秒数百 次页面浏览.我注意到 FPM 的状态报告显示多达 50 个活动进程，但这仍远低于我们配置的 300 个最大连接数.在这些高峰期间，Nginx 状态报告多达 5000 个活动连接，而不是正常的 1000 个连接.
The problem arises when huge spikes of traffic arrive, and hundreds of page-views per second are requested from the machine. I notice that FPM's status report then shows up to 50 active processes, but that is still way below the 300 max connections that we have configured. During these spikes Nginx status reports up to 5000 active connections instead of the normal average of 1000.
操作系统信息:CentOS 5.7 版(最终版)
OS Info: CentOS release 5.7 (Final)
CPU:Intel(R) Xeon(R) CPU E5620 @ 2.40GH(16 核)
CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GH (16 cores)
php-fpm.conf
daemonize = yes
listen = /tmp/fpm.sock
pm = static
pm.max_children = 300
pm.max_requests = 1000
我还没有设置 rlimit_files，因为据我所知，如果你不设置，它应该使用系统默认值.
I have not setup rlimit_files, because as far as I know it should use the system default if you don't.
fastcgi_params(仅向标准文件添加值)
fastcgi_params (only added values to standard file)
fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on;
fastcgi_pass unix:/tmp/fpm.sock;
nginx.conf
worker_processes 8;
worker_connections 16384;
sendfile on;
tcp_nopush on;
keepalive_timeout 4;
Nginx 通过 Unix Socket 连接到 FPM.
Nginx connects to FPM via Unix Socket.
sysctl.conf
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 1
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.tcp_timestamps = 0
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.eth0.rp_filter=1
net.ipv4.conf.lo.rp_filter=1
net.ipv4.ip_conntrack_max = 100000
limits.conf
* soft nofile 65536
* hard nofile 65536
这些是以下命令的结果:
These are the results for the following commands:
ulimit -n
65536
ulimit -Sn
65536
ulimit -Hn
65536
cat /proc/sys/fs/file-max
2390143
问题:如果 PHP-FPM 没有耗尽连接，负载仍然很低，并且有足够的 RAM 可用，那么在高流量期间可能导致这些随机 502 网关错误的瓶颈是什么?
Question: If PHP-FPM is not running out of connections, the load is still low, and there is plenty of RAM available, what bottleneck could be causing these random 502 gateway errors during high traffic?
注意:默认情况下这台机器的 ulimit 是 1024，因为我把它改成了 65536 我还没有完全重启机器，因为它是一台生产机器，这意味着停机时间太多.
推荐答案
这应该可以解决它...
This should fix it...
你有:fastcgi_buffers 4 256k;
改成:fastcgi_buffers 256 16k;//总共 4096k
Change it to: fastcgi_buffers 256 16k; // 4096k total
同时设置 fastcgi_max_temp_file_size 0，如果回复开始超过您的 fastcgi 缓冲区，这将禁用缓冲到磁盘.
Also set fastcgi_max_temp_file_size 0, that will disable buffering to disk if replies start to exceeed your fastcgi buffers.
这篇关于502 高负载下的网关错误 (nginx/php-fpm)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！