充分利用PERF用户空间堆栈信息

问题描述

我目前正在追查一些幻像I / O在一个PostgreSQL版本，我测试。这是一个多进程的服务器，它不是简单的磁盘关联的I / O背到一个特定的后端和查询。

I'm currently trying to track down some phantom I/O in a PostgreSQL build I'm testing. It's a multi-process server and it isn't simple to associate disk I/O back to a particular back-end and query.

我想的Linux的 PERF 工具将是理想的这个，但我努力捕捉块I / O性能计数器度量，并将它们与用户空间的活动联系起来。

I thought Linux's perf tool would be ideal for this, but I'm struggling to capture block I/O performance counter metrics and associate them with user-space activity.

这很容易记录块I / O请求和完成用，例如：

It's easy to record block I/O requests and completions with, eg:

sudo perf record -g -T -u postgres -e 'block:block_rq_*'

和用户空间的PID被记录，但有捕获没有内核或用户空间的堆栈，或者创建快照的用户空间进程的堆位（比如，查询文本）等，所以，当你有PID能力，你不知道什么过程在这一点上做的事情。刚 PERF的剧本像输出：

and the user-space pid is recorded, but there's no kernel or user-space stack captured, or ability to snapshot bits of the user-space process's heap (say, query text) etc. So while you have the pid, you don't know what the process was doing at that point. Just perf script output like:

postgres  7462 [002] 301125.113632: block:block_rq_issue: 8,0 W 0 () 208078848 + 1024 [postgres]

如果我在 -g 标志添加到 PERF纪录它会采取的内核的快照的叠层，但不捕获用户空间状态在内核捕获PERF事件。用户空间栈只到用户空间，如 LWLockRelease ， LWLockAcquire ，<$ C入口点$ C>的memcpy （mmap'd IO）， __ ___ GI libc_write 等

If I add the -g flag to perf record it'll take snapshots of the kernel stack, but doesn't capture user-space state for perf events captured in the kernel. The user-space stack only goes up to the entry-point from userspace, like LWLockRelease, LWLockAcquire, memcpy (mmap'd IO), __GI___libc_write, etc.

所以。任何提示吗？能够捕捉到的用户空间的快照的协议栈响应的内核的事件将是理想的。

So. Any tips? Being able to capture a snapshot of the user-space stack in response to kernel events would be ideal.

我在Fedora 19，3.11.3-201.fc19.x86_64，薛定谔的猫，与PERF版本3.10.9-200.fc19.x86_64。

I'm on Fedora 19, 3.11.3-201.fc19.x86_64, Schrödinger’s Cat, with perf version 3.10.9-200.fc19.x86_64.

推荐答案

OK，貌似有几个环节进行：

OK, looks like there are several parts to this:

• 我在x86_64，其中大多数发行版在默认情况下使用 -fomit-frame-pointer的构建和 PERF 不能按照无堆栈帧指针;

• I'm on x86_64, where most distros build with -fomit-frame-pointer by default, and perf can't follow the stack without frame pointers;

....除非它与 libunwind 支持内置一个较新的版本，在这种情况下，它支持 PERF记录-g矮。

.... unless it's a newer version built with libunwind support, in which case it supports perf record -g dwarf.

请参阅：

libunwind支持

• 。

• the patch adding libunwind support to Perf
• Debian bug 725075.
• linux perf: how to interpret and find hotspots

我在Fedora 18，但。因此，如果你分析code你的工作（这是很可能的堆栈溢出），重建与 -fno-省略帧指针和 -ggdb 。

I'm on Fedora 18, but the same issue applies. So if you're profiling code you're working on (as is likely on Stack Overflow), rebuild with -fno-omit-frame-pointer and -ggdb.

我登陆了重建 PERF ，因为我想能够比较的股票的RPM：

I landed up rebuilding perf because I wanted to be able to compare to the stock RPMs:

• sudo的yum的建设-DEP PERF


• 须藤荫安装yum-utils的rpmdevtools libunwind-DEVEL


• yumdownloader --source PERF 或下载相应的内核-.....的src.rpm SRPM


• rpmdev-setuptree


• 转-Uvh内核 - *的src.rpm


• CD $ HOME /的rpmbuild / SPECS


• 的rpmbuild -bp --target = $（uname -m输出）kernel.spec

• sudo yum build-dep perf
• sudo yum install yum-utils rpmdevtools libunwind-devel
• yumdownloader --source perf or download the appropriate kernel-.....src.rpm srpm
• rpmdev-setuptree
• rpm -Uvh kernel-*.src.rpm
• cd $HOME/rpmbuild/SPECS
• rpmbuild -bp --target=$(uname -m) kernel.spec

在这一点上你可以建立一个新的 PERF 如果你想要的：

At this point you can just build a new perf if you want:

• CD $ HOME /的rpmbuild / BUILD /内核 - * / Linux的 - * /工具/ PERF


• 制作

• cd $HOME/rpmbuild/BUILD/kernel-*/linux-*/tools/perf
• make

...我没有和测试更新后的 PERF 事实上确实捕获有用的堆栈，如果可用libunwind建造的。

... which I did and tested that the updated perf does in fact capture a useful stack if built with libunwind available.

您也可以建立一个新的转：

You can also build a new rpm:

• 编辑kernel.spec，取消行％定义buildid ... ，改变buildid喜欢的东西 .perfunwind 。注意它的％定义不是％定义。

• edit kernel.spec, uncomment the line %define buildid ..., change buildid to something like .perfunwind. Note it's %define not % define.

在同一规范文件中，查找：

In the same spec file, find:

%global perf_make \
make %{?_smp_mflags} -C tools/perf -s V=1 WERROR=0 NO_LIBUNWIND=1 HAVE_CPLUS_DEMANGLE=1 NO_GTK2=1 NO_LIBNUMA=1 NO_STRLCPY=1 prefix=%{_prefix}

和删除 NO_LIBUNWIND = 1

的rpmbuild -bb --without了--without熔点--without PAE --without调试--without DOC --without头--without debuginfo软--without bootwrapper --without with_vdso_install --with PERF kernel.spec 来产生新的 PERF 的RPM没有建立整个内核。或者，如果你愿意，忽略 - 没有你想要的内核的味道，在这种情况下，你还需要建立头，debuginfo软等

rpmbuild -bb --without up --without mp --without pae --without debug --without doc --without headers --without debuginfo --without bootwrapper --without with_vdso_install --with perf kernel.spec to produce new perf RPMs without building the whole kernel. Or if you want, omit the --without for the kernel flavour you want, in which case you'll also want to build headers, debuginfo, etc.

sudo的RPM -Uvh $ HOME /的rpmbuild / RPMS / x86_64的/ PERF - * fc19.x86_64.rpm

请参阅。

我所举报的问题Fedora的;他们不应该使用 NO_LIBUNWIND = 1 。请参见。

I've reported the issue to Fedora; they shouldn't be using NO_LIBUNWIND=1. See bug 1025603.

一旦你有一个重建 PERF 您可以使用 PERF记录-g侏儒来获得完整的堆栈。

Once you have a rebuilt perf you can use perf record -g dwarf to get full stacks.

这篇关于充分利用PERF用户空间堆栈信息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！