FHD 视频流上 OpenCV 文本覆盖的 CPU 占用空间太高

问题描述

我想显示 FHD 实时流 (25 fps) 并覆盖一些(变化的)文本.为此，我主要使用下面的代码.

I want to display a FHD live-stream (25 fps) and overlay some (changing) text. For this I essentially use the code below.

基本上是

1. 加载框架
2. (cv::putText 此处略过)
3. 如果是delay
的倍数，则显示帧

1. Load frame
2. (cv::putText skipped here)
3. Display frame if it's a multiple of delay

但是与例如相比，代码超级慢.mpv 并消耗大量 CPU 时间(cv::useOptimized() == true).

but the code is super super slow compared to e.g. mpv and consumes way to much cpu-time (cv::useOptimized() == true).

到目前为止 delay 是我不方便的小提琴参数，以某种方式使其可行.

So far delay is my inconvenient fiddle-parameter to somehow make it feasible.

• delay == 1 导致 180 % CPU 使用率(全帧率)
• delay == 5 导致 80% 的 CPU 使用率

• delay == 1 results in 180 % CPU usage (full frame-rate)
• delay == 5 results in 80 % CPU usage

但是delay == 5 或 5 fps 真的很慢，实际上 CPU 负载仍然太多.

But delay == 5 or 5 fps is really sluggish and actually still too much cpu load.

如何使此代码更快或更佳或以其他方式解决任务(我不受 opencv 约束)?

P.s. 没有 cv::imshow，CPU 使用率低于 30%，与 delay 无关.

P.s. Without cv::imshow the CPU usage is less than 30 %, regardless of delay.

#include <opencv2/opencv.hpp>
#include <X11/Xlib.h>

// process ever delayth frame
#define delay 5

Display* disp = XOpenDisplay(NULL);
Screen*  scrn = DefaultScreenOfDisplay(disp);
int screen_height = scrn->height;
int screen_width  = scrn->width;

int main(int argc, char** argv){

  cv::VideoCapture cap("rtsp://url");
  cv::Mat frame;

  if (cap.isOpened())
    cap.read(frame);

  cv::namedWindow(  "PREVIEW", cv::WINDOW_NORMAL );
  cv::resizeWindow( "PREVIEW", screen_width, screen_height );

  int framecounter = 0;
  while (true){

    if (cap.isOpened()){

      cap.read(frame);
      framecounter += 1;

      // Display only delay'th frame
      if (framecounter % delay == 0){
        /*
         * cv::putText
         */
        framecounter = 0;
        cv::imshow("PREVIEW", frame);
      }

    }
    cv::waitKey(1);
  }
}

推荐答案

我现在了解了 valgrind(存储库)和 gprof2dot(pip3 install --user gprof2dot):

valgrind --tool=callgrind /path/to/my/binary    # Produced file callgrind.out.157532
gprof2dot --format=callgrind --output=out.dot callgrind.out.157532
dot -Tpdf out.dot -o graph.pdf

这产生了一个美妙的图表，说明 cvResize 上超过 60% 蒸发.事实上，当我注释掉 cv::resizeWindow 时，cpu 使用率从 180% 降低到 ~ 60%.

That produced a wonderful graph saying that over 60 % evaporates on cvResize.And indeed, when I comment out cv::resizeWindow, the cpu usage lowers from 180 % to ~ 60 %.

由于屏幕的分辨率为 1920 x 1200，流的分辨率为 1920 x 1080，因此除了消耗 CPU 周期外，它基本上什么也没做.

Since the screen has a resolution of 1920 x 1200 and the stream 1920 x 1080, it essentially did nothing but burning CPU cycles.

到目前为止，这仍然很脆弱.一旦我将其切换到全屏模式并返回，cpu 负载就会恢复到 180%.

So far, this is still fragile. As soon as I switch it to full-screen mode and back, the cpu load goes back to 180 %.

为了解决这个问题，事实证明我可以使用 cv::WINDOW_AUTOSIZE ...

To fix this, it turned out that I can either disable resizing completely with cv::WINDOW_AUTOSIZE ...

cv::namedWindow( "PREVIEW", cv::WINDOW_AUTOSIZE );

... 或 -- 正如 Micka 建议的 -- 在使用 OpenGL 支持编译的 OpenCV 版本上(-DWITH_OPENGL=ON，我的 Debian 存储库版本不是)，使用 ...

... or -- as Micka suggested -- on OpenCV versions compiled with OpenGL support (-DWITH_OPENGL=ON, my Debian repository version was not), use ...

    cv::namedWindow( "PREVIEW", cv::WINDOW_OPENGL );

... 将渲染卸载到 GPU，结果与调整大小一起更快(CPU 为 55%，而我为 65%).它只是似乎不能与cv::WINDOW_KEEPRATIO一起工作代码>.*

... to offload the rendering to the GPU, what turns out to be even faster together with resizing (55 % CPU compared to 65 % for me).It just does not seem to work together with cv::WINDOW_KEEPRATIO.*

此外，事实证明 cv:UMat 可以用作 cv:Mat 的替代品，这进一步提高了性能(如 ps -e -o pcpu,args):

Furthermore, it turns out that cv:UMat can be used as a drop-in replacement for cv:Mat which additionally boosts the performance (as seen by ps -e -o pcpu,args):

[*] 所以我们必须手动缩放它并注意纵横比.

[*] So we have to manually scale it and take care of the aspect ratio.

float screen_aspratio = (float) screen_width / screen_height;
float image_aspratio  = (float) image_width  / image_height;

if ( image_aspratio >= screen_aspratio ) { // width limited, center window vertically
  cv::resizeWindow("PREVIEW", screen_width, screen_width / image_aspratio );
  cv::moveWindow(  "PREVIEW", 0, (screen_height - image_height) / 2 );
}
else { // height limited, center window horizontally
  cv::resizeWindow("PREVIEW", screen_height * image_aspratio, screen_height );
  cv::moveWindow(  "PREVIEW", (screen_width - image_width) / 2, 0 );
}

这篇关于FHD 视频流上 OpenCV 文本覆盖的 CPU 占用空间太高的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！