实时模板匹配 - OpenCV，C ++

本文介绍了实时模板匹配 - OpenCV，C ++的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述 29岁程序员，3月因学历无情被辞！ 我正在尝试使用模板实现实时跟踪。我希望每个框架更新模板。我做的主要修改是： 1）将模板匹配和minmaxLoc分成单独的模块，即 TplMatch（） 2）在 track（）函数中，select_flag始终为true，每个迭代将新模板复制到myTemplate。 3）函数 track（）的最后3行是更新模板（roiImg） 4）此外，我已删除 track（）函数的任何参数， img 和 roiImg 是全局变量，因此无需将它们传递给函数。 以下是代码： #include< iostream> #includeopencv2 / opencv.hpp #include< opencv2 / imgproc / imgproc.hpp> #include< Opencv2 / highgui / highgui.hpp> #include< opencv2 / objdetect / objdetect.hpp> #include< sstream> 使用命名空间cv; using namespace std; 点point1，point2; / *边界框的垂直点* / int drag = 0; Rect rect; / * bounding box * / Mat img，roiImg; / * roiImg - 边界框中的图像部分* / int select_flag = 0; bool go_fast = false; Mat mytemplate; /// -------模板匹配-------------------------- -------------------------------------------------- ------------------- Mat TplMatch（Mat& img，Mat& mytemplate） { Mat结果; matchTemplate（img，mytemplate，result，CV_TM_SQDIFF_NORMED）; normalize（result，result，0，1，NORM_MINMAX，-1，Mat（））; 返回结果; } /// -------本地化与minMaxLoc的最佳匹配----------------- -------------------------------------------------- ----- Point minmax（Mat& result） { double minVal，maxVal; Point minLoc，maxLoc，matchLoc; minMaxLoc（result，& minVal，& maxVal，& minLoc，& maxLoc，Mat（））; matchLoc = minLoc; 返回matchLoc; } /// ------- tracking ---------------------- -------------------------------------------------- -------------------------------- void track（） { if（select_flag） { roiImg.copyTo（mytemplate）; // select_flag = false; go_fast = true; } // imshow（mytemplate，mytemplate）; waitKey（0）; Mat result = TplMatch（img，mytemplate）; Point match = minmax（result）; rectangle（img，match，Point（match.x + mytemplate.cols，match.y + mytemplate.rows），CV_RGB（255，255，255），0.5）; std :: cout<< match：<匹配<< endl; ///最新匹配是新模板 Rect ROI = cv :: Rect（match.x，match.y，mytemplate.cols，mytemplate.rows）; roiImg = img（ROI）; imshow（roiImg，roiImg）; // waitKey（0）; } /// ------- MouseCallback函数--------------------- -------------------------------------------------- ------------------- void mouseHandler（int event，int x，int y，int flags，void * param） { if（event == CV_EVENT_LBUTTONDOWN&&！drag） { ///左键点击。 ROI选择开始 point1 = Point（x，y）; drag = 1; } if（event == CV_EVENT_MOUSEMOVE&& drag） { ///鼠标拖动。选择投资回报率 Mat img1 = img.clone（）; point2 = Point（x，y）; rectangle（img1，point1，point2，CV_RGB（255，0，0），3，8，0）; imshow（image，img1）; } if（event == CV_EVENT_LBUTTONUP&&&drag） { point2 = Point（x，y）; rect = Rect（point1.x，point1.y，x - point1.x，y - point1.y）; drag = 0; roiImg = img（rect）; // imshow（MOUSE roiImg，roiImg）; waitKey（0）; } if（event == CV_EVENT_LBUTTONUP） { /// ROI selected select_flag = 1; drag = 0; } } /// ------- Main（）------- -------------------------------------------------- ------------------------------------------------- int main（） { int k; / * /// open webcam VideoCapture cap（0）; if（！cap.isOpened（）） return 1; * / ///打开视频文件 VideoCapture cap; cap.open（Megamind.avi）; if（！cap.isOpened（）） {cout< 无法打开视频文件<< endl; return -1; } / * ///将视频设置为320x240 cap.set（CV_CAP_PROP_FRAME_WIDTH，320）; cap.set（CV_CAP_PROP_FRAME_HEIGHT，240）; * / cap>> img; GaussianBlur（img，img，Size（7,7），3.0）; imshow（image，img）; while（1） { cap>> img; if（img.empty（）） break; //水平翻转帧并添加模糊 cv :: flip（img，img，1）; GaussianBlur（img，img，Size（7,7），3.0）; if（rect.width == 0& rect.height == 0） cvSetMouseCallback（image，mouseHandler，NULL）; else track（）; imshow（image，img）; // waitKey（100）; k = waitKey（75）; k = waitKey（go_fast？30：10000）; if（k == 27） break; } return 0; } 更新的模板未被跟踪。我不能弄清楚为什么这是发生，因为我更新我的模板（roiImg）与每个迭代。 minmax（）函数的匹配值每次都返回相同的点（坐标）。测试视频的可用视频： http://www.youtube.com/watch?v=vpnkk7N2E0Q& 解决方案 p>我从您对此问题的修订版本中获取原始代码： http://stackoverflow.com/revisions/20180073/3 我对原始代码进行了最小的更改，结果代码如下： #include< iostream> #includeopencv2 / opencv.hpp #include< opencv2 / imgproc / imgproc.hpp> #include< Opencv2 / highgui / highgui.hpp> #include< opencv2 / objdetect / objdetect.hpp> #include< sstream> 使用命名空间cv; using namespace std; 点point1，point2; / *边界框的垂直点* / int drag = 0; Rect rect; / * bounding box * / Mat img，roiImg; / * roiImg - 边界框中的图像部分* / int select_flag = 0; bool go_fast = false; Mat mytemplate; /// -------模板匹配-------------------------- -------------------------------------------------- ------------------- Mat TplMatch（Mat& img，Mat& mytemplate） { Mat结果; matchTemplate（img，mytemplate，result，CV_TM_SQDIFF_NORMED）; normalize（result，result，0，1，NORM_MINMAX，-1，Mat（））; 返回结果; } /// -------本地化与minMaxLoc的最佳匹配----------------- -------------------------------------------------- ----- Point minmax（Mat& result） { double minVal，maxVal; Point minLoc，maxLoc，matchLoc; minMaxLoc（result，& minVal，& maxVal，& minLoc，& maxLoc，Mat（））; matchLoc = minLoc; 返回matchLoc; } /// ------- tracking ---------------------- -------------------------------------------------- -------------------------------- void track（） { if（select_flag） { //roiImg.copyTo (mytemplate）; // select_flag = false; go_fast = true; } // imshow（mytemplate，mytemplate）; waitKey（0）; Mat result = TplMatch（img，mytemplate）; Point match = minmax（result）; rectangle（img，match，Point（match.x + mytemplate.cols，match.y + mytemplate.rows），CV_RGB（255，255，255），0.5）; std :: cout<< match：<匹配<< endl; ///最新匹配是新模板 Rect ROI = cv :: Rect（match.x，match.y，mytemplate.cols，mytemplate.rows）; roiImg = img（ROI）; roiImg.copyTo（mytemplate）; imshow（roiImg，roiImg）; // waitKey（0）; } /// ------- MouseCallback函数--------------------- -------------------------------------------------- ------------------- void mouseHandler（int event，int x，int y，int flags，void * param） { if（event == CV_EVENT_LBUTTONDOWN&&！drag） { ///左键点击。 ROI选择开始 point1 = Point（x，y）; drag = 1; } if（event == CV_EVENT_MOUSEMOVE&& drag） { ///鼠标拖动。选择投资回报率 Mat img1 = img.clone（）; point2 = Point（x，y）; rectangle（img1，point1，point2，CV_RGB（255，0，0），3，8，0）; imshow（image，img1）; } if（event == CV_EVENT_LBUTTONUP&&&drag） { point2 = Point（x，y）; rect = Rect（point1.x，point1.y，x - point1.x，y - point1.y）; drag = 0; roiImg = img（rect）; roiImg.copyTo（mytemplate）; // imshow（MOUSE roiImg，roiImg）; waitKey（0）; } if（event == CV_EVENT_LBUTTONUP） { /// ROI selected select_flag = 1; drag = 0; } } /// ------- Main（）------- -------------------------------------------------- ------------------------------------------------- int main（） { int k; / * /// open webcam VideoCapture cap（0）; if（！cap.isOpened（）） return 1; * / ///打开视频文件 VideoCapture cap; cap.open（Megamind.avi）; if（！cap.isOpened（）） {cout< 无法打开视频文件<< endl; return -1; } / * ///将视频设置为320x240 cap.set（CV_CAP_PROP_FRAME_WIDTH，320）; cap.set（CV_CAP_PROP_FRAME_HEIGHT，240）; * / cap>> img; GaussianBlur（img，img，Size（7,7），3.0）; imshow（image，img）; while（1） { cap>> img; if（img.empty（）） break; //水平翻转帧并添加模糊 cv :: flip（img，img，1）; GaussianBlur（img，img，Size（7,7），3.0）; if（rect.width == 0& rect.height == 0） cvSetMouseCallback（image，mouseHandler，NULL）; else track（）; imshow（image，img）; // waitKey（100）; k = waitKey（75）; k = waitKey（go_fast？30：10000）; if（k == 27） break; } return 0; } https://www.youtube.com/watch?v=rBCopeneCos 显示上述程序的测试。 我会避免使用全局变量，因为我认为他们不帮助理解问题在哪里;此外，我也会注意OpenCV的 Mat 类的浅VS深拷贝，如 1在他的回答中写道： OpenCV的 Mat 类只是一个实际图像数据的头，它包含一个指针。 operator = 复制指针（以及标题中的其他信息，如图像尺寸），以便两个Mats共享相同的数据。这意味着修改一个Mat中的数据也会改变另一个Mat。这被称为shallow副本，因为只复制顶层（标题），而不是下层（数据）。 要创建基础数据（称为深度复制）的副本，请使用 clone（）方法。 编辑关于漂移的信息： / strong> 在评论中实时模板匹配 - OpenCV ，C ++ ，学习者询问跟踪偏移。 观看视频 https://www.youtube.com/watch?v=rBCopeneCos 我们看到在视频开始时程序正在跟踪女孩的右眼，而在0:15它开始跟踪女孩的眉毛，在0:19它开始跟踪男孩的眉毛，它从来没有跟踪女孩的眼睛，因为例如在0:27它跟踪女孩的右眉毛，而女孩的右眼在图像中清晰可见。 从跟踪眼睛到跟踪眉毛的漂移是一个简单的代码，因为我发布的一个正常，解释很简单：看到视频 https://www.youtube.com/watch?v=sGHEu3u9XvI ，视频开始跟踪（黑色内容矩形），然后我从场景中移除扑克牌，跟踪黑色矩形漂移到场景的左下角;毕竟我们是连续更新模板，所以行为是正确的：程序停止跟踪扑克牌，开始跟踪白色背景，所以你有漂移...换句话说，你的 TplMatch（）函数将总是返回一个有效的结果映像和当前实现的 minmax c $ c>将始终返回有效的最小值。 I am trying to implement real-time tracking using templates. I wish to update the template with every frame. The main modifications I have done are:1) separated the template matching and minmaxLoc into separate modules namely, TplMatch() and minmax() functions, respectively.2) Inside the track() function, the select_flag is kept always true so that new template is copied to 'myTemplate' with every iteration.3) The last 3 lines of function track() are to update the template (roiImg).4) Also, I have removed any arguments to track() function, since, img and roiImg are global variables and hence no need to pass them to functions.Following is the code:#include <iostream>#include "opencv2/opencv.hpp"#include <opencv2/imgproc/imgproc.hpp>#include <opencv2/highgui/highgui.hpp>#include <opencv2/objdetect/objdetect.hpp>#include <sstream>using namespace cv;using namespace std;Point point1, point2; /* vertical points of the bounding box */int drag = 0;Rect rect; /* bounding box */Mat img, roiImg; /* roiImg - the part of the image in the bounding box */int select_flag = 0;bool go_fast = false;Mat mytemplate;///------- template matching -----------------------------------------------------------------------------------------------Mat TplMatch( Mat &img, Mat &mytemplate ){ Mat result; matchTemplate( img, mytemplate, result, CV_TM_SQDIFF_NORMED ); normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() ); return result;}///------- Localizing the best match with minMaxLoc ------------------------------------------------------------------------Point minmax( Mat &result ){ double minVal, maxVal; Point minLoc, maxLoc, matchLoc; minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() ); matchLoc = minLoc; return matchLoc;}///------- tracking --------------------------------------------------------------------------------------------------------void track(){ if (select_flag) { roiImg.copyTo(mytemplate);// select_flag = false; go_fast = true; }// imshow( "mytemplate", mytemplate ); waitKey(0); Mat result = TplMatch( img, mytemplate ); Point match = minmax( result ); rectangle( img, match, Point( match.x + mytemplate.cols , match.y + mytemplate.rows ), CV_RGB(255, 255, 255), 0.5 ); std::cout << "match: " << match << endl; /// latest match is the new template Rect ROI = cv::Rect( match.x, match.y, mytemplate.cols, mytemplate.rows ); roiImg = img( ROI ); imshow( "roiImg", roiImg ); //waitKey(0);}///------- MouseCallback function ------------------------------------------------------------------------------------------void mouseHandler(int event, int x, int y, int flags, void *param){ if (event == CV_EVENT_LBUTTONDOWN && !drag) { /// left button clicked. ROI selection begins point1 = Point(x, y); drag = 1; } if (event == CV_EVENT_MOUSEMOVE && drag) { /// mouse dragged. ROI being selected Mat img1 = img.clone(); point2 = Point(x, y); rectangle(img1, point1, point2, CV_RGB(255, 0, 0), 3, 8, 0); imshow("image", img1); } if (event == CV_EVENT_LBUTTONUP && drag) { point2 = Point(x, y); rect = Rect(point1.x, point1.y, x - point1.x, y - point1.y); drag = 0; roiImg = img(rect);// imshow("MOUSE roiImg", roiImg); waitKey(0); } if (event == CV_EVENT_LBUTTONUP) { /// ROI selected select_flag = 1; drag = 0; }}///------- Main() ----------------------------------------------------------------------------------------------------------int main(){ int k;/*///open webcam VideoCapture cap(0); if (!cap.isOpened()) return 1;*/ ///open video file VideoCapture cap; cap.open( "Megamind.avi" ); if ( !cap.isOpened() ) { cout << "Unable to open video file" << endl; return -1; }/* /// Set video to 320x240 cap.set(CV_CAP_PROP_FRAME_WIDTH, 320); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);*/ cap >> img; GaussianBlur( img, img, Size(7,7), 3.0 ); imshow( "image", img ); while (1) { cap >> img; if ( img.empty() ) break; // Flip the frame horizontally and add blur cv::flip( img, img, 1 ); GaussianBlur( img, img, Size(7,7), 3.0 ); if ( rect.width == 0 && rect.height == 0 ) cvSetMouseCallback( "image", mouseHandler, NULL ); else track(); imshow("image", img);// waitKey(100); k = waitKey(75); k = waitKey(go_fast ? 30 : 10000); if (k == 27) break; } return 0;}The updated template is not being tracked. I am not able to figure out why this is happening since I am updating my template (roiImg) with each iteration. The match value from minmax() function is returning the same point (coordinates) every-time. Test video is availbale at: http://www.youtube.com/watch?v=vpnkk7N2E0Q&feature=youtu.bePlease look into it and guide ahead...thanks a lot! 解决方案 I get your original code from this revision of your question: http://stackoverflow.com/revisions/20180073/3I made the smallest change to your original code, my resulting code is the following:#include <iostream>#include "opencv2/opencv.hpp"#include <opencv2/imgproc/imgproc.hpp>#include <opencv2/highgui/highgui.hpp>#include <opencv2/objdetect/objdetect.hpp>#include <sstream>using namespace cv;using namespace std;Point point1, point2; /* vertical points of the bounding box */int drag = 0;Rect rect; /* bounding box */Mat img, roiImg; /* roiImg - the part of the image in the bounding box */int select_flag = 0;bool go_fast = false;Mat mytemplate;///------- template matching -----------------------------------------------------------------------------------------------Mat TplMatch( Mat &img, Mat &mytemplate ){ Mat result; matchTemplate( img, mytemplate, result, CV_TM_SQDIFF_NORMED ); normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() ); return result;}///------- Localizing the best match with minMaxLoc ------------------------------------------------------------------------Point minmax( Mat &result ){ double minVal, maxVal; Point minLoc, maxLoc, matchLoc; minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() ); matchLoc = minLoc; return matchLoc;}///------- tracking --------------------------------------------------------------------------------------------------------void track(){ if (select_flag) { //roiImg.copyTo(mytemplate);// select_flag = false; go_fast = true; }// imshow( "mytemplate", mytemplate ); waitKey(0); Mat result = TplMatch( img, mytemplate ); Point match = minmax( result ); rectangle( img, match, Point( match.x + mytemplate.cols , match.y + mytemplate.rows ), CV_RGB(255, 255, 255), 0.5 ); std::cout << "match: " << match << endl; /// latest match is the new template Rect ROI = cv::Rect( match.x, match.y, mytemplate.cols, mytemplate.rows ); roiImg = img( ROI ); roiImg.copyTo(mytemplate); imshow( "roiImg", roiImg ); //waitKey(0);}///------- MouseCallback function ------------------------------------------------------------------------------------------void mouseHandler(int event, int x, int y, int flags, void *param){ if (event == CV_EVENT_LBUTTONDOWN && !drag) { /// left button clicked. ROI selection begins point1 = Point(x, y); drag = 1; } if (event == CV_EVENT_MOUSEMOVE && drag) { /// mouse dragged. ROI being selected Mat img1 = img.clone(); point2 = Point(x, y); rectangle(img1, point1, point2, CV_RGB(255, 0, 0), 3, 8, 0); imshow("image", img1); } if (event == CV_EVENT_LBUTTONUP && drag) { point2 = Point(x, y); rect = Rect(point1.x, point1.y, x - point1.x, y - point1.y); drag = 0; roiImg = img(rect); roiImg.copyTo(mytemplate);// imshow("MOUSE roiImg", roiImg); waitKey(0); } if (event == CV_EVENT_LBUTTONUP) { /// ROI selected select_flag = 1; drag = 0; }}///------- Main() ----------------------------------------------------------------------------------------------------------int main(){ int k;/*///open webcam VideoCapture cap(0); if (!cap.isOpened()) return 1;*/ ///open video file VideoCapture cap; cap.open( "Megamind.avi" ); if ( !cap.isOpened() ) { cout << "Unable to open video file" << endl; return -1; }/* /// Set video to 320x240 cap.set(CV_CAP_PROP_FRAME_WIDTH, 320); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);*/ cap >> img; GaussianBlur( img, img, Size(7,7), 3.0 ); imshow( "image", img ); while (1) { cap >> img; if ( img.empty() ) break; // Flip the frame horizontally and add blur cv::flip( img, img, 1 ); GaussianBlur( img, img, Size(7,7), 3.0 ); if ( rect.width == 0 && rect.height == 0 ) cvSetMouseCallback( "image", mouseHandler, NULL ); else track(); imshow("image", img);// waitKey(100); k = waitKey(75); k = waitKey(go_fast ? 30 : 10000); if (k == 27) break; } return 0;}The video at https://www.youtube.com/watch?v=rBCopeneCos shows a test of the above program.I would avoid the use of global variable because I think they do not help in understanding where the problems lie; furthermore I also would pay attention to the shallow vs deep copy for OpenCV's Mat class, as 1'' wrote in his answer: OpenCV's Mat class is simply a header for the actual image data, which it contains a pointer to. The operator= copies the pointer (and the other information in the header, like the image dimensions) so that both Mats share the same data. This means that modifying the data in one Mat also changes it in the other. This is called a "shallow" copy, since only the top layer (the header) is copied, not the lower layer (the data). To make a copy of the underlying data (called a "deep copy"), use the clone() method. You can find information about it on the page that you linked to.Edit about the drift:In comment Real-time template matching - OpenCV, C++, learner asks about the tracking drift.Looking at the video https://www.youtube.com/watch?v=rBCopeneCos we see that at the beginning of the video the program is tracking the girl's right eye while at 0:15 it starts to track the girl's eyebrows, at 0:19 it starts to track the boy's eyebrows and it never tracks anymore the girl's eye, for example at 0:27 it tracks the girl's right eyebrow while the girl's right eye is clearly visible in the image.This drift from tracking the eye to tracking the eyebrow is normal in a simple code as the one I posted and the explanation is quite simple: see the video at https://www.youtube.com/watch?v=sGHEu3u9XvI, the video starts with the tracking (contents of the black rectangle) of the playing card, then I remove the playing card from the scene and the tracking black rectangle "drifts" to the bottom left of the scene; after all we are continuosly updating the template and so the behavior is correct: the program stops to track the playing card and starts to track a white background and so you have the "drift"... in other words, your TplMatch() function will always return a valid result image and your current implementation of minmax() will always return a valid a minimum. 这篇关于实时模板匹配 - OpenCV，C ++的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！