问题描述
我能够成功地对 RGB 图像进行失真处理.
I am able to undistort RGB image successfully.
现在，我正在直接处理 I420 数据，而不是先将其转换为 RGB.
Now, I am working on directly undistort I420 data, instead of first converting it to RGB.
以下是我在相机校准后遵循的步骤.
Below are the steps I followed after camera calibration.
K = cv::Matx33d(541.2152931632737, 0.0, 661.7479652584254,
0.0, 541.0606969363056, 317.4524205037745,
0.0, 0.0, 1.0);
D = cv::Vec4d(-0.042166406281296365, -0.001223961942208027, -0.0017036710622692108, 0.00023929900459453295);
newSize = cv::Size(3400, 1940);
cv::Matx33d new_K;
cv::fisheye::estimateNewCameraMatrixForUndistortRectify(K, D, cv::Size(W, H), cv::Mat::eye(3, 3, CV_64F), new_K, 1, newSize); // W,H are the distorted image size
cv::fisheye::initUndistortRectifyMap(K, D, cv::Mat::eye(3, 3, CV_64F), new_K, newSize, CV_16SC2, mapx, mapy);
cv::remap(src, dst, mapx, mapy, cv::INTER_LINEAR);
上面的代码成功地给了我未失真的图像.
Above code is giving me undistorted image successfully.
现在我想不失真 I420 数据.所以，现在我的 src 将是一个 I420/YV12 数据.如何在不先将 I420 数据转换为 RGB 的情况下对其进行失真处理?
顺便说一句I420 是一种只有 1 个通道的图像格式(与 RGB 中的 3 个通道不同).它的高度 = 1.5*图像高度.它的宽度等于图像宽度.
By the wayI420 is an image format with only 1 channel(unlike 3 channels in RGB). It has height = 1.5*image height. Its width is equal to image width.
下面的代码是将 I420 转换为 BGR
Below code is to convert I420 to BGR
cvtColor(src, BGR, CV_YUV2BGR_I420, 3);
BGR - 像素排列I420 - 像素排列
推荐答案
最有效的解决方案是调整 mapx 和 mapy 的大小，并在下采样的 U 和V频道:
The most efficient solution is resizing mapx and mapy and applying shrunk maps on down-sampled U and V channels:
- 在每个轴上将
mapx和mapy缩小 x2 倍 - 创建更小的地图矩阵. - 将缩小地图的所有元素除以 2(适用于映射较低分辨率的图像).
- 在
Y颜色通道上应用mapx和mapy. - 在下采样的
U和V颜色通道上应用shrunk_mapx和shrunk_mapy.
- Shrink
mapxandmapyby a factor of x2 in each axis - create smaller maps matrices. - Divide all elements of shrank maps by 2 (applies mapping lower resolution image).
- Apply
mapxandmapyonYcolor channel. - Apply
shrunk_mapxandshrunk_mapyon down-sampledUandVcolor channels.
这是一个 Python OpenCV 示例代码(请阅读评论):
Here is a Python OpenCV sample code (please read the comments):
import cv2 as cv
import numpy as np
# For the example, read Y, U and V as separate images.
srcY = cv.imread('DistortedChessBoardY.png', cv.IMREAD_GRAYSCALE) # Y color channel (1280x720)
srcU = cv.imread('DistortedChessBoardU.png', cv.IMREAD_GRAYSCALE) # U color channel (640x360)
srcV = cv.imread('DistortedChessBoardV.png', cv.IMREAD_GRAYSCALE) # V color channel (640x360)
H, W = srcY.shape[0], srcY.shape[1]
K = np.array([[541.2152931632737, 0.0, 661.7479652584254],
[0.0, 541.0606969363056, 317.4524205037745],
[0.0, 0.0, 1.0]])
D = np.array([-0.042166406281296365, -0.001223961942208027, -0.0017036710622692108, 0.00023929900459453295])
# newSize = cv::Size(3400, 1940);
newSize = (850, 480)
# cv::Matx33d new_K;
new_K = np.eye(3)
# cv::fisheye::estimateNewCameraMatrixForUndistortRectify(K, D, cv::Size(W, H), cv::Mat::eye(3, 3, CV_64F), new_K, 1, newSize); // W,H are the distorted image size
new_K = cv.fisheye.estimateNewCameraMatrixForUndistortRectify(K, D, (W, H), np.eye(3), new_K, 1, newSize)
# cv::fisheye::initUndistortRectifyMap(K, D, cv::Mat::eye(3, 3, CV_64F), new_K, newSize, CV_16SC2, mapx, mapy);
mapx, mapy = cv.fisheye.initUndistortRectifyMap(K, D, np.eye(3), new_K, newSize, cv.CV_16SC2);
# cv::remap(src, dst, mapx, mapy, cv::INTER_LINEAR);
dstY = cv.remap(srcY, mapx, mapy, cv.INTER_LINEAR)
# Resize mapx and mapy by a factor of x2 in each axis, and divide each element in the map by 2
shrank_mapSize = (mapx.shape[1]//2, mapx.shape[0]//2)
shrunk_mapx = cv.resize(mapx, shrank_mapSize, interpolation = cv.INTER_LINEAR) // 2
shrunk_mapy = cv.resize(mapy, shrank_mapSize, interpolation = cv.INTER_LINEAR) // 2
# Remap U and V using shunk maps
dstU = cv.remap(srcU, shrunk_mapx, shrunk_mapy, cv.INTER_LINEAR, borderValue=128)
dstV = cv.remap(srcV, shrunk_mapx, shrunk_mapy, cv.INTER_LINEAR, borderValue=128)
cv.imshow('dstY', dstY)
cv.imshow('dstU', dstU)
cv.imshow('dstV', dstV)
cv.waitKey(0)
cv.destroyAllWindows()
结果:
是:
U:
V:
转换为RGB后:
After converting to RGB:
C++ 实现注意事项:
C++ implementation considerations:
由于 I420 格式在内存中将 Y、U 和 V 排列为 3 个连续平面，因此很容易为每个平面"设置一个指针，并将其视为灰度图像.
相同的数据排序适用于输出图像 - 将 3 指针设置为输出平面".
Since I420 format arranges Y, U and V as 3 continuous planes in memory, it's simple to set a pointer to each "plane", and treat it as a Grayscale image.
Same data ordering applies the output image - set 3 pointer to output "planes".
插图(假设宽度和高度为偶数，并假设字节步幅等于宽度):
Illustration (assuming even width and height, and assume byte stride equals width):
srcY -> YYYYYYYY dstY -> YYYYYYYYYYYY
YYYYYYYY YYYYYYYYYYYY
YYYYYYYY YYYYYYYYYYYY
YYYYYYYY YYYYYYYYYYYY
YYYYYYYY remap YYYYYYYYYYYY
YYYYYYYY ======> YYYYYYYYYYYY
srcU -> UUUU YYYYYYYYYYYY
UUUU dstU -> YYYYYYYYYYYY
UUUU UUUUUU
srcV -> VVVV UUUUUU
VVVV UUUUUU
VVVV UUUUUU
dstV -> VVVVVV
VVVVVV
VVVVVV
VVVVVV
上图的实现是C++
假设宽高是偶数，字节步幅等于宽，可以使用下面的C++例子将I420转换为Y、U和V平面:
Under the assumption that width and height are even, and byte stride equals width, you can use the following C++ example for converting I420 to Y, U and V planes:
假设:srcI420是I420格式的Wx(H*3/2)矩阵，如cv::Mat srcI420(cv::Size(W), H * 3/2), CV_8UC1);.
Assume: srcI420 is Wx(H*3/2) matrix in I420 format, like cv::Mat srcI420(cv::Size(W, H * 3 / 2), CV_8UC1);.
int W = 1280, H = 720; //Assume resolution of Y plane is 1280x720
//Pointer to Y plane
unsigned char *pY = (unsigned char*)srcI420.data;
//Y plane as cv::Mat, resolution of srcY is 1280x720
cv::Mat srcY = cv::Mat(cv::Size(W, H), CV_8UC1, (void*)pY);
//U plane as cv::Mat, resolution of srcU is 640x360 (in memory buffer, U plane is placed after Y).
cv::Mat srcU = cv::Mat(cv::Size(W/2, H/2), CV_8UC1, (void*)(pY + W*H));
//V plane as cv::Mat, resolution of srcV is 640x360 (in memory buffer, V plane is placed after U).
cv::Mat srcV = cv::Mat(cv::Size(W / 2, H / 2), CV_8UC1, (void*)(pY + W*H + (W/2*H/2)));
//Display srcY, srcU, srcV for testing
cv::imshow("srcY", srcY);
cv::imshow("srcU", srcU);
cv::imshow("srcV", srcV);
cv::waitKey(0);
以上示例使用指针操作，无需复制数据.
您可以对目标 I420 图像使用相同的指针操作.
Above example uses pointer manipulations, without the need for copying the data.
You can use the same pointer manipulations for your destination I420 image.
注意:该解决方案在大多数情况下都有效，但不能保证在所有情况下都有效.
Note: The solution is going to work in most cases, but not guaranteed to work in all cases.
这篇关于如何不失真 I420 图像数据?有效率的的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！