问题描述
我一直在试图让H264编码,输入在Android平板电脑拍摄的相机采用全新的低级别的。我已经经历了这个有些困难了,因为媒体codeCAPI的记录不完整,但我已经得到了一些工作,在最后。
I have been trying to get H264 encoding to work with input captured by the camera on an Android tablet using the new low-level MediaCodec. I have gone through some difficulties with this, since the MediaCodecAPI is poorly documented, but I've gotten something to work at last.
我设置了摄像头,如下所示:
I'm setting up the camera as follows:
Camera.Parameters parameters = mCamera.getParameters();
parameters.setPreviewFormat(ImageFormat.YV12); // <1>
parameters.setPreviewFpsRange(4000,60000);
parameters.setPreviewSize(640, 480);
mCamera.setParameters(parameters);
有关编码部分,我实例化媒体codeC目标如下:
For the encoding part, I'm instantiating the MediaCodec object as follows:
mediaCodec = MediaCodec.createEncoderByType("video/avc");
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 640, 480);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 500000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 15);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT,
MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar); // <2>
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
mediaCodec.start();
的最终目标是建立一个RTP流(并与Skype对应),但到目前为止,我只直接流原始H264到我的桌面。在那里,我用下面的的GStreamer流水线显示结果:
The final goal is to create an RTP-stream (and correspond with Skype), but so far I am only streaming the raw H264 directly to my desktop. There I use the following GStreamer-pipeline to show the result:
gst-launch udpsrc port=5555 ! video/x-h264,width=640,height=480,framerate=15/1 ! ffdec_h264 ! autovideosink
一切运作良好,除了颜色。我需要设置2 colorformats计算机:一为CAMERA-preVIEW(标记&LT线; 1&GT;
),一个用于媒体codeC -object(标记 2&GT;
)
All works well, except for the colors. I need to set 2 colorformats in the computer: one for the camera-preview (line tagged with <1>
) and one for the MediaCodec-object (tagged with <2>
)
要确定该行的可接受值&LT; 1&GT;
我用 parameters.getSupported previewFormats()
。从此,我知道,在相机上唯一支持的格式有ImageFormat.NV21和ImageFormat.YV2.
To determine the acceptable values for the lines <1>
I used parameters.getSupportedPreviewFormats()
. From this, I know that the only supported formats on the camera are ImageFormat.NV21 and ImageFormat.YV2.
有关 2&GT;
,我检索到的<一个href="http://developer.android.com/reference/android/media/Media$c$ccInfo.$c$ccCapabilities.html">Media$c$ccInfo.$c$ccCapabilities-object类型的视频/ AVC 的,是整数值19(对应<一个href="http://developer.android.com/reference/android/media/Media$c$ccInfo.$c$ccCapabilities.html#COLOR_FormatYUV420Planar">Media$c$ccInfo.$c$ccCapabilities.COLOR_FormatYUV420Planar和2130708361(不与<一个任意值对应href="http://developer.android.com/reference/android/media/Media$c$ccInfo.$c$ccCapabilities.html">Media$c$ccInfo.$c$ccCapabilities).
For <2>
, I retrieved the MediaCodecInfo.CodecCapabilities-object for type video/avc, being the integer values 19 (corresponding with MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar and 2130708361 (which doesn't correspond with any value of MediaCodecInfo.CodecCapabilities).
任何其他值比崩溃上述结果。
Any other value than the above results in a crash.
结合这些设置,提供了不同的结果,我将在下面显示。下面是Android上的截图(即真实的颜色):下面是结果如图的Gstreamer:
Combining these settings gives different results, which I'll show below. Here's the screenshot on Android (i.e. the "real" colors):Here are the results as shown by Gstreamer:
&LT; 1&GT;
= NV21, 2&GT;
= COLOR_FormatYUV420Planar
<1>
= NV21, <2>
= COLOR_FormatYUV420Planar
&LT; 1&GT;
= NV21, 2&GT;
= 2130708361
<1>
= NV21, <2>
= 2130708361
&LT; 1&GT;
= YV2, 2&GT;
= COLOR_FormatYUV420Planar
<1>
= YV2, <2>
= COLOR_FormatYUV420Planar
&LT; 1&GT;
= YV2, 2&GT;
= 2130708361
<1>
= YV2, <2>
= 2130708361
可以看出,没有这些都满足。该YV2-色彩看起来最有前途的,但它看起来像红(CR),蓝(CB)被反转。在NV21看起来隔行我猜(但是,我在这一领域不是专家)。
As can be seen, none of these are satisfying. The YV2-colorspace looks the most promising, but it looks like red (Cr) and blue (Cb) are inverted. The NV21 looks interlaced I guess (however, I'm no expert in this field).
由于目的是与Skype进行沟通,我想我应该不会改变德codeR(即GStreamer的命令),对不对?这是要解决在Android和如果是的话:怎么了?或者可以这样通过添加特定RTP有效载荷信息解决了吗?任何其他建议?
Since the purpose is to communicate with Skype, I assume I shouldn't change the decoder (i.e. the Gstreamer-command), right? Is this to be solved in Android and if so: how? Or can this be solved by adding certain RTP payload information? Any other suggestion?
推荐答案
我通过交换byteplanes自己在Android上水平,采用了简单的功能解决了这个问题:
I solved it by swapping the byteplanes myself on Android level, using a simple function:
public byte[] swapYV12toI420(byte[] yv12bytes, int width, int height) {
byte[] i420bytes = new byte[yv12bytes.length];
for (int i = 0; i < width*height; i++)
i420bytes[i] = yv12bytes[i];
for (int i = width*height; i < width*height + (width/2*height/2); i++)
i420bytes[i] = yv12bytes[i + (width/2*height/2)];
for (int i = width*height + (width/2*height/2); i < width*height + 2*(width/2*height/2); i++)
i420bytes[i] = yv12bytes[i - (width/2*height/2)];
return i420bytes;
}
这篇关于媒体codeC和摄像头:色彩空间不匹配的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!