问题描述
TL; DR::我想从AVI/MP4文件中读取原始的h264流,甚至损坏/不完整.
TL;DR: I want to read raw h264 streams from AVI/MP4 files, even broken/incomplete.
关于h264的几乎每个文档都告诉我它由NAL数据包组成.好的.几乎到处都告诉我,数据包应该以00 00 01
或00 00 00 01
之类的签名开头.例如, https://stackoverflow.com/a/18638298/8167678 ,https://stackoverflow.com/a/17625537/8167678
Almost every document about h264 tells me that it consists of NAL packets. Okay. Almost everywhere told to me that the packet should start with a signature like 00 00 01
or 00 00 00 01
. For example, https://stackoverflow.com/a/18638298/8167678, https://stackoverflow.com/a/17625537/8167678
好的.
我下载了random_youtube_video.mp4并从中删除了一帧:
I downloaded random_youtube_video.mp4 and strip out one frame from it:
ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.avi
得到了:红色部分-这是AVI容器的一部分,其他部分-实际数据.如您所见,这里有00 00 24 A9
而不是00 00 00 01
And got:Red part - this is part of AVI container, other - actual data.As you can see, here I have 00 00 24 A9
instead of 00 00 00 01
此AVI文件播放完美
我对mp4容器也是如此:
I do same for mp4 container:
如您所见,这里的字节数完全相同.此MP4文件播放完美
As you can see, here exact same bytes.This MP4 file plays perfectly
我尝试去除原始数据:ffmpeg -i pic.avi -c copy pic.h264
I try to strip out raw data:ffmpeg -i pic.avi -c copy pic.h264
此文件无法在VLC中播放,甚至生成该文件的ffmpeg也无法对其进行解析:
This file can't play in VLC or even ffmpeg, which produced this file, can't parse it:
我下载了mp4流分析器,并得到:
I downloaded mp4 stream analyzer and got:
MP4Box
告诉我:
Cannot find H264 start code
Error importing pic.h264: BitStream Not Compliant
什么都不起作用,很难学习h264的内部原理.
It very hard to learn internals of h264, when nothing works.
所以,我有问题:
- mp4里面有什么实际数据?
- 我必须阅读以解码该数据(我的意思是不同的附件)
- 如何从此中断的"原始流中读取流并获取解码的图像(甚至使用ffmpeg)?
更新:
ffmpeg中似乎有错误:
It seems bug in ffmpeg:
当我进行两次转换时:
ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.mp4
ffmpeg pic.mp4 -c copy pic.h264
但是当我直接转换文件时:
But when I convert file directly:
ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.h264
我有NAL签名和一个额外的NAL单元.其他字节相同(已选择).
I have NALs signatures and one extra NAL unit. Other bytes are same (selected).
这是虫子吗?
更新
不是,这不是错误,U必须使用-bsf h264_mp4toannexb选项将流另存为附件B"格式(带前缀)
Not, this is not bug, U must use option -bsf h264_mp4toannexb to save stream as "Annex B" format (with prefixes)
推荐答案
几乎到处都告诉我,该数据包应该以如下签名开头:
00 00 01
或00 00 00 01
"
"...如您所见,这里有00 00 24 A9
而不是00 00 00 01
"
"...As you can see, here I have 00 00 24 A9
instead of 00 00 00 01
"
您的H264为AVCC格式,这意味着它使用数据 size (而不是数据起始代码).只有附件B会将您提到的签名作为起始代码.
Your H264 is in AVCC format which means it uses data sizes (instead of data start codes). It is only Annex-B that will have your mentioned signature as start code.
您查找帧不是通过查找起始代码,而是只是跳过帧大小以达到(请求的)帧的最终正确偏移量...
You seek frames, not by looking for start codes, but instead you just do skipping by frame sizes to reach the final correct offset of a (requested) frame...
AVI处理:
-
读取大小(四个)字节(32位整数,小尾数).
提取接下来的后续字节,直到最大大小.
Extract the next following bytes up to size amount.
这是您的H.264帧(采用AVCC格式),对字节进行解码以查看图像.
This is your H.264 frame (in AVCC format), decode the bytes to view image.
要转换为附件B,请尝试用00 00 00 01
替换H.264 帧字节的前4个字节.
To convert into Annex-B, try replacing first 4 bytes of H.264 frame bytes with 00 00 00 01
.
考虑显示的AVI字节(请参见> 第一 图片):
Consider your shown AVI bytes (see first picture) :
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 4C 49 53 54 BA 24 00 00 6D 6F 76 69 ....LISTº$..movi
30 30 64 63 AD 24 00 00 00 00 24 A9 65 88 84 27 00dc.$....$©eˆ„'
C7 11 FE B3 C7 83 08 00 08 2A 7B 6E 59 B5 71 E1 Ç.þ³Çƒ...*{nYµqá
E3 9C 0E 73 E7 10 50 00 18 E9 25 F7 AA 7D 9C 30 ãœ.sç.P..é%÷ª}œ0
E6 2F 0F 20 00 3A 64 AA CA 5E 4F CA FF AE 20 04 æ/. .:dªÊ^OÊÿ® .
07 81 40 00 48 00 0A 28 71 21 84 48 06 18 90 0C ..@.H..(q!„H....
31 14 57 9E 7A CD 63 A0 E0 9B 96 69 C5 18 AE F2 1.WžzÍc à›–iÅ.®ò
E6 07 02 29 01 20 10 70 A1 0F 8C BC 73 F0 78 FA æ..). .p¡.Œ¼sðxú
9E 1D E1 C2 BF 8C 62 CE CE AC 14 5A A4 E1 45 44 ž.á¿ŒbÎά.Z¤áED
38 38 85 DB 12 57 3E F6 E0 FB AE 03 04 21 62 8D 88…Û.W>öàû®..!b.
F6 F1 1E 37 1C A2 FF 75 1C F1 02 66 0C 92 07 06 öñ.7.¢ÿu.ñ.f.’..
15 7C 90 15 6F 7D FC BD 13 1E 2B 0C 14 3C 0C 00 .|..o}ü½..+..<..
B0 EA 6F 53 B4 98 D7 80 7A 68 3E 34 69 20 D2 FA °êoS´˜×€zh>4i Òú
F0 91 FC 75 C6 00 01 18 C0 00 3B 9A C5 E2 7D BF ð‘üuÆ...À.;šÅâ}¿
一些解释:
-
忽略前导多个
00
个字节.
4C 49 53 54 D6 3C 00 00 6D 6F 76 69
包括30 30 64 63
= AVI列表"标头.
4C 49 53 54 D6 3C 00 00 6D 6F 76 69
including 30 30 64 63
= AVI "List" header.
AD 24 00 00
==十进制9389
是AVI自身的H264大小(必须在 Little Endian 中读取).
AD 24 00 00
== decimal 9389
is AVI's own size of H264 item (must read in Little Endian).
请注意,AVI字节包括...
-记录 item 的总大小(AD 24 00 00
...或对Little Endian取反:00 00 24 AD
)-,后跟 item 数据(00 00 24 A9 65 88 84 27 ... etc ... C5 E2 7D BF
).
Notice that the AVI bytes include...
- a note of item's total size (AD 24 00 00
... or reverse for Little Endian : 00 00 24 AD
)
- followed by item data (00 00 24 A9 65 88 84 27 ... etc ... C5 E2 7D BF
).
此 size 既包含AVI的" size "条目的4个字节,又包含 item 自身字节的预期字节长度.可以简单地写为:
This size includes both the 4 bytes of the AVI's"size" entry + expected bytes length of the item's own bytes. Can be written simply as:
AVI_Item_Size = ( 4 + item_H264_Frame.length );
AVI中的H.264视频帧字节:
接下来是 item 数据,它是H.264 视频帧.由于格式/字节布局的巧合,它也保存了 data 的 size 的4个字节的条目(因为您的H264是AVCC格式,如果它是附件- B,那么您将在此处看到起始码字节,而不是大小字节).
Next follows the item data, which is the H.264 video frame. By sheer coincidence of formats/bytes layout, it too holds a 4-byte entry for data's size (since your H264 is in AVCC format, if it was Annex-B then you would be seeing start code bytes here instead of size bytes).
与AVI字节不同,这些H264 大小字节以 Big Endian 格式写入.
Unlike AVI bytes, these H264 size bytes are written in Big Endian format.
-
00 00 24 A9
=此视频帧的字节大小(而不是起始代码:00 00 00 01
).
00 00 24 A9
= size of bytes for this video frame (instead of start code :00 00 00 01
).
65 88 84 27 C7 11 FE B3 C7
= H.264 关键帧(始终以X5开头,其中X
值基于其他设置).
65 88 84 27 C7 11 FE B3 C7
= H.264 keyframe (always begins X5, where the X
value is based on other settings).
如果后面跟有...,请记住四个大小的字节(甚至是起始代码)后的内容.
Remember after four size bytes (or even start codes) if followed by...
- 字节
X5
=关键帧(IDR),例如字节65
. - 字节
X1
= P或B帧,例如字节41
. - 字节
X6
= SEI(补充增强信息). - 字节
X7
= SPS(序列参数集). - 字节
X8
= PPS(图片参数集). - 字节
00 00 00 X9
=访问单元定界符.
- byte
X5
= keyframe (IDR), example byte65
. - byte
X1
= P or B frame, example byte41
. - byte
X6
= SEI (Supplemental Enhancement Information). - byte
X7
= SPS (Sequence Parameter Set). - byte
X8
= PPS (Picture Parameter Set). - bytes
00 00 00 X9
= Access unit delimiter.
如果在AVI文件中搜索完全相同的字节,则可以找到H.264.请参见 第三 图片,这些是您的H.264字节(它们被剪切并粘贴到AVI容器中).
You can find the H.264 if you search for exact same bytes within AVI file. See third picture, these are your H.264 bytes (they are cut & pasted into the AVI container).
有时,一帧被切成不同的NAL单元.因此,如果您提取关键帧而只显示1/2或1/3而不是完整图像,则只需抓住下一个或两个NAL,然后重试解码即可.
Sometimes a frame is sliced into different NAL units. So if you extract a key frame and it only shows 1/2 or 1/3 instead of full image, just grab next one or two NAL and re-try the decode.
这篇关于AVI,MP4和"Raw"内部的h264 h264流.不同格式的NAL单元(或ffmpeg错误)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!