在分段的实时MP4流中发送定期的元数据?

问题描述

正如该主题所建议的那样，我想知道是否有可能在分段的MP4实时流中定期发送有关流内容的元数据.

As suggested by the topic, I'm wondering if it's possible to send metadata about the stream contents periodically in a fragmented MP4 live stream.

我正在使用以下命令(1)来获取碎片化的MP4:

I'm using the following command (1) to get fragmented MP4:

ffmpeg -i rtsp://admin:[email protected] -c:v copy -an -movflags empty_moov+omit_tfhd_offset+frag_keyframe+default_base_moof -f mp4 ...

我的主程序从stdout或(unix域)套接字读取此命令的片段，并获得:

My main program reads the fragments from this command from either stdout or from a (unix domain) socket and gets:

ftyp
moov
moof
mdat
moof
mdat
moof
mdat
...

因此，我得到的第一个片段是元数据并描述流内容的 ftyp 和 moov .

So, the first fragments I get are ftyp and moov which are metadata and describe the stream contents.

现在，客户端程序稍后会连接到主程序.问题在于，此时 ftype 和 moov 片段早已消失.

Now a client program connects at a later time to the main program. The problem is, that at that point, the ftype and moov fragments are long gone.

是否有一种方法(= ffmpeg命令选项)使该工作类似于MPEGTS(又名mpeg传输流)，并与该流定期重新发送元数据?像这样:

ftyp
moov
moof
mdat
moof
mdat
moof
mdat
ftyp
moov
moof
mdat
moof
mdat
moof
mdat
...

..或我的唯一选择是将 ftyp 和 moov 数据包缓存在我的主程序中，然后将它们重新发送到 client程序?

.. or is my only option to cache the ftyp and moov packets in my main program and re-send them to the client program when it requests the stream?

相关链接:什么片段mp4(fMP4)到底是什么?与普通的mp4有何不同?

缓存并重新发送 ftyp 和 moov 也不那么简单..因为它以某种方式中断了流(至少浏览器MSE扩展没有不喜欢这样的流). moof 数据包中似乎有很多序列号和内容，需要修改. (+)

Caching and resending ftyp and moov each time a new client connects is not that straightforward either .. as it somehow brokes the stream (at least the browser MSE extensions don't like such a stream). There seems to be lots of sequence numbers and stuff in the moof packets that should be modified. (+)

另一种选择是使流通过另一个进行重混合的FFmpeg进程(并纠正 moof 数据包).由于命令(1)不会提供干净分隔的 ftyp ， moov ， moof 等数据包，因此情况变得更加复杂.

Another option is to pass the stream through another FFmpeg process that does the remuxing (and corrects moof packets). Things are further complicated by the fact that command (1) does not give cleanly-separated ftyp, moov, moof, etc. packets.

感谢任何想法/解决方案.

Any thoughts / solutions appreciated.

编辑:关于(+)，MSE似乎在播放零散的MP4时遇到问题: https://bugs.chromium.org/p/chromium/issues/detail?id=516114

EDIT : regarding (+), MSE seems to have problems playing fragmented MP4 with gaps: https://bugs.chromium.org/p/chromium/issues/detail?id=516114

推荐答案

我终于能够将片段化的MP4毫无问题地提供给浏览器MSE扩展.

I was finally able to feed fragmented MP4 to browser MSE extensions without problems.

如果有人开始向MSE扩展程序添加 moof 和 mdat 数据包，而这些数据包并不是在原始 ftyp 和 moov之后立即出现的，然后是..

If one starts feeding the MSE extensions with moof and mdat packets that did not come immediately after the original ftyp and moov, then ..

..进入MSE扩展的第一个 moof 数据包必须是具有设置为 first_sample_flags_preset 的特殊标志的 moof 数据包. (有关更多信息，请参见ISO/IEC 14496-12:2012(E)规范)

.. the very first moof packet that goes into the MSE extension must be a moof packet that has a special flag called first_sample_flags_preset set (see the ISO/IEC 14496-12:2012(E) specs for more info)

..否则，所有流行的浏览器中的MSE都将冻结，并且不会播放视频(顺便说一句， moof 从> 1开始的序列号完全没有问题).

.. otherwise the MSEs in all popular browsers freeze and there is no video playback (btw, moof sequence numbers starting from > 1 posed no problem at all).

此python软件包对于分析非常有用: https://github.com/beardypig/pymp4

This python package was very useful for the analysis: https://github.com/beardypig/pymp4

要获取该标志，此答案中提供了客户端javascript功能.

To pick up that flag, client-side javascript functions are provided in this answer.

使用功能 getBox 找出盒子的类型( ftyp ， moov ， moof ，等).

Use the function getBox to find out the type of the box (ftyp, moov, moof, etc.).

对于 moof 框，应用函数 findFirstSampleFlag 来查看 moof 框是否启用了 first_sample_flags_preset .

For moof boxes, apply the function findFirstSampleFlag to see if the moof box has the first_sample_flags_preset enabled.

function toInt(arr, index) { // From bytes to big-endian 32-bit integer.  Input: Uint8Array, index
    var dv = new DataView(arr.buffer, 0);
    return dv.getInt32(index, false); // big endian
}

function toString(arr, fr, to) { // From bytes to string.  Input: Uint8Array, start index, stop index.
    return String.fromCharCode.apply(null, arr.slice(fr,to));
}

function getBox(arr, i) { // input Uint8Array, start index
    return [toInt(arr, i), toString(arr, i+4, i+8)]
}

function getSubBox(arr, box_name) { // input Uint8Array, box name
    var i = 0;
    res = getBox(arr, i);
    main_length = res[0]; name = res[1]; // this boxes length and name
    i = i + 8;

    var sub_box = null;

    while (i < main_length) {
        res = getBox(arr, i);
        l = res[0]; name = res[1];

        if (box_name == name) {
            sub_box = arr.slice(i, i+l)
        }
        i = i + l;
    }
    return sub_box;
}

function findFirstSampleFlag(arr) { // input Uint8Array
    // [moof [mfhd] [traf [tfhd] [tfdt] [trun]]]

    var traf = getSubBox(arr, "traf");
    if (traf==null) { return false; }

    var trun = getSubBox(traf, "trun");
    if (trun==null) { return false; }

    // ISO/IEC 14496-12:2012(E) .. pages 5 and 57
    // bytes: (size 4), (name 4), (version 1 + tr_flags 3)
    var flags = trun.slice(10,13); // console.log(flags);
    f = flags[1] & 4; // console.log(f);
    return f == 4;
}

这篇关于在分段的实时MP4流中发送定期的元数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！