音频采样添加使用水槽作家在Windows媒体基金会的视频

问题描述

我可以写使用我与这个样本此处。它采用 IMFSample 和 IMFSinkWriter 。现在我想音频添加到它。假设有Audio.wma文件，我想在视频文件写到这个音频。

I can write a video file using images which I learned with this sample here. It uses IMFSample and IMFSinkWriter. Now I want to add audio to it. Suppose there is Audio.wma file and I want this audio to be written in that video file.

但无法弄清楚如何做到这样本。事情是这样的输入和放大器;输出类型设置， IMFSample 音频缓冲创作等，这将是一个伟大的，如果有人可以告诉我如何使用音频作家沉添加到视频文件。

But cannot figure out how to do that in this sample. Things like input & output type setup, IMFSample creation for audio buffer etc. It would be a great if someone could show me how to add audio to a video file using sink writer.

推荐答案

媒体基金会是伟大的工作，我确信你将能够快速修改项目以完成这件事。

Media Foundation is great to work with, and I am certain you will be able to quickly modify your project to get this done.

概述：结果
创建新 IMFMediaSource 从音频文件中读取的样本，增加一个音频流水槽，最后交错使用相应的流索引水槽写道。

OVERVIEW:
Create new IMFMediaSource to read the samples from the audio file, add an audio stream to the sink, and finally interleave the sink writes using the corresponding stream index.

联系方式：

修改 VideoGenerator :: InitializeSinkWriter（..）函数正确初始化水槽，以适应音频流。在该功能，正确地创建 audioTypeOut 和 audioTypeIn （ IMFMediaType ）。您可能要重命名的 mediaTypeOut 和 mediaTypeIn 为 videoTypeOut 和 videoTypeIn 为清楚起见，这似乎像下面这样：

Modify the VideoGenerator::InitializeSinkWriter(..) function to properly initialize the sink to accommodate the audio stream. In that function, properly create the audioTypeOut and audioTypeIn (IMFMediaType). You may want to rename mediaTypeOut and mediaTypeIn to videoTypeOut and videoTypeIn for clarity, which would appear like the following:

ComPtr<IMFMediaType>  videoTypeOut;  // <-- previously mediaTypeOut
ComPtr<IMFMediaType>  videoTypeIn;   // <-- previously mediaTypeIn
ComPtr<IMFMediaType>  audioTypeOut = nullptr;
ComPtr<IMFMediaType>  audioTypeIn = nullptr;

接下来，配置您的视频类型兼容的音频输出类型。既然你似乎是创建一个Windows媒体视频，你可能会想使用 MFAudioFormat_WMAudioV9 。以确保每次采样的通道，采样率，和比特是正确的，我一般列举可用的类型，并找到所希望的特性，类似于以下（错误检查已被忽略）：

Next, configure the output audio type compatible with your video type. Since you appear to be creating a windows media video, you will likely want to use MFAudioFormat_WMAudioV9. To ensure the channels, sample rate, and bits per sample are correct, I generally enumerate the available types and find the desired characteristics, similar to the following (error checking has been omitted):

ComPtr<IMFCollection> availableTypes = nullptr;
HRESULT hr = MFTranscodeGetAudioOutputAvailableTypes(MFAudioFormat_WMAudioV9, MFT_ENUM_FLAG_ALL, NULL, availableTypes.GetAddressOf());

DWORD count = 0;
hr = availableTypes->GetElementCount(&count));  // Get the number of elements in the list.

ComPtr<IUnknown>     pUnkAudioType = nullptr;
ComPtr<IMFMediaType> audioOutputType = nullptr;
for (DWORD i = 0; i < count; ++i)
{
    hr = availableTypes->GetElement(i, pUnkAudioType.GetAddressOf());
    hr = pUnkAudioType.Get()->QueryInterface(IID_PPV_ARGS(audioTypeOut.GetAddressOf()));

    // compare channels, sampleRate, and bitsPerSample to target numbers
    {
        // audioTypeOut is set!
        break;
    }

    audioOutputType.Reset();
}

availableTypes.Reset();

如果audioTypeOut设置成功，这种类型的流添加到水槽，并得到结果索引：

if audioTypeOut was set successfully, add that type of stream to the sink and get the resulting index:

hr = sinkWriter->AddStream(audioTypeOut.Get(), &audioStreamIndex);
audioTypeOut.Reset();  // <-- audioTypeOut not needed anymore

最后的接收器，音频输入类型必须设置，这将取决于你正在阅读的文件，以及音频源（ IMFMediaSource ）。马上有更多介绍，但添加了音频输入水槽将类似于以下内容：

Finally for the sink, the audio input type must be set, and that will depend on the file you are reading, and the audio source (IMFMediaSource). More on that shortly, but adding the audio input to the sink would look similar to the following:

ComPtr<IMFMediaType> audioTypeIn = nullptr;  // <-- declaration from above
// NOTE: audioReader is an IMFMediaSource used to read the audio file
hr = audioReader->GetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, audioTypeIn.GetAddressOf());
hr = sinkWriter->SetInputMediaType(_audioOutStreamIndex, audioTypeIn.Get(), nullptr);
audioTypeIn.Reset();

有可用于创建许多例子在 audioReader （ IMFMediaSource ），并从文件中读取样品，但的是简单和直接的。在code是

There are many examples available to create the audioReader (IMFMediaSource) and read samples from the file, but this one is simple and straight forward. The code is here.

最后，写你会发现，是很容易的，因为水槽可以直接拿样品音频（ IMFSample ），您从文件中读取。你来管理写操作，但一种解决方案是交织的写入（视频/音频）。音频采样的持续时间处理，但您将需要重订的时间戳。保证写入水槽，当你有正确的流索引。

Finally, writing the audio you will find to be really easy, since the sink can take the samples directly (IMFSample) that you read from the file. You get to manage the writes, but one solution is to interleave the writes (video / audio). The duration of the audio sample is handled, but you will need to rebase the timestamp. Ensure you have the correct stream index when writing to the sink.

阅读使用的样品和异步回调：

Reading samples using and async callback:

// if you are using an async callback, the function would look similar to the following:
HRESULT OnReadAudioSample(HRESULT status, DWORD streamIndex, DWORD streamFlags, LONGLONG timestamp, IMFSample *sample)
{
    // .. other code
    hr = sample->SetSampleTime(timestamp - _baseRecordTime);
    hr = sinkWriter->WriteSample(audioStreamIndex, sample);
    // .. other code

    // trigger the next asyc read...
    hr = audioReader->ReadSample((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, 0, nullptr, nullptr, nullptr, nullptr);
}

阅读样本同步：

// otherwise, you will only use a synchronous read
hr = audioReader->ReadSample((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, 0, nullptr, &dwFlags, &timestamp, &sample);
hr = sample->SetSampleTime(timestamp - _baseRecordTime);
hr = sinkWriter->WriteSample(audioStreamIndex, sample);
hr = WriteFrame(target.get(), rtStart, rtDuration);  // <-- write video frame as before

听起来像一个有趣的小项目。祝你好运，玩得开心，希望这有助于！

Sounds like a fun little project. Good luck, have fun, and hope this helps!

这篇关于音频采样添加使用水槽作家在Windows媒体基金会的视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！