本文介绍了使用AVAudioConverter Swift将AAC解码为PCM格式的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

如何在Swift上使用AVAudioConverter,AVAudioCompressedBuffer和AVAudioPCMBuffer将AAC转换为PCM?

How convert AAC to PCM using AVAudioConverter, AVAudioCompressedBuffer and AVAudioPCMBuffer on Swift?

在WWDC 2015上,有人说507 Session,AVAudioConverter可以对PCM缓冲区进行编码和解码,仅显示了编码示例,但未显示带有解码的示例.我尝试解码,但无法正常工作.我不知道什么:(

On WWDC 2015, 507 Session was said, that AVAudioConverter can encode and decode PCM buffer, was showed encode example, but wasn't showed examples with decoding.I tried decode, and something doesn't work. I don't know what:(

通话:

//buffer - it's AVAudioPCMBuffer from AVAudioInputNode(AVAudioEngine)
let aacBuffer = AudioBufferConverter.convertToAAC(from: buffer, error: nil) //has data
let data = Data(bytes: aacBuffer!.data, count: Int(aacBuffer!.byteLength)) //has data
let aacReverseBuffer = AudioBufferConverter.convertToAAC(from: data) //has data
let pcmReverseBuffer = AudioBufferConverter.convertToPCM(from: aacBuffer2!, error: nil) //zeros data. data object exist, but filled by zeros

用于转换的代码:

class AudioBufferFormatHelper {

    static func PCMFormat() -> AVAudioFormat? {

        return AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 44100, channels: 1, interleaved: false)
    }

    static func AACFormat() -> AVAudioFormat? {

        var outDesc = AudioStreamBasicDescription(
                mSampleRate: 44100,
                mFormatID: kAudioFormatMPEG4AAC,
                mFormatFlags: 0,
                mBytesPerPacket: 0,
                mFramesPerPacket: 0,
                mBytesPerFrame: 0,
                mChannelsPerFrame: 1,
                mBitsPerChannel: 0,
                mReserved: 0)
        let outFormat = AVAudioFormat(streamDescription: &outDesc)
        return outFormat
    }
}

class AudioBufferConverter {

    static func convertToAAC(from buffer: AVAudioBuffer, error outError: NSErrorPointer) -> AVAudioCompressedBuffer? {

        let outputFormat = AudioBufferFormatHelper.AACFormat()
        let outBuffer = AVAudioCompressedBuffer(format: outputFormat!, packetCapacity: 8, maximumPacketSize: 768)

        self.convert(from: buffer, to: outBuffer, error: outError)

        return outBuffer
    }

    static func convertToPCM(from buffer: AVAudioBuffer, error outError: NSErrorPointer) -> AVAudioPCMBuffer? {

        let outputFormat = AudioBufferFormatHelper.PCMFormat()
        guard let outBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat!, frameCapacity: 4410) else {
            return nil
        }

        outBuffer.frameLength = 4410
        self.convert(from: buffer, to: outBuffer, error: outError)

        return outBuffer
    }

    static func convertToAAC(from data: Data) -> AVAudioCompressedBuffer? {

        let nsData = NSData(data: data)
        let inputFormat = AudioBufferFormatHelper.AACFormat()
        let buffer = AVAudioCompressedBuffer(format: inputFormat!, packetCapacity: 8, maximumPacketSize: 768)
        buffer.byteLength = UInt32(data.count)
        buffer.packetCount = 8

        buffer.data.copyMemory(from: nsData.bytes, byteCount: nsData.length)
        buffer.packetDescriptions!.pointee.mDataByteSize = 4

        return buffer
    }

    private static func convert(from sourceBuffer: AVAudioBuffer, to destinationBuffer: AVAudioBuffer, error outError: NSErrorPointer) {

        //init converter
        let inputFormat = sourceBuffer.format
        let outputFormat = destinationBuffer.format
        let converter = AVAudioConverter(from: inputFormat, to: outputFormat)

        converter!.bitRate = 32000

        let inputBlock : AVAudioConverterInputBlock = { inNumPackets, outStatus in

            outStatus.pointee = AVAudioConverterInputStatus.haveData
            return sourceBuffer
        }

        _ = converter!.convert(to: destinationBuffer, error: outError, withInputFrom: inputBlock)
    }
}

结果AVAudioPCMBuffer的数据为零.在邮件中,我看到了错误:

In result AVAudioPCMBuffer has data with zeros.And in messages I see errors:

AACDecoder.cpp:192:Deserialize:  Unmatched number of channel elements in payload
AACDecoder.cpp:220:DecodeFrame:  Error deserializing packet
[ac] ACMP4AACBaseDecoder.cpp:1337:ProduceOutputBufferList: (0x14f81b840) Error decoding packet 1: err = -1, packet length: 0
AACDecoder.cpp:192:Deserialize:  Unmatched number of channel elements in payload
AACDecoder.cpp:220:DecodeFrame:  Error deserializing packet
[ac] ACMP4AACBaseDecoder.cpp:1337:ProduceOutputBufferList: (0x14f81b840) Error decoding packet 3: err = -1, packet length: 0
AACDecoder.cpp:192:Deserialize:  Unmatched number of channel elements in payload
AACDecoder.cpp:220:DecodeFrame:  Error deserializing packet
[ac] ACMP4AACBaseDecoder.cpp:1337:ProduceOutputBufferList: (0x14f81b840) Error decoding packet 5: err = -1, packet length: 0
AACDecoder.cpp:192:Deserialize:  Unmatched number of channel elements in payload
AACDecoder.cpp:220:DecodeFrame:  Error deserializing packet
[ac] ACMP4AACBaseDecoder.cpp:1337:ProduceOutputBufferList: (0x14f81b840) Error decoding packet 7: err = -1, packet length: 0

推荐答案

您的尝试存在一些问题:

There were a few problems with your attempt:

  1. 在转换数据-> AVAudioCompressedBuffer时,您未设置 multiple 数据包描述.您需要创建它们,因为AAC数据包的大小可变.您可以从原始AAC缓冲区复制它们,也可以手动(ouch)或使用AudioFileStream api从数据中解析它们.

  1. you're not setting the multiple packet descriptions when you convert data -> AVAudioCompressedBuffer. You need to create them, as AAC packets are of variable size. You can either copy them from the original AAC buffer, or parse them from your data by hand (ouch) or by using the AudioFileStream api.

您一次又一次地重新创建AVAudioConverter-对每个缓冲区一次,丢弃它们的状态.例如AAC编码器出于自身的原因,需要添加2112帧静音,然后才能开始再现音频,因此重新创建转换器会使您完全安静.

you re-create your AVAudioConverters over and over again - once for each buffer, throwing away their state. e.g. the AAC encoder for its own personal reasons needs to add 2112 frames of silence before it can get around to reproducing your audio, so recreating the converter gets you a whole lot of silence.

您将相同的缓冲区一遍又一遍地提供给AVAudioConverter的输入块.每个缓冲区只能显示一次.

you present the same buffer over and over to the AVAudioConverter's input block. You should only present each buffer once.

32000的比特率不起作用(对我来说)

the bit rate of 32000 didn't work (for me)

这就是我现在能想到的.尝试对您的代码进行以下修改,现在您将其称为:

That's all I can think of right now. Try the following modifications to your code instead which you now call like so:

(请注意,我将某些单声道更改为立体声,因此我可以在Mac上播放往返缓冲区,因为其麦克风输入是立体声,您可能需要将其改回)

(p.s. I changed some of the mono to stereo so I could play the round trip buffers on my mac, whose microphone input is strangely stereo - you might need to change it back)

(pps这里显然有某种往返/串行化/反序列化的尝试,但是您到底想做什么?您想将AAC音频从一台设备流到另一台设备吗?因为它可能更容易让另一个AVPlayer这样的API会播放结果流,而不是自己处理数据包)

(p.p.s there's obviously some kind of round trip / serialising/deserialising attempt going on here, but what exactly are you trying to do? do you want to stream AAC audio from one device to another? because it might be easier to let another API like AVPlayer play the resulting stream instead of dealing with the packets yourself)

let aacBuffer = AudioBufferConverter.convertToAAC(from: buffer, error: nil)!
let data = Data(bytes: aacBuffer.data, count: Int(aacBuffer.byteLength))
let packetDescriptions = Array(UnsafeBufferPointer(start: aacBuffer.packetDescriptions, count: Int(aacBuffer.packetCount)))
let aacReverseBuffer = AudioBufferConverter.convertToAAC(from: data, packetDescriptions: packetDescriptions)!
// was aacBuffer2
let pcmReverseBuffer = AudioBufferConverter.convertToPCM(from: aacReverseBuffer, error: nil)

class AudioBufferFormatHelper {

    static func PCMFormat() -> AVAudioFormat? {
        return AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 44100, channels: 1, interleaved: false)
    }

    static func AACFormat() -> AVAudioFormat? {

        var outDesc = AudioStreamBasicDescription(
            mSampleRate: 44100,
            mFormatID: kAudioFormatMPEG4AAC,
            mFormatFlags: 0,
            mBytesPerPacket: 0,
            mFramesPerPacket: 0,
            mBytesPerFrame: 0,
            mChannelsPerFrame: 1,
            mBitsPerChannel: 0,
            mReserved: 0)
        let outFormat = AVAudioFormat(streamDescription: &outDesc)
        return outFormat
    }
}

class AudioBufferConverter {
    static var lpcmToAACConverter: AVAudioConverter! = nil

    static func convertToAAC(from buffer: AVAudioBuffer, error outError: NSErrorPointer) -> AVAudioCompressedBuffer? {

        let outputFormat = AudioBufferFormatHelper.AACFormat()
        let outBuffer = AVAudioCompressedBuffer(format: outputFormat!, packetCapacity: 8, maximumPacketSize: 768)

        //init converter once
        if lpcmToAACConverter == nil {
            let inputFormat = buffer.format

            lpcmToAACConverter = AVAudioConverter(from: inputFormat, to: outputFormat!)
//            print("available rates \(lpcmToAACConverter.applicableEncodeBitRates)")
//          lpcmToAACConverter!.bitRate = 96000
            lpcmToAACConverter.bitRate = 32000    // have end of stream problems with this, not sure why
        }

        self.convert(withConverter:lpcmToAACConverter, from: buffer, to: outBuffer, error: outError)

        return outBuffer
    }

    static var aacToLPCMConverter: AVAudioConverter! = nil

    static func convertToPCM(from buffer: AVAudioBuffer, error outError: NSErrorPointer) -> AVAudioPCMBuffer? {

        let outputFormat = AudioBufferFormatHelper.PCMFormat()
        guard let outBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat!, frameCapacity: 4410) else {
            return nil
        }

        //init converter once
        if aacToLPCMConverter == nil {
            let inputFormat = buffer.format

            aacToLPCMConverter = AVAudioConverter(from: inputFormat, to: outputFormat!)
        }

        self.convert(withConverter: aacToLPCMConverter, from: buffer, to: outBuffer, error: outError)

        return outBuffer
    }

    static func convertToAAC(from data: Data, packetDescriptions: [AudioStreamPacketDescription]) -> AVAudioCompressedBuffer? {

        let nsData = NSData(data: data)
        let inputFormat = AudioBufferFormatHelper.AACFormat()
        let maximumPacketSize = packetDescriptions.map { $0.mDataByteSize }.max()!
        let buffer = AVAudioCompressedBuffer(format: inputFormat!, packetCapacity: AVAudioPacketCount(packetDescriptions.count), maximumPacketSize: Int(maximumPacketSize))
        buffer.byteLength = UInt32(data.count)
        buffer.packetCount = AVAudioPacketCount(packetDescriptions.count)

        buffer.data.copyMemory(from: nsData.bytes, byteCount: nsData.length)
        buffer.packetDescriptions!.pointee.mDataByteSize = UInt32(data.count)
        buffer.packetDescriptions!.initialize(from: packetDescriptions, count: packetDescriptions.count)

        return buffer
    }


    private static func convert(withConverter: AVAudioConverter, from sourceBuffer: AVAudioBuffer, to destinationBuffer: AVAudioBuffer, error outError: NSErrorPointer) {
        // input each buffer only once
        var newBufferAvailable = true

        let inputBlock : AVAudioConverterInputBlock = {
            inNumPackets, outStatus in
            if newBufferAvailable {
                outStatus.pointee = .haveData
                newBufferAvailable = false
                return sourceBuffer
            } else {
                outStatus.pointee = .noDataNow
                return nil
            }
        }

        let status = withConverter.convert(to: destinationBuffer, error: outError, withInputFrom: inputBlock)
        print("status: \(status.rawValue)")
    }
}

这篇关于使用AVAudioConverter Swift将AAC解码为PCM格式的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

07-23 15:14
查看更多