如何在IOS中使用OpusCodec编码和解码实时音频?

我已经搜索了很多有关如何在IOS中使用OpusCodec的方法，但是还没有解决方案.如何使用OpusCodec编码和解码此数据?我需要jitterBuffer吗?如果我需要如何在IOS中使用它此Opus编解码器代码，但声音不清晰 #import"OpusManager.h"#import< opus/opus.h>#定义SAMPLE_RATE 16000#定义频道1#define BITRATE SAMPLE_RATE *渠道/***音频帧大小*除以时间.通话时，必须使用正好一帧(2.5毫秒的倍数:2.5、5、10、20、40、60毫秒).* Fs/毫秒2.5 5 10 20 40 60* 8kHz 20 40 80 160 320 480* 16kHz 40 80160320640640960* 24KHz 601202404804809601440* 48kHz 120240480480960 1920 2880*/#定义FRAME_SIZE 320#定义应用程序OPUS_APPLICATION_VOIP#定义MAX_PACKET_BYTES(FRAME_SIZE * CHANNELS * sizeof(float))#定义MAX_FRAME_SIZE(FRAME_SIZE * CHANNELS * sizeof(float))typedef opus_int16 OPUS_DATA_SIZE_T;@implementation OpusManager {OpusEncoder * _encoder;OpusDecoder * _decoder;}整数大小；int错误；未签名的char encodingPacket [MAX_PACKET_BYTES];-(instancetype)init {自我= [超级初始化];如果(自己){大小= opus_encoder_get_size(CHANNELS);_encoder = malloc(大小);错误= opus_encoder_init(_encoder，SAMPLE_RATE，CHANNELS，APPLICATION);_encoder = opus_encoder_create(SAMPLE_RATE，CHANNELS，APPLICATION，& error);_decoder = opus_decoder_create(SAMPLE_RATE，CHANNELS和&error);opus_encoder_ctl(_encoder，OPUS_SET_BITRATE(BITRATE));opus_encoder_ctl(_encoder，OPUS_SET_COMPLEXITY(10));opus_encoder_ctl(_encoder，OPUS_SET_SIGNAL(OPUS_SIGNAL_VOICE));opus_encoder_ctl(_encoder，OPUS_SET_VBR(0));opus_encoder_ctl(_encoder，OPUS_SET_APPLICATION(APPLICATION));opus_encoder_ctl(_encoder，OPUS_SET_DTX(1));opus_encoder_ctl(_encoder，OPUS_SET_INBAND_FEC(0));opus_encoder_ctl(_encoder，OPUS_SET_BANDWIDTH(12000));opus_encoder_ctl(_encoder，OPUS_SET_PACKET_LOSS_PERC(1));opus_encoder_ctl(_encoder，OPUS_SET_INBAND_FEC(1));opus_encoder_ctl(_encoder，OPUS_SET_FORCE_CHANNELS(CHANNELS));opus_encoder_ctl(_encoder，OPUS_SET_PACKET_LOSS_PERC(1));}回归自我}-(NSData *)编码:(NSData *)PCM {opus_int16 * PCMPtr =(opus_int16 *)PCM.bytes;int PCMSize =(int)PCM.length/sizeof(opus_int16);opus_int16 * PCMEnd = PCMPtr + PCMSize;NSMutableData * mutData = [NSMutableData数据];未签名的char encodingPacket [MAX_PACKET_BYTES];//记录作品块大小OPUS_DATA_SIZE_T encodingBytes = 0;而(PCMPtr + FRAME_SIZE< PCMEnd){encodeBytes = opus_encode_float(_encoder，(const float *)PCMPtr，FRAME_SIZE，encodePacket，MAX_PACKET_BYTES);如果(encodedBytes< = 0){NSLog(@错误:encodedBytes< = 0");返回零；}NSLog(@"encodedBytes:％d"，encodeBytes);//保存作品块大小[mutData appendBytes:& encodedBytes长度:sizeof(encodedBytes)]；//保存作品数据[mutData appendBytes:encodedPacket长度:encodedBytes];PCMPtr + = FRAME_SIZE;}NSLog(@"mutData:％lu"，(unsigned long)mutData.length);NSLog(@"encodedPacket:％s"，encodePacket);返回mutData.length>0?mutData:nil;}-(NSData *)解码:(NSData *)opus {unsigned char * opusPtr =(unsigned char *)opus.bytes;int opusSize =(int)opus.length;无符号字符* opusEnd = opusPtr + opusSize;NSMutableData * mutData = [NSMutableData数据];浮动解码的数据包[MAX_FRAME_SIZE]；int的decodedSamples = 0;//保存数据以获取opus块大小OPUS_DATA_SIZE_T nBytes = 0;while(opusPtr< opusEnd){//取出opus块大小数据nBytes = *(OPUS_DATA_SIZE_T *)opusPtr;opusPtr + = sizeof(nBytes);encodedSamples = opus_decode_float(_decoder，opusPtr，nBytes，decodedPacket，MAX_FRAME_SIZE，0);如果(decodedSamples< = 0){NSLog(@错误:decodedSamples< = 0");返回零；}NSLog(@"decodedSamples:％d"，encodesSamples);[mutData appendBytes:decodedPacket长度:decodedSamples * sizeof(opus_int16)]];opusPtr + = nBytes;}NSLog(@"mutData:％lu"，(unsigned long)mutData.length);返回mutData.length>0?mutData:nil;}@结尾 解决方案我建议您尝试使用比特率和带宽.我已成功使其与此处描述的参数一起使用: https://ddanilov.me/how-to-在带FEC中为作品编解码器/启用.I am working on a app which has following requirements :Record real time audio from iOS device (iPhone)Encode this audio data to Opus data and send it to server over WebSocketDecode received data to pcm againPlay received audio from WebSocket server on iOS device(iPhone)I've used AVAudioEngine for this. var engine = AVAudioEngine() var input: AVAudioInputNode = engine.inputNode var format: AVAudioFormat = input.outputFormat(forBus: AVAudioNodeBus(0)) input.installTap(onBus: AVAudioNodeBus(0), bufferSize: AVAudioFrameCount(8192), format: format, block: { buf, when in // ‘buf' contains audio captured from input node at time 'when' }) // start engineAnd I converted AVAudioPCMBuffer to Data using this functionfunc toData(PCMBuffer: AVAudioPCMBuffer) -> Data { let channelCount = 1 let channels = UnsafeBufferPointer(start: PCMBuffer.floatChannelData, count: channelCount) let ch0Data = NSData(bytes: channels[0], length:Int(PCMBuffer.frameLength * PCMBuffer.format.streamDescription.pointee.mBytesPerFrame)) return ch0Data as Data}I've found Opus Library from CocoaPod libopus libopusI have searched a lot on how to use OpusCodec in IOS but haven't got the solution.How to encode and decode this data using OpusCodec ? And Am I need jitterBuffer ? If I need How to use it in IOSThis Code for Opus Codec but voice doesn't clear#import "OpusManager.h"#import <opus/opus.h>#define SAMPLE_RATE 16000#define CHANNELS 1#define BITRATE SAMPLE_RATE * CHANNELS/*** Audio frame size* It is divided by time. When calling, you must use the audio data ofexactly one frame (multiple of 2.5ms: 2.5, 5, 10, 20, 40, 60ms).* Fs/ms 2.5 5 10 20 40 60* 8kHz 20 40 80 160 320 480* 16kHz 40 80 160 320 640 960* 24KHz 60 120 240 480 960 1440* 48kHz 120 240 480 960 1920 2880*/#define FRAME_SIZE 320#define APPLICATION OPUS_APPLICATION_VOIP#define MAX_PACKET_BYTES (FRAME_SIZE * CHANNELS * sizeof(float))#define MAX_FRAME_SIZE (FRAME_SIZE * CHANNELS * sizeof(float))typedef opus_int16 OPUS_DATA_SIZE_T;@implementation OpusManager { OpusEncoder *_encoder; OpusDecoder *_decoder;}int size;int error;unsigned char encodedPacket[MAX_PACKET_BYTES];- (instancetype)init { self = [super init]; if (self) { size = opus_encoder_get_size(CHANNELS); _encoder = malloc(size); error = opus_encoder_init(_encoder, SAMPLE_RATE, CHANNELS, APPLICATION); _encoder = opus_encoder_create(SAMPLE_RATE, CHANNELS, APPLICATION, &error); _decoder = opus_decoder_create(SAMPLE_RATE, CHANNELS, &error); opus_encoder_ctl(_encoder, OPUS_SET_BITRATE(BITRATE)); opus_encoder_ctl(_encoder, OPUS_SET_COMPLEXITY(10)); opus_encoder_ctl(_encoder, OPUS_SET_SIGNAL(OPUS_SIGNAL_VOICE)); opus_encoder_ctl(_encoder, OPUS_SET_VBR(0)); opus_encoder_ctl(_encoder, OPUS_SET_APPLICATION(APPLICATION)); opus_encoder_ctl(_encoder, OPUS_SET_DTX(1)); opus_encoder_ctl(_encoder, OPUS_SET_INBAND_FEC(0)); opus_encoder_ctl(_encoder, OPUS_SET_BANDWIDTH(12000)); opus_encoder_ctl(_encoder, OPUS_SET_PACKET_LOSS_PERC(1)); opus_encoder_ctl(_encoder, OPUS_SET_INBAND_FEC(1)); opus_encoder_ctl(_encoder, OPUS_SET_FORCE_CHANNELS(CHANNELS)); opus_encoder_ctl(_encoder, OPUS_SET_PACKET_LOSS_PERC(1)); } return self;}- (NSData *)encode:(NSData *)PCM { opus_int16 *PCMPtr = (opus_int16 *)PCM.bytes; int PCMSize = (int)PCM.length / sizeof(opus_int16); opus_int16 *PCMEnd = PCMPtr + PCMSize; NSMutableData *mutData = [NSMutableData data]; unsigned char encodedPacket[MAX_PACKET_BYTES]; // Record opus block size OPUS_DATA_SIZE_T encodedBytes = 0; while (PCMPtr + FRAME_SIZE < PCMEnd) { encodedBytes = opus_encode_float(_encoder, (const float *) PCMPtr, FRAME_SIZE, encodedPacket, MAX_PACKET_BYTES); if (encodedBytes <= 0) { NSLog(@"ERROR: encodedBytes<=0"); return nil; } NSLog(@"encodedBytes: %d", encodedBytes); // Save the opus block size [mutData appendBytes:&encodedBytes length:sizeof(encodedBytes)]; // Save opus data [mutData appendBytes:encodedPacket length:encodedBytes]; PCMPtr += FRAME_SIZE; } NSLog(@"mutData: %lu", (unsigned long)mutData.length); NSLog(@"encodedPacket: %s", encodedPacket); return mutData.length > 0 ? mutData : nil;}- (NSData *)decode:(NSData *)opus { unsigned char *opusPtr = (unsigned char *)opus.bytes; int opusSize = (int)opus.length; unsigned char *opusEnd = opusPtr + opusSize; NSMutableData *mutData = [NSMutableData data]; float decodedPacket[MAX_FRAME_SIZE]; int decodedSamples = 0; // Save data for opus block size OPUS_DATA_SIZE_T nBytes = 0; while (opusPtr < opusEnd) { // Take out the opus block size data nBytes = *(OPUS_DATA_SIZE_T *)opusPtr; opusPtr += sizeof(nBytes); decodedSamples = opus_decode_float(_decoder, opusPtr, nBytes,decodedPacket, MAX_FRAME_SIZE, 0); if (decodedSamples <= 0) { NSLog(@"ERROR: decodedSamples<=0"); return nil; } NSLog(@"decodedSamples:%d", decodedSamples); [mutData appendBytes:decodedPacket length:decodedSamples *sizeof(opus_int16)]; opusPtr += nBytes; } NSLog(@"mutData: %lu", (unsigned long)mutData.length); return mutData.length > 0 ? mutData : nil;}@end 解决方案 I would suggest playing around with bitrate and bandwidth.I have succeeded to make it work with the parameters described here:https://ddanilov.me/how-to-enable-in-band-fec-for-opus-codec/. 这篇关于如何在IOS中使用OpusCodec编码和解码实时音频?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！