[Arduino SDK] Audio encoder for Opus codec

[jens6151 0 1 1]

I would like to use the opus codec to decode audio. There is the option AS_CODECTYPE_OPUS. However there is no file to flash like for MP3.

SPRESENSE/hardware/spresense/2.5.2/libraries/Audio/Audio.cpp tries to load a DSP file OPUSENC. However that file is not available to download on the documentation.

Where can I get the OPUSENC file?

If I cannot get it, what is the interface to integrate libopus myself?

Background:
The idea is to stream audio over the LTEM connection for remote listening. I would need a codec that produces least data. In a quick test the connection was roughly 16 KB/s, but I do not want to max this out if possible. Quality would not be top priority. I assume speech would be most interesting to stream.

What also concerned me is that the MP3ENC uses 2 memory tiles and memory is sparse as I also use camera and a display with a JCK font. So I want to reduce memory consumption as much as possible.

[CamilaSouza]

@jens6151-0-1-1

This feature is unfortunately not available right now.
The documentation will be rectified to proper reflect the types that we do support.

If you'd like to do this yourseld, you would have to implement the DSP firmware to encode the audio and audio DSP driver to communicate with the DSP.
https://developer.sony.com/develop/spresense/docs/sdk_developer_guide_en.html#_about_layer_structure

In any case, we don't recommend this since this format requires a lot of processing power and it may not be possible to encode audio in real time.

[jens6151 0 1 1]

@CamilaSouza
Thanks for your investigations and recommendations.

I think I will lower the priority for now and later reconsider what to do.

My original plan to stream audio also has also other obstacles.

• My SIM provider requires a special plan to allow initiating connections from remote ("web server"). That plan costs more than 20x of the monthly fee of the data plan. Too much for me.
• Audio requires a lot of memory. 2 tiles for MP3 encoder, 2 tiles for memory, buffers for the Audio Library etc. I would like to spend more memory for higher quality camera images.
  • btw MEMORY_UTIL_TINY reduces 1 tile for memory I did not see this in the documentation. Might be worth mentioning?

[CamilaSouza]

@jens6151-0-1-1

Hmmmm I understand
Wish I could help out more.

Thank you for the heads up about the documentation. I'll investigate why it isn't there and suggest the correction.

[jens6151 0 1 1]

@CamilaSouza
I gave it a try to integrate libopus. I was able to cross compile the library, integrate it in a sketch for the main core and then port the encoding part to a sub core. I was streaming in real time with 1 channel 48 KHz complexity 0 setting over Serial to ffplay with an ogg format.
I did not succeed in creating a DSP binary, likely due to size/memory constraints. It communicates when opus encoder is not initialized or in use. For now I am fine with my own ASMP worker.

If anyone is interested, all the details are here.
https://github.com/jens6151/opus-audio-encoding-on-Spresense

[CamilaSouza]

@jens6151-0-1-1
Thank you for sharing your solution!
That is great.
I will take a look at it in more detail when I have some spare time