Code always got stuck while it should not, whether the core powerful enough? Also have questions for MP issues

[Larry]

Hi All,

I modified the Arduino IDE built-in AudioFFT example and use the additional subcore2 to display the spectrum. While it also has a MP-related issue. The subcore2 cannot receive the message from subcore1 and it also shows stack dump.

Would someone take a look at this (Please check the attached folder for the code)? https://drive.google.com/drive/folders/19XyTPO4F3Pp5x9N8d9BEyB55u6U25Rg-?usp=sharing
This is a very small project and should be simple to debug. While I have no idea about this issue right now.

I also try another similar open-source project online https://www.hackster.io/karl-sony/spresense-audio-scope-e0c3d3. The original project uses ST7735, I change it to make it works for ILI9341. It also got stuck after running for a few seconds.

The change from ST7735 to ILI9341 will make the whole project not work? this is state machine related?

I suspect the computation performance of this board (maybe it is my bad). Does it mean the audio capturing and TFT displaying must be in different cores?

Finally, for a multicore application:

1. Whether flashing .ino to board sequence matter? It seems I can either flash the main core application or flash the subscore application.
2. Is there any method I can erase the application to subcore? For example. if I use the main core, subcore1, and subcore2 for project A, then I only use the main core for project B. The previous application for project A still exists on subcores. Are there any methods I can erase them? Or it is totally not needed?

Thank you in advance for any help!

Best,

Larry

[CamilaSouza]

Hey, @Larry
It feels like we have 2 problems. A problem in the communication of the two cores and maybe a memory issue. You know what I would try?
If I'm not mistaken the other built in examples communicate from main core to subcore and the way back (like maincore <-> subcore1, maincore <-> subcore2, etc.) but not actually maincore <-> subcore1 <-> subcore2.

So I would first make the subcore2 code as simple as possible. To just make it light a led or something, to debug the communication from subcore1 <-> subcore2.

After I was sure the communication was working, I would add the rest of the code to actually receive the spectrum and show on the display.

If the communication is working and stack dump is still there, I would play around with the memory allocated to the maincore. I found this note in the documentation:

In this case, default SubCore memory will not be enough, so you need to select Tools→ Memory→ 640kB from the Arduino IDE to set the MainCore memory size to 640kB.
It wasn't referring to another case, but it might be helpful to try.

Regarding your other questions:

1. The sequence of flashing does not matter. You only need to pay attention if on the bottom left of your arduino screen it shows the right core is selected. If you change the core selection for one arduino ide window, it can change to all the others open, so that might cause some confusion.
2. You can erase the application if you flash bootloader again. Please refer to https://developer.sony.com/develop/spresense/docs/arduino_tutorials_en.html#_notices
   But that is not necessary since SubCore does not run unless you call MP.begin() from MainCore. Therefore, there is no problem even if the old SubCore remains on your Spresense board.

[Larry]

@CamilaSouza Hi Camila, Thank you for your kind reply. I still have communication issues. I think all existing MP examples are maincore <-> subcore1, maincore <-> subcore2, while based on my modification, the audio fft with TFT display indeed becomes maincore <-> subcore1 <-> subcore2. To be specific, the project actually conducted in this way: main core MP.Send to subcore1, subcore1 MP.Recv main core and after fft processing, MP.Send to subcore2, subcore2 only has MP.Recv for subcore1 for displaying.

I minimize the code in subcore2 and only a printf exists in the loop. However, the code still got stuck after running, I have no idea why it has such an issue. If possible, would you please take a look?
https://drive.google.com/drive/folders/1vNwJ5casIAMxAaEScYTRkHyugOF5fPts?usp=sharing
The basic 'maincore <-> subcore1, maincore <-> subcore2' is working, while my structure is not working while it should.

Anyway， this project should be working based on the current structure. I got the ideas from this video: https://www.bilibili.com/video/BV1ZU4y1a7CY?spm_id_from=333.337.search-card.all.click&vd_source=2fb4c4d8419494f431c2260343a2dcf9

Thank you in advance for any help.

Best,

Larry

[CamilaSouza]

@Larry
Hmmm.. interesting video.
Is your code exactly the same as the one they are using? I can't see anything haha It's all blurry for me. Did they share the code anywhere (github for example)?

I'm taking a look at the maincore <-> subcore1 <-> subcore2 portion of the code to see if I can figure out the issue.

[CamilaSouza]

Hey! @Larry
So I did some digging..

I created very simple sketches just with the maincore <-> subcore1 <-> subcore2 communication. At first I was getting the same error you were, even in a minimal application.
Then I realized that it only worked when I removed the print statements from the subcores.
I think that since I had no delays in my subcores tasks, the speed that I was sending the prints was just way too much.

So my first advice would be to remove any prints from the subcores.
Notice that in the Arduino MessageData example there are no prints in the code for the subcore.

I'm giving you my minimal sketch where the communication is working. My advice, if the removal of the prints is not enough, is to add your code to this minimal sketch little by little to see where it breaks. And where you might need to add delays for example.

MAINCORE:

/*
 *  MainAudio.ino - MP Example for Audio FFT 
 *  Copyright 2019 Sony Semiconductor Solutions Corporation
 *
 *  This library is free software; you can redistribute it and/or
 *  modify it under the terms of the GNU Lesser General Public
 *  License as published by the Free Software Foundation; either
 *  version 2.1 of the License, or (at your option) any later version.
 *
 *  This library is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 *  Lesser General Public License for more details.
 *
 *  You should have received a copy of the GNU Lesser General Public
 *  License along with this library; if not, write to the Free Software
 *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 */

#ifdef SUBCORE
#error "Core selection is wrong!!"
#endif

#include <MP.h>

const int subcore = 1;
const int render_core = 2;

void setup()
{
  int ret;

  Serial.begin(115200);
  while (!Serial);

  /* Launch SubCore1 */
  
  ret = MP.begin(subcore);
  if (ret < 0) {
    printf("MP.begin subcore1 error = %d\n", ret);
  }

  /* Launch SubCore2 */
  
  ret = MP.begin(render_core);
  if (ret < 0) {
    printf("MP.begin subcore2 error = %d\n", ret);
  }
  
}

void loop()
{
  int ret;
  uint32_t snddata = 22;
  int8_t   sndid = 100; /* user-defined msgid */

  printf("Send: id=%d data=0x%08lx\n", sndid, snddata);

  ret = MP.Send(sndid, snddata, subcore);
  if (ret < 0) {
    printf("Main Core Send Fail = %d\n", ret); 
  }

}

SUBCORE1:

/*
 *  SubFFT.ino - MP Example for Audio FFT 
 *  Copyright 2019 Sony Semiconductor Solutions Corporation
 *
 *  This library is free software; you can redistribute it and/or
 *  modify it under the terms of the GNU Lesser General Public
 *  License as published by the Free Software Foundation; either
 *  version 2.1 of the License, or (at your option) any later version.
 *
 *  This library is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 *  Lesser General Public License for more details.
 *
 *  You should have received a copy of the GNU Lesser General Public
 *  License along with this library; if not, write to the Free Software
 *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 */

#if (SUBCORE != 1)
#error "Core selection is wrong!!"
#endif

#include <MP.h>

const int main_core = 0;
const int fft_core = 1;
const int render_core = 2;

void setup()
{
  ret = MP.begin();
  
  /* receive with non-blocking */
  MP.RecvTimeout(MP_RECV_POLLING);
}

void loop()
{
  int      ret;
  int8_t   msgid;
  int8_t   sndid = 66;
  uint32_t snddata = 35;  // Liwei debug use
  uint32_t msgdata;
  
  ret = MP.Recv(&msgid, &msgdata, main_core);
  if (ret >= 0) {
    ret = MP.Send(sndid, snddata, render_core);
  }

}

SUBCORE2:

#if (SUBCORE != 2)
#error "Core selection is wrong!!"
#endif

#include <MP.h>

const int main_core = 0;
const int fft_core = 1;
const int render_core = 2;
 
void setup()
{
  ret = MP.begin();
}

void loop()
{
  int      ret = 0;
  int8_t   msgid;
  uint32_t msgdata;
  /* Echo back */

  ret = MP.Recv(&msgid, &msgdata, fft_core);
}

[Larry]

@CamilaSouza Hi Camila, Thank you for your kind reply. I tried your code. I am wondering how you determine that the communication between subcore1 and subcore2 is working without the usage of print? And it is indeed very strange and abnormal that the code will stop working after adding printf. Based on your explanation, does it mean that after adding delays, then I can use print in subcore2?

If I hope to complete this project, does it mean that I must use subcore2 for display since the main core and subcore1 do not have enough computation capacity for doing so?

If possible, would you please still take about 10 minutes to try this online project created by Sony: https://forum.developer.sony.com/topic/665/spresense-fft-analyzer-example-not-working?_=1657255949810

Thank you in advance for any help.

Best,

Larry

[CamilaSouza]

Hey, @Larry

I simply tried this with the prints first and it was not working. Then I removed the prints and realized it did work. When you ask for a print, the subcore actually sends the data to the maincore so it can print. So I guess since I was printing in a very high speed, it was too much data being sent between cores.

I'm not one hundred percent sure if you will need the subcore2 or not, I think you might have to test that out.

I'll answer the other project in the post you originally created it