Shared Memory tiles with activated SMP

[jens6151 0 1 1]

Can you tell me if Shared Memory works when SMP is active?

The function

int mpshm_remap(mpshm_t *shm, void *vaddr)

Does not return when you are not on CPU0 (NuttX local id).

The CPU ID is hardcoded at mpshm.c

static void *mpshm_map(int8_t tag, uintptr_t paddr, size_t size)
{
  sysctl_map_t arg;
  uint32_t va = (uint32_t)tag << 16;
  int ret;

  /* Setup map arguments
   * This API map memory range for this (NuttX) CPU. So cpuid field is
   * constant 2.
   */

  arg.cpuid = 2;
  arg.virt = va;
  arg.phys = paddr;
  arg.size = size;

  ret = cxd56_sysctlcmd(SYSCTL_MAP, (uint32_t)(uintptr_t)&arg);
  if (ret)
    {
      return NULL;
    }

  return (void *)((uintptr_t)tag << 16);
}

I quickly hacked in setting the correct id, but it does not work.

  arg.cpuid = 2 +  sched_getcpu();

What works is pinning to CPU0

#ifdef CONFIG_SMP
  cpu_set_t cpuset = 1 << 0;
  sched_setaffinity(getpid(), sizeof(cpu_set_t), &cpuset);
  sleep(2);
  printf("CPU: %d\n", sched_getcpu());
#endif

But I would like to access from all CPUs ... I worry that this works.
Do you have any hints?

Are there working projects that make use of SMP?
I am only aware of 10n2-smart-student-driver-assistant. They are not using the audio sub system as in the samples.

[jens6151 0 1 1]

Btw mem_layout.py parses an option called UseMultiCore at line 454.

What does it mean? Can it be used? There is no documentation

[CamilaSouza]

Hi @jens6151-0-1-1

mpshm_remap is the API for ASMP.
If you want to use the shared memory for SMP you need to use the interface of Nuttx shared memory https://nuttx.apache.org/docs/latest/reference/user/12_shared_memory.html

About projects that use SMP.
Maybe this blog post is helpful https://www.aps-web.jp/academy/amp/22917/
Also maybe check skd/apps/testing/smp/smp_main.c

About UseMultiCore, Iet me look into that one and get back to you.

[jens6151 0 1 1]

@CamilaSouza Sorry I was missing a little background. I am using this shared memory because the audio samples depend on the shared memory for message queues and audio buffers. Even if I have ASMP off in the case I only use the Microphone Frontend Object with 48 kHz and do not need any DSPs, this shared memory is required. (Is my assumption, right?)

I was not aware of Nuttx shared memory. I will check it out and see if it works well with the audio samples (only recording without DSPs).

I checked out the sample and remember this blog post (and read it again now). It very nicely explains SMP and ASMP. It even has very detailed instructions on how to try it out. However it "just" uses the sample test. What I would like to see is an SMP sample or application that uses it together with Spresense features.

I am having trouble that my application works without, but it starts freezing with SMP. (It worked before starting with adding audio, but now it froze in the early stage of my application). Reasons could be

• my programming error (likely),
• my wrong usage (SMP requires more care in synchronizing concurrency with the risk of freezing)
• or the used libraries having issues with SMP.

So I hoped to get some hints from a real-world application sample.

[jens6151 0 1 1]

@CamilaSouza Could you please check what restrictions apply with SMP?

With ASMP you restrict libraries to the main core. According to the documentation Camera, LTE, GNSS does not work on Sub Cores.

I mentioned several times that my application freezes ...
It looks like communication with other cores via FAR API must occur on Core 0 (NuttX counting).

Example: LTE communication works on other cores, but the power on command must be issued from core 0.

What can be done about it?

[jens6151 0 1 1]

@CamilaSouza I did a lot of research about this freezing issue I have.
I came to the conclusion that GNSS does not work with SMP. It causes freezes, crashes or might work. The behavior changes if you modify your code.

I can see that (likely) when receiving data from the GNSS CPU (1). Some random memory gets overwritten with ff. Depending on where this is either freeze, crash or normal behavior happens.

[CPU0] arm_hardfault: PANIC!!! Hard fault: 40000000
[CPU0] up_assert: Assertion failed CPU0 at file:armv7-m/arm_hardfault.c line: 135 task: gnss_receiver
[CPU0] up_registerdump: R0: 000000ff 000000ff 00000000 deadbeef 000000ff 2d0378b0 00000000 2d03089c
[CPU0] up_registerdump: R8: 00000000 00000000 00000000 00000000 0d013bd1 2d03ceb8 0d006413 0d0063d0
[CPU0] up_registerdump: xPSR: 61000000 BASEPRI: 00000080 CONTROL: 00000000
[CPU0] up_registerdump: EXC_RETURN: ffffffe9
[CPU0] up_dumpstate: sp:     2d02a618
[CPU0] up_dumpstate: IRQ stack:
[CPU0] up_dumpstate:   base: 2d029e80
[CPU0] up_dumpstate:   size: 00000800
[CPU0] up_dumpstate:   used: 000001b8

Could you please check?

tools/config.py examples/gnss examples/smp_asmp
# or tools/config.py examples/gnss feature/smp
# or tools/config.py examples/gnss feature/lte_stub_mbedtls feature/smp
make
tools/flash.sh -c /dev/ttyUSB0 nuttx.spk

Play a little with the configurations by modifying them.
This is not necessarily due to parallel running on the CPUs. Pinning all threads to CPU 0 also does not work.

+SMP_NCPUS=1

I even compiled https://github.com/ds-dyad/10n2-app/blob/master/sdk.config. This works. I cannot say what is the difference causing the issue or not. (I have LTE feature active?)
I also tried the GNSS sample some time ago with SMP. I remember it worked.
Like I said it seems random (but reproducible with the same build).

I already spend a lot of time. I do not know how to continue. I am about to close this investigation concluding GNSS @ SMP is not stable on Spresense.

Could you please check if there is anything I could do?

Btw SMP seems to not yet fully ready (It is considered stable, but there are open issues)

• https://github.com/apache/nuttx/pull/8383 is not yet in the spresense sdk.
• and see here https://github.com/apache/nuttx/issues/3600
  ... nuttx/arch/arm/src/cxd56xx/cxd56_cpu1signal.c still uses shed_lock() and this is related with GNSS.