Opening of files and message queues takes long time (10s-30s)

[jens6151 0 1 1]

Open commands sometimes take a lot of time. Usually it is fast, but sometimes it takes really long which is 10 seconds or 30 seconds.

What is in common is that it is opening some path. It does not matter if it is internal flash or SD card or a message queue.

Any hints what could cause this?

I have several threads. Open commands might interleave. SMP is not active.
I tried to increase some configuration like NFILE_DESCRIPTORS_PER_BLOCK, FS_NEPOLL_DESCRIPTORS or FS_MQUEUE_NPOLLWAITERS, but there is hardly any documentation about it, so I do not exactly know what it does.

These are sample calls that take long.

#define ROOTCA_FILE "/mnt/spif/cert/root-CA.crt"
File rootCertsFile = File(ROOTCA_FILE, FILE_READ);

#define QUEUE_MAXMSG 2
#define QUEUE_MSGSIZE 420
#define QUEUE_MQTT "/MQ_MQTT"

struct mq_attr attr = {QUEUE_MAXMSG, QUEUE_MSGSIZE, 0, 0};
mqd_t mqfd = mq_open(QUEUE_MQTT, O_RDWR | O_CREAT, 0666, &attr);

[jens6151 0 1 1]

OK. For the files I found out that another location takes up the time.

  File rootCertsFile = File(ROOTCA_FILE, FILE_READ);
  if (rootCertsFile) {
    client.setCACert(rootCertsFile, rootCertsFile.available());
    rootCertsFile.close();
  }

client.setCACert is slow.

void LTETLSClient::setCertificate(Stream& stream, size_t size)
...
  if (size != stream.readBytes(_clientCA, size)) {
    delete[] _clientCA;
    _clientCA = NULL;
    return;
  }

stream.readBytes is slow

size_t Stream::readBytes(char *buffer, size_t length)
{
  size_t count = 0;
  while (count < length) {
        int c = timedRead();
    if (c < 0) break;
    *buffer++ = (char)c;
    count++;
  }
  return count;
}
...
int Stream::timedRead()
{
  int c;
  _startMillis = millis();
  do {
    c = read();
    if (c >= 0) return c;
  } while(millis() - _startMillis < _timeout);
  return -1;     // -1 indicates timeout
}

It copies byte by byte from the file.

I will look around the mq_open in more detail.

Looking into the LTE and ArduinoMqtt libraries, there are several locations copying byte by byte ...

[CamilaSouza]

@jens6151-0-1-1 said in Opening of files and message queues takes long time (10s-30s):

Looking into the LTE and ArduinoMqtt libraries, there are several locations copying byte by byte ...

I see.
I'll share this feedback with the team and investigate if there is a way to make this process faster.
Will get back to you if I find a solution.

[jens6151 0 1 1]

@CamilaSouza You can overload with File additional to Stream.
When you know it is a file then you can use File::read(void *buf, size_t nbyte) defined in your File class.

[CamilaSouza]

Hi @jens6151-0-1-1

We have reproduced the issue, and it will be fixed in next SDK release

Thank you for your support in finding this issues so we can improve our SDK.