Building a custom Image Classification Model

[KamilTomaszewski]

Hi @suburban-daredevil

input_num is the number of inputs you defined in the array inputs for your neural network. input_num equals to tflm_runtime_input_num()
output_index is index to specify an output. You can check the number of outputs of your neural network using tflm_runtime_output_num()
dim_indexis index to specify a dimension. You can check the number of dimensions of your specific output using tflm_runtime_output_ndim()

Below is a short code that uses these functions to print information about the output of your neural network:

  int output_num = tflm_runtime_output_num(&rt);
  printf("output num: %d\n", output_num);
  for (int i = 0; i < output_num; i++)
  {
    printf("output: %d, size: %d\n", i, tflm_runtime_output_size(&rt, i));
    printf("output: %d, dim num: %d\n", i, tflm_runtime_output_ndim(&rt, i));
    for (int j = 0; j< tflm_runtime_output_ndim(&rt, i); j++)
    {
      printf("output: %d, dim: %d, shape: %d\n", i, j, tflm_runtime_output_shape(&rt, i, j));
    }
  }

Best Regards,
Kamil Tomaszewski

[suburban-daredevil]

Hi @kamiltomaszewski

Is there any resource / documentation explaining the usage of all the function calls (especially the ones in the runtime.h file) used in the tflmrt_lenet program?

Thanks

[KamilTomaszewski]

Hi @suburban-daredevil
You can find the TFLMRT documentation here: https://developer.sony.com/develop/spresense/docs/sdk_developer_guide_en.html#_tflm_runtime
tflmrt_lenet example is described here:
https://developer.sony.com/develop/spresense/docs/sdk_tutorials_en.html#_tflmrt_sample_application
You can find the description of the functions in the comments in this file:
https://github.com/sonydevworld/spresense/blob/master/sdk/modules/include/tflmrt/runtime.h

Best Regards,
Kamil Tomaszewski

[suburban-daredevil]

Hi @kamiltomaszewski

We have the tflmrt_lenet example which was trained on 28x28 grayscale images. What are the changes that has to be made and in what files (like app_main.c, pnm_util.c etc.) to accept and process RGB images of different size (say 96x96) ?

Thanks

[suburban-daredevil]

Hi @KamilTomaszewski

I just wanted to know, to embed our model's C-byte array code in our application, is it sufficient to replace the existing model0.c file's array contents with that of our new model?

I tried emedding the model0.c file's array contents with that of my new model and also changed the model length. But every time I run inference, for any given input I'm getting the same output. No change in output. Am I missing something?

And also model_tflite is the buffer that holds the model.

And network is the variable that holds our NN. It is assigned in the else part of the code block below

But I think the network variable doesn't read the builtin model and hence it is not able to perform the right inference ?

Any help is appreciated

Thanks

[KamilTomaszewski]

Hi @suburban-daredevil,

You need to change #define MNIST_SIZE_PX (28 * 28) to #define MNIST_SIZE_PX (96 * 96 * 3) in the tflmrt_lenet_main.c file and #define MY_BUFSIZ (28 * 28) to #define MY_BUFSIZ (96 * 96 * 3) in the pnm_util.c file. I think that should be enough.

Does your model array have __attribute__((aligned))?

Best Regards,
Kamil Tomaszewski

[suburban-daredevil]

Hi @kamiltomaszewski

I think you are referring to this right?

I'm not sure how to check if my new model has __attribute__((aligned))

Thanks

[KamilTomaszewski]

Hi @suburban-daredevil

Yes, that is right.

Are you using the model as a C array or as a binary file that you load from the SD card?

[suburban-daredevil]

Hi @kamiltomaszewski

I have 2 queries

• Currently I'm loading my model from the SD card. But I want to embed my model code onto my application folder itself. Is it enough to replace the contents of the model_tflite[] array with that of my new model or is there any other changes to be made?
• When running the app from the nuttx prompt, I try to give the path of the image that should be used for inference. But currently only 28*28 grayscale images are being accepted. When I try to give images of any other dimensions, it says pgm image load failed . I have made the change that you have suggested above. Are there any other changes to be made?

Thanks

[KamilTomaszewski]

Hi @suburban-daredevil

It should be enough to replace the contents of the model_tflitearray with that of your new model.

Could you check where exactly the pnm_load function returns an error?

Best Regards,
Kamil Tomaszewski

[suburban-daredevil]

Hi @KamilTomaszewski

When I try to run build and flash from VS code, I get the following error. But I do have a file called project_name.nuttx.spx inside my out directory after building.

Can you help me out with this?

Thanks

[KamilTomaszewski]

Hi @suburban-daredevil,

I think there is a bug in the latest VS code release. Could you please try an older release? For example: https://update.code.visualstudio.com/1.63.2/linux-deb-x64/stable

Best Regards,
Kamil Tomaszewski

[suburban-daredevil]

Hi @KamilTomaszewski

It worked and solved the issue. Thanks for your help

Thanks