Slow refresh rate streaming the camera preview to a display

[jens6151 0 1 1]

@jens6151-0-1-1
There seems to be an issue with the loop that some data is skipped. Need to check that maybe later.

[yokonav]

@jens6151-0-1-1 said in Slow refresh rate streaming the camera preview to a display:

There seems to be an issue with the loop that some data is skipped. Need to check that maybe later.

Thanks for your time! By the way I was checking the function call imageproc_convert_yuv2rgb which seems using the hardware to convert YUV to RGB. They are setting some register value ROT_RGB_ALIGNMENT which looks like the order but I am not sure. If we can directly output BGR instead of RGB, would it be same as colors swapping?

[jens6151 0 1 1]

@yokonav
I do not think that ROT_RGB_ALIGNMENT is related to the byte order we need.
I assume it is for the RGB 888 or BGR 888 format. (1 byte per color)
We have 5-6-5 bit for the color channels. That makes up 2 bytes or 16 bit.
Byte order means that 1st byte (bit 0-7) and 2nd byte (8-15) is exchanged for a 2 byte number.

[yokonav]

@jens6151-0-1-1
I think it is RGB565. See the code snippet below from the Arduino SDK.

CamErr CamImage::convertPixFormat(CAM_IMAGE_PIX_FMT to_fmt)
{
  CAM_IMAGE_PIX_FMT from_fmt = getPixFormat();
  int               width    = getWidth();
  int               height   = getHeight();
  uint8_t           *buff    = getImgBuff();

  if (buff == NULL)
    {
      return CAM_ERR_NOT_PERMITTED;
    }

  switch (from_fmt)
    {
      case CAM_IMAGE_PIX_FMT_YUV422:
        switch (to_fmt)
          {
            case CAM_IMAGE_PIX_FMT_RGB565:
              imageproc_convert_yuv2rgb(buff, width, height);

The code converts YUV422 (16bit) input to RGB565 (16bit) output and overwrites the input buffer with the output by the HW.

[jens6151 0 1 1]

@yokonav
I tried to change this line to use 1 or 2. But I do not see any difference. I hope I did not make any mistake. I recompiled, replaced the sdk, cleaned by deleting all intemerdiate files ...

  putreg32(0, ROT_RGB_ALIGNMENT);

Without any documentation difficult to say. Even the https://www.sony-semicon.co.jp/products/common/pdf/CXD5602_user_manual.pdf did not mention the registers.
@CamilaSouza Is it possible that the 2D Graphics hardware acceleration returns "display byte order"?

[jens6151 0 1 1]

@yokonav
Just had the idea to look up the display handling inside the Spresense SDK.

This looks promising but might not be supported.
spresense-sdk/spresense/nuttx/include/nuttx/spi/spi.h

#ifdef CONFIG_SPI_HWFEATURES
  /* If there are multiple SPI drivers, some may not support hardware
   * feature selection.
   */

...
   *   Bit 4: HWFEAT_LSBFIRST
   *          Data transferred LSB first (default is MSB first)
   */
...

Need to continue to investigate on the sdk samples later.

[yokonav]

@jens6151-0-1-1

This should be working.

putreg32(1, ROT_RGB_ALIGNMENT);

@jens6151-0-1-1

There is an entry for the register in the https://github.com/sonydevworld/spresense/blob/master/sdk/tools/SVD/rot.xml file.

<register>
      <name>RGB_ALIGNMENT</name>
      <description>RGB format selector</description>
      <addressOffset>0x38</addressOffset>
      <access>read-write</access>
      <resetValue>0x00000000</resetValue>
      <resetMask>0x00000001</resetMask>

      <fields>
        <field>
          <name>FORMAT</name>
          <description>RGB Format</description>
          <bitRange>[0:0]</bitRange>
          <enumeratedValues>
            <enumeratedValue><name>RGB</name><value>0</value></enumeratedValue>
            <enumeratedValue><name>BGR</name><value>1</value></enumeratedValue>
          </enumeratedValues>
        </field>
      </fields>
    </register>

[jens6151 0 1 1]

@yokonav
It seems like imageproc_convert_yuv2rgb is not called. It is only called if you convert formats, but not called if you set the desired format in the first place.
I tried forcing a call by converting formats. Unfortunately it was slow. I assume the asm optimized conversion is the best trade off for now.

[jens6151 0 1 1]

@yokonav
I used the imageproc for rotation and found out that it automatically converts byte order, even if you set to no change. This was my code with rotation.
It took about 1-2ms.

  putreg32(1, ROT_INTR_ENABLE);
  putreg32(0, ROT_INTR_DISABLE);

  putreg32(hsize, ROT_SET_SRC_HSIZE);
  putreg32(vsize, ROT_SET_SRC_VSIZE);
  putreg32(CXD56_PHYSADDR(ibuf), ROT_SET_SRC_ADDRESS);

  putreg32(hsize, ROT_SET_SRC_PITCH);
  putreg32(CXD56_PHYSADDR(obuf), ROT_SET_DST_ADDRESS);

  putreg32(hsize, ROT_SET_DST_PITCH);

  putreg32(0, ROT_CONV_CTRL);     // 0:NOCONVERT 1:YCBCR422_RGB565 2:RGB565_YCBCR422
  putreg32(0, ROT_RGB_ALIGNMENT); // 0:RGB; 1:BGR
  putreg32(2, ROT_SET_DIRECTION); // No rotation  Right 90 degrees  Right 180 degrees   Right 270 degrees
  putreg32(1, ROT_COMMAND);       // start rotation

[jens6151 0 1 1]

short addition.
Input and Output is different in my case.