[Ovmsdev] esp-idf update fixing spi_flash mutex priority inheritance bug

[Michael Balzer]

Everyone,

TL;DR: update your local esp-idf clone from our esp-idf repository 
before doing the next firmware build.

While testing the Mongoose API lock I found out the Mongoose task 
priority would still occasionally get raised to 22 = wifi task priority. 
In combination with the Mongoose task essentially being 100% busy while 
outside the locked poll call, this lead to blocking all other tasks from 
Mongoose, which caused at least one of the crash effects observed 
(watchdog timeout from our events task). Adding a priority fix to our 
netmanager eliminated quite a lot of these crashes.

I then investigated this, as I thought that priority bug was originally 
coming from the buggy POSIX mutex implementation we fixed in July 2020 
(→ http://lists.openvehicles.com/pipermail/ovmsdev/2020-July/006971.html).

It turned out I was wrong, the actual culprit is a bug in the esp-idf 
spi_flash component: each access to the SPI flash memory needs to be 
running at maximum priority. The spi_flash methods did this by 
temporarily changing the current task priority, and reverting to the 
previous priority without taking into account that the task may have had 
an inherited priority from an aquired mutex lock. Thus the priority 
inherited from e.g. the Wifi task would stick.

That was especially present and reproducable when opening the web UI's 
Config→Firmware page, as that page handler reads the OTA status, which 
in turn reads the current boot configuration from flash. It also 
affected the AutoFlash task during firmware updates, and there may be 
more paths, basically running any "ota" command via a network channel.

As config reads & writes also use SPI flash, these also could produce 
the bug for any task trying to lock some mutex also being requested by a 
higher priority task.

This SPI flash bug has been found by other esp-idf users, and has 
finally been fixed, but only for esp-idf 4.3 & higher:

  * https://github.com/espressif/esp-idf/issues/5116
  * https://github.com/espressif/esp-idf/issues/7580

I have now backported the fix to our version, and haven't had a single 
unplanned priority change since.

Positive side effect: I see no event queue overflows & almost no effect 
on the overall performance during an OTA flash process now.

This is though probably not the only cause of remaining watchdog 
timeouts -- crash reports will tell.

Regards,
Michael

-- 
Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
Fon 02330 9104094 * Handy 0176 20698926

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260127/2eb6b61a/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260127/2eb6b61a/attachment.sig>