[Ovmsdev] ESP-IDF v4 / v5 - baby steps

Sat Feb 25 09:07:19 HKT 2023

Hi Michael,

Thanks for your time in reviewing those patches.

Do not hesitate to tell me when I can rework a PR to simplify it, or 
make it more readable.

If you still have some energy, a new round of PRs is ready here : 
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pulls?q=is%3Aopen+is%3Apr+author%3Allange+draft%3Afalse

After this round, there will still be 2 or 3 PRs that will apply 
cleanly, then we will have to discuss a few things:

  * We need to re-implement (if possible) the crash handling that were
    implemented as part as our ESP-IDF fork
    (`xt_set_error_handler_callback` and
    `esp_task_wdt_get_trigger_tasknames`). There is a new API
    (`esp_core_dump_image_get` and `esp_core_dump_get_summary`) but I'm
    not familiar enough with the crash handling mechanism (both the old
    one, and the new API) to handle it at the moment :-)
    I just disabled it in the `experimental-esp-idf-build-workflow`
    branch, which is not the proper approach

  * `spi_flash_erase_range` has been replaced by
    `esp_flash_erase_region` (but with esp_flash_default_chip that may
    need to be initialised via `esp_flash_init()`) - it is used in
    `module_perform_factoryreset` of
    `vehicle/OVMS.V3/main/ovms_module.cpp`. We would need to check this
    (I just disabled it, not good)

  * Our ESP-IDF fork added a field `uxMutexesHeld` to `struct
    xTASK_STATUS` :
    https://github.com/openvehicles/esp-idf/commit/95e43fc2c4360f8126a0985bf037a293bebe3767
    , which is then used in `module_tasks()` in
    `vehicle/OVMS.V3/main/ovms_module.cpp`. Do we want to continue to
    have this field (and need to maintain a fork of ESP-IDF) ?

  * Mongoose SSL (using mbedTLS) is a complex issue... We may be able to
    switch to the latest Mongoose be we would loose a lot of our patches
    we are very difficult to port to the new version (which changed a
    lot). A very small number of those patches seems not needed any
    more, but for the others we need to decide what to do if we pursue
    this route.

  * And many more interesting discussions :-)

Regarding ESP-IDF v4 : I used it because it was an intermediary step 
(between v3 and v5) with less deprecations, so it was easy to port from 
v3. ESP-IDF v5 introduced a lot of deprecations and changes, but having 
done the v4 port gave me confidence that it could be done.
(/Note: v4 needs a patch to support WHOLE_ARCHIVE - which I don't know 
if it is important, or not, to have/)

As ESP-IDF v5 is a "recent" release (5.0.0 is 2022-12-2, 5.0.1 is 
2023-02-17), it may be possible that some of our dependencies out there 
are not (yet) ready to be compatible with v5 (while they are OK with v4):

  * For example wolfSSL (I know I know... but we may need it for
    wolfSSH) seems to have some issues related to ESP-IDFv5 +
    hardware-acceleration,
  * https://github.com/trombik/esp_wireguard is not (yet) working with
    v5:
    https://github.com/trombik/esp_wireguard/issues/33#issuecomment-1430615198
    (This is a pet project of mine :
    https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752
    )

So, to answer your question, I'll try to keep v4 compatibility in my 
branches for some time in case it's needed - but it's perfectly OK to 
skip it in the "official" OVMSv3 if it works fine without.

Regards,

Le 24/02/2023 à 17:34, Michael Balzer a écrit :
> Ludovic,
>
> I've merged all your PRs, including the two "touchy" ones. I've seen 
> no issues but a couple of bug fixes included.
>
> I still need to test these with idf5 (i.e. the new idf5 branch state), 
> idf3 builds & runs fine.
>
> The IDF version switches of course don't add readability, as expected. 
> But if they can be kept that small, I don't see any problems.
>
> I would opt for excluding the IDF 4 branches, to aid in keeping these 
> as small as possible. I don't see any advantage in IDF 4 over 5 -- do you?
>
> Regards,
> Michael
>
>
> Am 22.02.23 um 17:23 schrieb Ludovic LANGE:
>> Hi Mark,
>>
>> You're certainly right : WolfSSH writes here 
>> https://github.com/wolfSSL/wolfssh : "wolfSSH is dependent on 
>> wolfCrypt, found as a part of wolfSSL".
>>
>> So it seems we need to bundle it still, and it's a good thing we 
>> managed to have it more or less compile with ESP-IDF 5+ (a few bugs 
>> are still being ironed out - especially the OPENSSL compatibility 
>> layer, which may not be used if mongoose does use mbedTLS instead).
>>
>> Regarding the integration, I tried (my best :-)) to have it build on 
>> both ESP-IDF 3, ESP-IDF 5 (and also ESP-IDF 4 with a small patch to 
>> the ESP-IDF build system itself). Of course I need help for testing / 
>> reproducing the builds and I'm grateful to those of you sparing some 
>> time to do exactly that.
>>
>> I have another round of PRs waiting for some peer-review : 
>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pulls?q=is%3Aopen+is%3Apr+author%3Allange+draft%3Afalse 
>>
>>
>> -> @List, do not hesitate to have a look - especially for the ones 
>> that are a little bit touchy:
>>
>>   * https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/831
>>     is trying to guess whether the multiple switch case falls-through
>>     are wanted, or not. It impacts the code of multiple people, so it
>>     would be interesting if those people could have a look at it.
>>   * https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/835
>>     is also a big piece and would benefit from a peer-review.
>>
>> Thanks in advance !
>>
>> Regards,
>>
>> Le 20/02/2023 à 08:07, Mark Webb-Johnson a écrit :
>>> Michael, Ludovic,
>>>
>>>> Actually one of the next things I wanted to try/determine is why 
>>>> you want/need to use wolfSSL instead of mbedTLS, which we've been 
>>>> using since wolfSSL failed so miserably about the Let's encrypt 
>>>> root certificate transition. Is there an issue with mbedTLS in idf5?
>>>
>>> My memory was that wolfssh required wolfssl? But I may be mistaken.
>>>
>>>> We still need to decide on the integration way. Mark's suggestion 
>>>> was doing the idf5 transition in a separate branch that will later 
>>>> become the new "master". That would keep the idf3 build clean from 
>>>> any regressions, but need merging any work done on that branch into 
>>>> the idf5 branch, which may become a lot of additional work, 
>>>> depending on the time needed to finish the transition.
>>>
>>> If this can cleanly be applied to the existing work, to allow 
>>> building under either v3 or v5 IDF, that would be ideal. IMHO, the 
>>> separate branch would only be required if the changes were very 
>>> extensive and breaking to v3.
>>>
>>> Regards, Mark.
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
> -- 
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20230225/8be7c800/attachment-0001.htm>