[Ovmsdev] ESP-IDF v4 / v5 - baby steps

[Michael Balzer]

Ludovic,

really great & appreciated you're going ahead on this. I second skipping 
idf 4 and going directly for version 5. I'll try to get my idf 5 build 
environment up ASAP.

I need to get into cmake myself as well. Needing to hard code 
dependencies doesn't sound good though, I thought cmake is supposed to 
be smart about this.

I'll have a look at porting the custom crash handler to idf 5. There 
were some other additions that haven't been accepted by Espressif, IIRC. 
I'll see if I can check them as well.

Regards,
Michael


Am 23.01.23 um 11:04 schrieb Ludovic LANGE:
>
> Hi list,
>
> I am interested in upgrading our codebase to a latest ESP-IDF version 
> (preferentially v5 which has just happened end of 2022 - Happy New 
> Year ! :-)) - mostly to be able to use some components for which a 
> backport seems difficult (for example : 
> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752 
> ), and as a matter of balance between stability vs latest bugfixes, 
> security fixes, new features etc...
>
> In the past it seems that some endeavours already set the basis to 
> make this happen 
> (https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263, 
> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360 
> ).
>
> I'm now trying to revive this effort, and after some work managed to have:
>
>   * OVMS codebase building under v4.x using Makefile
>   * OVMS codebase building under v4.x using cmake / idf.py (not 100%
>     finished, multiple vehicle_* components are missing)
>
> (/*WARNING*: some important features of OVMS have been deactivated for 
> the moment, see end of post)/
>
> The produced binaries seem to "work on my machine board", as they end 
> with a "OVMS#" prompt and a few (very) basic tests look OK (to be 
> honest, unfortunately no real tests have been done except to check 
> that the binary does not crash during boot and a few seconds of run, 
> checking that WiFi / web Interface is OK, sdcard is OK - but very basic)
>
>
> You'll find the changes here : 
> https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network
> (In the README.md, there is a short notice at the top with a few 
> useful informations)
>
>
> I'm now reaching out here mostly for :
>
>   * gathering feedback both around the effort (is it worth pursuing
>     this goal ?) and the implementation (do my CMakeLists.txt look
>     weird ? yes they do.)
>   * having volunteers using those builds on real-world workload (which
>     I cannot do at the moment) and finding bugs
>   * guidance and fixes regarding the CMakeLists.txt files
>
>
> My next steps could be (of course, subject to feedback) :
>
>  1. Have 100% of the cmake implementation finished (all the missing
>     vehicle_* components) -> easy (only missing time)
>  2. Work on the warnings (currently disabled in the builds) and try to
>     fix as much as possible
>  3. Iron out all the bugs, improve the dev documentation -> need help
>     on finding them with real-world use
>  4. 1st goal is to propose this result as a merge request (either as a
>     whole, or split depending on feedback - ideally I would like to
>     have a first (big) MR accepted quickly so that people can play
>     with it, and deliver incremental updates as expected)
>  5. then work on the v5 and do the same.
>
>
> (Please note that my cmake skills are inexistent - so be careful when 
> opening those files to not hurt yourself.)
>
> # The guidelines I tried to follow:
>
>   * Stay compatible with our 3.3.x branch which should still compile
>     without any issue nor loss of functionality
>   * Keep Makefiles running for 3.3.x (and 4.x if possible)
>   * When possible, isolate differences (between 3.x, 4.x, 5.x) by
>     testing the versions of ESP-IDF.
>   * Focus on 5.x (if there are impossibilities, let's make it easier
>     to build v5.x than v4.x)
>   * https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html
>   * https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system
>   * https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html
>   * https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html
>
>
> # Missing features:
>
> In the course of this work I had to make some decisions to speed up 
> the work  - that will need to be acted upon:
>
>   * I disabled the crash handler (`xt_set_error_handler_callback` and
>     `esp_task_wdt_get_trigger_tasknames`) for the moment, we need to
>     decide whether we "fork" ESP-IDF again to port it ; or if the new
>     APIs are enough to (partially ?) reimplement it
>   * There is a crash in `OvmsConsole::Poll` which I did not analyse
>     (yet) and which I worked around by declaring a variable static.
>   * I decided to transform our local copies of `wolfssh` and `wolfssl`
>     in submodules (and move them one level below in terms of
>     directories) - mainly to be able to have a CMakeLists.txt
>     different from the upstream one.
>     In the process, I "lost" one of our patches :
>     https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20
>     and we need to decide how to handle this.
>   * A lot of dependencies are now explicitly (hard-)coded in the
>     CMakeLists.txt - which may, or may not be a good thing. Let's
>     discuss it.
>   * I had to change a set of defines into a header generation (in
>     ovms_webserver) because I was unable to implement those in a
>     satisfying manner in cmake. If you manage to do it, it's a win !
>   * Build warnings were disabled to keep the output clean and focus on
>     the build issues - but now they need to be re-activated.
>   * etc...
>   * Even if there is experimental support for cmake / idf.py in
>     ESP-IDF 3.3.x, I did not make any effort to be compatible with it:
>       o Our own fork does not seem to include this build system
>       o It was experimental, the component API changed and I did not
>         want to have edge case for something not used at the moment.
>
>
> Let me know your comments and questions.
>
> Best regards,
>

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20230123/d193d88e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20230123/d193d88e/attachment-0001.sig>