[Ovmsdev] ESP-IDF v4 / v5 - baby steps

Tue Jan 24 05:19:28 HKT 2023

Michael,

Thanks for your help. FYI I've just pushed the last touches to the 
vehicle components which are now all converted to cmake and "should 
work". Still compiles under 3.3.x.

A new things that need to be checked : in 
https://github.com/llange/Open-Vehicle-Monitoring-System-3/commit/f31117a48d4e293b2be01e8b48c1d580d0af834e 
i had to replace `INT` with another type. This time I choose `int32_t` 
but I'm not really sure because I could not find a way to see where and 
how INT was defined. I could only verify (by compilation tricks) that 
sizeof(INT) == sizeof(int32_t).

Regarding cmake, I understand your concern with dependencies, and I did 
have the same reaction. There certainly is a way - I did not spend too 
much time on it. We will however have some "bugs" in the current state 
of things when disabling components in menuconfig where a dependency was 
implied and is now exposed. I'll let the cmake experts guide us :-)

I'll now work on the compilation warnings.

Btw, another thing that I need to mention is that from 3.3 to 5.x, many 
constants (used in sdkconfig) have been renamed etc... So our sdkconfig 
is upgraded by the build toolchain when we compile with 4.x or 5.x. I 
recommend doing a backup if it has been customized - in case of.

Regards,

Ludovic

Le 23/01/2023 à 21:11, Michael Balzer a écrit :
> Ludovic,
>
> really great & appreciated you're going ahead on this. I second 
> skipping idf 4 and going directly for version 5. I'll try to get my 
> idf 5 build environment up ASAP.
>
> I need to get into cmake myself as well. Needing to hard code 
> dependencies doesn't sound good though, I thought cmake is supposed to 
> be smart about this.
>
> I'll have a look at porting the custom crash handler to idf 5. There 
> were some other additions that haven't been accepted by Espressif, 
> IIRC. I'll see if I can check them as well.
>
> Regards,
> Michael
>
>
> Am 23.01.23 um 11:04 schrieb Ludovic LANGE:
>>
>> Hi list,
>>
>> I am interested in upgrading our codebase to a latest ESP-IDF version 
>> (preferentially v5 which has just happened end of 2022 - Happy New 
>> Year ! :-)) - mostly to be able to use some components for which a 
>> backport seems difficult (for example : 
>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752 
>> ), and as a matter of balance between stability vs latest bugfixes, 
>> security fixes, new features etc...
>>
>> In the past it seems that some endeavours already set the basis to 
>> make this happen 
>> (https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263, 
>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360 
>> ).
>>
>> I'm now trying to revive this effort, and after some work managed to 
>> have:
>>
>>   * OVMS codebase building under v4.x using Makefile
>>   * OVMS codebase building under v4.x using cmake / idf.py (not 100%
>>     finished, multiple vehicle_* components are missing)
>>
>> (/*WARNING*: some important features of OVMS have been deactivated 
>> for the moment, see end of post)/
>>
>> The produced binaries seem to "work on my machine board", as they end 
>> with a "OVMS#" prompt and a few (very) basic tests look OK (to be 
>> honest, unfortunately no real tests have been done except to check 
>> that the binary does not crash during boot and a few seconds of run, 
>> checking that WiFi / web Interface is OK, sdcard is OK - but very basic)
>>
>>
>> You'll find the changes here : 
>> https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network
>> (In the README.md, there is a short notice at the top with a few 
>> useful informations)
>>
>>
>> I'm now reaching out here mostly for :
>>
>>   * gathering feedback both around the effort (is it worth pursuing
>>     this goal ?) and the implementation (do my CMakeLists.txt look
>>     weird ? yes they do.)
>>   * having volunteers using those builds on real-world workload
>>     (which I cannot do at the moment) and finding bugs
>>   * guidance and fixes regarding the CMakeLists.txt files
>>
>>
>> My next steps could be (of course, subject to feedback) :
>>
>>  1. Have 100% of the cmake implementation finished (all the missing
>>     vehicle_* components) -> easy (only missing time)
>>  2. Work on the warnings (currently disabled in the builds) and try
>>     to fix as much as possible
>>  3. Iron out all the bugs, improve the dev documentation -> need help
>>     on finding them with real-world use
>>  4. 1st goal is to propose this result as a merge request (either as
>>     a whole, or split depending on feedback - ideally I would like to
>>     have a first (big) MR accepted quickly so that people can play
>>     with it, and deliver incremental updates as expected)
>>  5. then work on the v5 and do the same.
>>
>>
>> (Please note that my cmake skills are inexistent - so be careful when 
>> opening those files to not hurt yourself.)
>>
>> # The guidelines I tried to follow:
>>
>>   * Stay compatible with our 3.3.x branch which should still compile
>>     without any issue nor loss of functionality
>>   * Keep Makefiles running for 3.3.x (and 4.x if possible)
>>   * When possible, isolate differences (between 3.x, 4.x, 5.x) by
>>     testing the versions of ESP-IDF.
>>   * Focus on 5.x (if there are impossibilities, let's make it easier
>>     to build v5.x than v4.x)
>>   * https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html
>>   * https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system
>>   * https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html
>>   * https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html
>>
>>
>> # Missing features:
>>
>> In the course of this work I had to make some decisions to speed up 
>> the work  - that will need to be acted upon:
>>
>>   * I disabled the crash handler (`xt_set_error_handler_callback` and
>>     `esp_task_wdt_get_trigger_tasknames`) for the moment, we need to
>>     decide whether we "fork" ESP-IDF again to port it ; or if the new
>>     APIs are enough to (partially ?) reimplement it
>>   * There is a crash in `OvmsConsole::Poll` which I did not analyse
>>     (yet) and which I worked around by declaring a variable static.
>>   * I decided to transform our local copies of `wolfssh` and
>>     `wolfssl` in submodules (and move them one level below in terms
>>     of directories) - mainly to be able to have a CMakeLists.txt
>>     different from the upstream one.
>>     In the process, I "lost" one of our patches :
>>     https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20
>>     and we need to decide how to handle this.
>>   * A lot of dependencies are now explicitly (hard-)coded in the
>>     CMakeLists.txt - which may, or may not be a good thing. Let's
>>     discuss it.
>>   * I had to change a set of defines into a header generation (in
>>     ovms_webserver) because I was unable to implement those in a
>>     satisfying manner in cmake. If you manage to do it, it's a win !
>>   * Build warnings were disabled to keep the output clean and focus
>>     on the build issues - but now they need to be re-activated.
>>   * etc...
>>   * Even if there is experimental support for cmake / idf.py in
>>     ESP-IDF 3.3.x, I did not make any effort to be compatible with it:
>>       o Our own fork does not seem to include this build system
>>       o It was experimental, the component API changed and I did not
>>         want to have edge case for something not used at the moment.
>>
>>
>> Let me know your comments and questions.
>>
>> Best regards,
>>
>
> -- 
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20230123/b4135b11/attachment.htm>