Hi List, If possible, I'd like a peer-review / discussion of the following PRs: * ESP-IDF v4+: we do not have uxMutexesHeld any more #901 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/901>
In ESP-IDF 3.3.x builds of OVMS, we're using a fork of ESP-IDF. In this fork, FreeRTOS has been patched with openvehicles/esp-idf@95e43fc <https://github.com/openvehicles/esp-idf/commit/95e43fc2c4360f8126a0985bf037a293bebe3767> to add a |uxMutexesHeld| field in |TaskStatus_t|.
However, in the ESP-IDF >= 4 build we're using mainstream ESP-IDF (for the moment), which does not include this field.
So we adapt the display to handle both cases.
* ESP-IDF4+ : crash handler and boot status #902 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/902>
o error handling rework (|xt_set_error_handler_callback| is only available in our ESP-IDF 3.3.4 fork.) o |esp_task_wdt_get_trigger_tasknames()| not available (*feature removed* from ESP-IDF4+ builds) TODO o first steps to support other archs than XTensa in |boot_data.crash_data|
These are the *last 2 PRs* before being able to build with any of these ESP-IDF versions : 3.3.4-ovms, 4.4.4, 5.0, 5.0.1, branch-v5.0, branch-v5.1, latest - the last mile ! (One workaround PR <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/903> is pending to be able to *run* one of these new versions, preventing a crash at boot - but it needs a proper analysis first) Also, with these 2 PRs, we will be able to enable automatic builds on all these versions on master (using GitHub "Actions") - you can have a preview of what I mean here : https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions As we're slowly entering a period of transition between ESP-IDF 3.3.4-ovms and a more recent one (5.0.1 ?), we will need a way to ensure that the PRs / commits are OK across all these versions. Automatic builds can be configured to build on all targeted ESP-IDF versions, and can report their result back in the PRs, so that we can trap potential compilation errors before merging. (Especially as in the newer ESP-IDF versions, more and more warnings are treated as compilation errors) I also prepared a check for documentation errors using the same mechanism. Thanks in advance. Regards,
I reviewed both, and they seem ok to me. We lose some debug functionality, but that tradeoff is outweighed by the benefit of a standard unpatched IDF. I doubt whether Espressif would accept most of these patched back into their mainstream, as they are of limited use for general users. However, note that I am only really looking at this from our existing code and IDF point of view. The primary question being whether these changes would impact existing builds? When we have a complete set of changes, I think we can then start to build in v5 IDF and really start testing on real devices then. Regards, Mark
On 18 May 2023, at 6:05 AM, Ludovic LANGE <ll-ovmsdev@lange.nom.fr> wrote:
Hi List,
If possible, I'd like a peer-review / discussion of the following PRs:
ESP-IDF v4+: we do not have uxMutexesHeld any more #901 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/901>
In ESP-IDF 3.3.x builds of OVMS, we're using a fork of ESP-IDF. In this fork, FreeRTOS has been patched with openvehicles/esp-idf@95e43fc <https://github.com/openvehicles/esp-idf/commit/95e43fc2c4360f8126a0985bf037a293bebe3767> to add a uxMutexesHeld field in TaskStatus_t.
However, in the ESP-IDF >= 4 build we're using mainstream ESP-IDF (for the moment), which does not include this field.
So we adapt the display to handle both cases.
ESP-IDF4+ : crash handler and boot status #902 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/902>
error handling rework (xt_set_error_handler_callback is only available in our ESP-IDF 3.3.4 fork.) esp_task_wdt_get_trigger_tasknames() not available (feature removed from ESP-IDF4+ builds) TODO first steps to support other archs than XTensa in boot_data.crash_data
These are the last 2 PRs before being able to build with any of these ESP-IDF versions : 3.3.4-ovms, 4.4.4, 5.0, 5.0.1, branch-v5.0, branch-v5.1, latest - the last mile !
(One workaround PR <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/903> is pending to be able to run one of these new versions, preventing a crash at boot - but it needs a proper analysis first)
Also, with these 2 PRs, we will be able to enable automatic builds on all these versions on master (using GitHub "Actions") - you can have a preview of what I mean here : https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions
As we're slowly entering a period of transition between ESP-IDF 3.3.4-ovms and a more recent one (5.0.1 ?), we will need a way to ensure that the PRs / commits are OK across all these versions. Automatic builds can be configured to build on all targeted ESP-IDF versions, and can report their result back in the PRs, so that we can trap potential compilation errors before merging. (Especially as in the newer ESP-IDF versions, more and more warnings are treated as compilation errors)
I also prepared a check for documentation errors using the same mechanism.
Thanks in advance. Regards,
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Hi Ludovic Thank you for all your efforts to get ESP-IDF to the newest version. The reason why I have not been able to test along the way is, that I was not able to set up a native, non-docker, working test environment with the newer ESP-IDF versions. Is there a "step by step" documentation for setting up these newer versions without docker? That would be really helpfull. Regards Chris Am Donnerstag, dem 18.05.2023 um 00:05 +0200 schrieb Ludovic LANGE:
Hi List, If possible, I'd like a peer-review / discussion of the following PRs: * ESP-IDF v4+: we do not have uxMutexesHeld any more #901
In ESP-IDF 3.3.x builds of OVMS, we're using a fork of ESP-IDF. In this fork, FreeRTOS has been patched with openvehicles/esp- idf@95e43fc to add a uxMutexesHeld field in TaskStatus_t.However, in the ESP-IDF >= 4 build we're using mainstream ESP-IDF (for the moment), which does not include this field.So we adapt the display to handle both cases. * ESP-IDF4+ : crash handler and boot status #902
error handling rework (xt_set_error_handler_callback is only available in our ESP-IDF 3.3.4 fork.)esp_task_wdt_get_trigger_tasknames() not available (feature removed from ESP-IDF4+ builds) TODOfirst steps to support other archs than XTensa in boot_data.crash_data These are the last 2 PRs before being able to build with any of these ESP-IDF versions : 3.3.4-ovms, 4.4.4, 5.0, 5.0.1, branch-v5.0, branch-v5.1, latest - the last mile ! (One workaround PR is pending to be able to run one of these new versions, preventing a crash at boot - but it needs a proper analysis first)
Also, with these 2 PRs, we will be able to enable automatic builds on all these versions on master (using GitHub "Actions") - you can have a preview of what I mean here : https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions As we're slowly entering a period of transition between ESP-IDF 3.3.4-ovms and a more recent one (5.0.1 ?), we will need a way to ensure that the PRs / commits are OK across all these versions. Automatic builds can be configured to build on all targeted ESP-IDF versions, and can report their result back in the PRs, so that we can trap potential compilation errors before merging. (Especially as in the newer ESP-IDF versions, more and more warnings are treated as compilation errors) I also prepared a check for documentation errors using the same mechanism.
Thanks in advance. Regards, _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Hi Chris, Thanks for your kind words. For building with the newer ESP-IDF versions, I believe the 2 recent PRs that have just been merged a few minutes ago should help you progress towards this goal. If not, please share more of the issues you're facing (error messages, etc...) There is not (to my knowledge...) a real documentation, but here is a summary of the steps I can share: * First and obvious step : ensure that you have _independent_ ESP-IDF installs on your build environment, and that all are able to build the "hello world" example of the documentation ( https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linu... ) * When starting from scratch, I often remove the "build" directory (rm -rf build), and start with a known "sdkconfig" file. * For the standard build (which is at the moment based on our custom fork of ESP-IDF 3.3.4), you will certainly use your own "*sdkconfig*", or the one coming from "support/sdkconfig.default.hw31" with : "cp support/sdkconfig.default.hw31 sdkconfig") o The build command would be "make -j all" * For newest ESP-IDF, and assuming you are synch'd with master from a few minutes ago, there are is an additional step to add: o First you need to patch "mongoose" with a small patch from here : https://github.com/llange/Open-Vehicle-Monitoring-System-3/blob/build-master... o And you will use a different "*sdkconfig.defaults*" - I'd suggest using for the moment the one from here : https://github.com/llange/Open-Vehicle-Monitoring-System-3/blob/build-master... with "rm sdkconfig* ; cp /tmp/sdkconfig.defaults.esp5.0.1 sdkconfig.defaults" o The build command would be "idf.py build" Please note a different philosophy regarding the sdkconfig file: * In ESP-IDF 3.3.4 we use a *"sdkconfig"* file from the "support/" directory * while is ESP-IDF >=4 I suggest that we use a "*sdkconfig.defaults*" (which could live in support/ but could also be in vehicle/OVMS.V3 in source control) and copy it as "*sdkconfig.defaults*" The difference in approach is that with the "sdkconfig.defaults" it can live alongside you own "sdkconfig" and will only specify defaults values ; so that your changes to "sdkconfig" are not lost when we modify the "sdkconfig.defaults" file. You could have "sdkconfig.defaults" under source control and "sdkconfig" in gitignore living side by side. Cf https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/kc... You can also get inspiration from the automatic builds on my branch: Ex: https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions/runs/5017... You'll find the workflow file describing the build instructions here : https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions/runs/5017... And an example for each supported build on the left : If you click on a build, you'll find the steps that have been run, and if I ever forgot to write something here you'll find it in those logs. (Forget about the "cache" things, the various "updates" etc..) While those automated builds use docker, in fact it doesn't change the way to build. Let me know if you have issue with these builds, your separate ESP-IDF installs, or how to switch from one to the other. Regards, Le 18/05/2023 à 10:16, Chris van der Meijden a écrit :
Hi Ludovic
Thank you for all your efforts to get ESP-IDF to the newest version.
The reason why I have not been able to test along the way is, that I was not able to set up a native, non-docker, working test environment with the newer ESP-IDF versions.
Is there a "step by step" documentation for setting up these newer versions without docker? That would be really helpfull.
Regards
Chris
Hi Ludovic, thanx for your detailed response. I will give it a try. Regards Chris Am Donnerstag, dem 18.05.2023 um 21:22 +0200 schrieb Ludovic LANGE:
Hi Chris,
Thanks for your kind words.
For building with the newer ESP-IDF versions, I believe the 2 recent PRs that have just been merged a few minutes ago should help you progress towards this goal.
If not, please share more of the issues you're facing (error messages, etc...)
There is not (to my knowledge...) a real documentation, but here is a summary of the steps I can share:
* First and obvious step : ensure that you have independent ESP-IDF installs on your build environment, and that all are able to build the "hello world" example of the documentation (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get- started/linux-macos-setup.html#get-started-linux-macos-first-steps )
* When starting from scratch, I often remove the "build" directory (rm -rf build), and start with a known "sdkconfig" file.
* For the standard build (which is at the moment based on our custom fork of ESP-IDF 3.3.4), you will certainly use your own "sdkconfig", or the one coming from "support/sdkconfig.default.hw31" with : "cp support/sdkconfig.default.hw31 sdkconfig") - The build command would be "make -j all"
* For newest ESP-IDF, and assuming you are synch'd with master from a few minutes ago, there are is an additional step to add: - First you need to patch "mongoose" with a small patch from here :https://github.com/llange/Open-Vehicle-Monitoring-System- 3/blob/build-master/vehicle/OVMS.V3/support/mongoose-espv5.patch - And you will use a different "sdkconfig.defaults" - I'd suggest using for the moment the one from here :https://github.com/llange/Open-Vehicle-Monitoring-System- 3/blob/build- master/vehicle/OVMS.V3/support/sdkconfig.defaults.esp5.0.1 with "rm sdkconfig* ; cp /tmp/sdkconfig.defaults.esp5.0.1 sdkconfig.defaults" - The build command would be "idf.py build" Please note a different philosophy regarding the sdkconfig file: * In ESP-IDF 3.3.4 we use a "sdkconfig" file from the "support/" directory * while is ESP-IDF >=4 I suggest that we use a "sdkconfig.defaults" (which could live in support/ but could also be in vehicle/OVMS.V3 in source control) and copy it as "sdkconfig.defaults" The difference in approach is that with the "sdkconfig.defaults" it can live alongside you own "sdkconfig" and will only specify defaults values ; so that your changes to "sdkconfig" are not lost when we modify the "sdkconfig.defaults" file. You could have "sdkconfig.defaults" under source control and "sdkconfig" in gitignore living side by side. Cfhttps://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/kc...
You can also get inspiration from the automatic builds on my branch: Ex:https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions/runs/5017...
You'll find the workflow file describing the build instructions here :https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions/runs/5017...
And an example for each supported build on the left : If you click on a build, you'll find the steps that have been run, and if I ever forgot to write something here you'll find it in those logs. (Forget about the "cache" things, the various "updates" etc..)
While those automated builds use docker, in fact it doesn't change the way to build.
Let me know if you have issue with these builds, your separate ESP- IDF installs, or how to switch from one to the other.
Regards,
Le 18/05/2023 à 10:16, Chris van der Meijden a écrit :
Hi Ludovic
Thank you for all your efforts to get ESP-IDF to the newest version.
The reason why I have not been able to test along the way is, that I was not able to set up a native, non-docker, working test environment with the newer ESP-IDF versions.
Is there a "step by step" documentation for setting up these newer versions without docker? That would be really helpfull.
Regards
Chris
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
participants (3)
-
Chris van der Meijden -
Ludovic LANGE -
Mark Webb-Johnson