Instructions for building the "new idf" version?
[wow, I guess I suck at mailing lists!] Apologies for asking the list but I'm not having any luck searching the archives by eye, a month at a time (and google doesn't appear to crawl lists.openvehicles.com). Where are the instructions for building the version of ovms that uses a newer idf? What version of the toolchain will I need? Thanks! Craig
Hi Craig, I've received your 3 mails so it seems that everything went fine - however there may be a little delay between posting and receiving ? Regarding your "new idf" endeavour it's a good news that you're trying to get your hands dirty on it ! I'll try to help you as much as I can to get you started, however, bear with me, it's been a long time since I did it from scratch so I may omit some (crucial) steps here and there. First thing - I only really compiled with ESP-IDF 5+ : 5.0.1 / 5.0.2 / 5.0.6 / 5.1.4 / 5.2.2 / 5.3.0 (and also the corresponding "live" branches release-v5.0 / release-v5.1 / release-v5.2 / release-v5.3 , but as these are "moving targets", you can sometime have a working compile and an issue a few months later). So first I'd set up a (reliable) system to isolate you from any environment variables / default paths on your environment, and have a way to target exactly the ESP-IDF release you want. You may choose anything from docker, to a combination of environment variables (AFAIC I'm using an environment variable trick mainly around PATH, IDF_TOOLS_PATH, IDF_PATH and using $IDF_PATH/export.sh) Once you're confident with your setup (i.e. able to compile ESP-IDF's hello world https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linu... ) then I'd recommend you to checkout OVMS's master in a dedicated tree, and there is a little bit of patching this master before everything compiles cleanly: * You'll need to patch our version of Mongoose 6.11 to remove a few compile error. You'll find the patch here https://github.com/llange/Open-Vehicle-Monitoring-System-3/blob/build-master... o Or may be you'll need to upgrade to at least Mongoose 7.10 * You'll need to have this file idf_component.yml <https://github.com/llange/Open-Vehicle-Monitoring-System-3/blob/build-master/vehicle/OVMS.V3/support/idf_component.yml.esp5> in your master directory * You'll need to use this sdkconfig <https://github.com/llange/Open-Vehicle-Monitoring-System-3/blob/build-master/vehicle/OVMS.V3/support/sdkconfig.defaults.esp5> file in your master directory * You'll also have some fun with wolfssh / wolfssl : you'll need wolfssl version 5.3.0 so will have to switch your submodule to `v5.3.0-stable` of our tree https://github.com/openvehicles/wolfssl.git You can have a look at the GitHub Action build recipe here https://github.com/llange/Open-Vehicle-Monitoring-System-3/blob/build-master... that should be able to build a few different builds based on different ESP-IDF versions. Please note that there are quite a number of pending issues: * To my knowledge I had to disable a lot of module to make it work. It's possible to re-enable these one after the other but not sure I ported them all to the new build system * I had some issues with the SSH connection * Our crash handling mechanism hadn't been ported to the new system at the time I worked on it * Our ESP-IDF fork added a field `uxMutexesHeld` to `struct xTASK_STATUS` : https://github.com/openvehicles/esp-idf/commit/95e43fc2c4360f8126a0985bf037a... , which is then used in `module_tasks()` in `vehicle/OVMS.V3/main/ovms_module.cpp`. Not available in the ESP-IDF 5+ unless we patch it * A few things have changed (since ESP-IDF v3), been ported to new subsystem so a few bugs are still hidden here and there. I know it's not a full recipe, and I'll try to update this branch in the following days (but no guarantee that I'll have to time to do so). I'll also try to answer as quick as I can on the list :-) Let me know your progress and see how I can help. Best regards, Le 15/09/2024 à 20:05, Craig Leres via OvmsDev a écrit :
[wow, I guess I suck at mailing lists!]
Apologies for asking the list but I'm not having any luck searching the archives by eye, a month at a time (and google doesn't appear to crawl lists.openvehicles.com).
Where are the instructions for building the version of ovms that uses a newer idf? What version of the toolchain will I need?
Thanks!
Craig _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
On 9/15/24 13:23, Ludovic LANGE via OvmsDev wrote:
I've received your 3 mails so it seems that everything went fine - however there may be a little delay between posting and receiving ?
Yeah, sorry about the noise. I replied to and existing thread and edited the fields but thunderbird didn't show that it was including a "in-reply-to" header. Once that thread had my news subject, the mailing list software will always treat the message as part of that thread...
Regarding your "new idf" endeavour it's a good news that you're trying to get your hands dirty on it ! I'll try to help you as much as I can to get you started, however, bear with me, it's been a long time since I did it from scratch so I may omit some (crucial) steps here and there.
I did a tiny amount of work in this direction a few years ago. My vehicles are so far pretty simple and I'm getting pressure to update the xtensa toolchain for FreeBSD so this seems like a good time to jump in.
First thing - I only really compiled with ESP-IDF 5+ : 5.0.1 / 5.0.2 / 5.0.6 / 5.1.4 / 5.2.2 / 5.3.0 (and also the corresponding "live" branches release-v5.0 / release-v5.1 / release-v5.2 / release-v5.3 , but as these are "moving targets", you can sometime have a working compile and an issue a few months later).
I want to update the FreeBSD port with a specific release; sounds like I should use release-v5.3? (I'll probably switch to off-list email with you once I get further along.) Thanks! Craig
I would like to be included in the loop here if you are doing some off-list discussions and I'll follow along at home. I've already been building with the updated framework and have contributed a few commits to the main branch where I can... Though it's been a bit since I last built - I've had some other things I've been doing in OVMS. I would like to add to the discussion whether we try to move to a MQTT framework that supports topic aliases. (Since that's going to be how we make the server v3 consume less data). WolfMQTT does - but would require someone to get their head around it! //.ichael On Mon, 16 Sept 2024 at 06:37, Craig Leres via OvmsDev < ovmsdev@lists.openvehicles.com> wrote:
On 9/15/24 13:23, Ludovic LANGE via OvmsDev wrote:
I've received your 3 mails so it seems that everything went fine - however there may be a little delay between posting and receiving ?
Yeah, sorry about the noise. I replied to and existing thread and edited the fields but thunderbird didn't show that it was including a "in-reply-to" header. Once that thread had my news subject, the mailing list software will always treat the message as part of that thread...
Regarding your "new idf" endeavour it's a good news that you're trying to get your hands dirty on it ! I'll try to help you as much as I can to get you started, however, bear with me, it's been a long time since I did it from scratch so I may omit some (crucial) steps here and there.
I did a tiny amount of work in this direction a few years ago. My vehicles are so far pretty simple and I'm getting pressure to update the xtensa toolchain for FreeBSD so this seems like a good time to jump in.
First thing - I only really compiled with ESP-IDF 5+ : 5.0.1 / 5.0.2 / 5.0.6 / 5.1.4 / 5.2.2 / 5.3.0 (and also the corresponding "live" branches release-v5.0 / release-v5.1 / release-v5.2 / release-v5.3 , but as these are "moving targets", you can sometime have a working compile and an issue a few months later).
I want to update the FreeBSD port with a specific release; sounds like I should use release-v5.3?
(I'll probably switch to off-list email with you once I get further along.)
Thanks!
Craig _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
On 9/15/24 16:06, Michael Geddes via OvmsDev wrote:
I would like to be included in the loop here if you are doing some off- list discussions and I'll follow along at home. I've already been building with the updated framework and have contributed a few commits to the main branch where I can... Though it's been a bit since I last built - I've had some other things I've been doing in OVMS.
I would like to add to the discussion whether we try to move to a MQTT framework that supports topic aliases. (Since that's going to be how we make the server v3 consume less data). WolfMQTT does - but would require someone to get their head around it!
Well, maybe it'd be better to keep this on the list then. What version of the toolchain are you building with? Craig
On 9/15/24 16:08, Craig Leres via OvmsDev wrote:
What version of the toolchain are you building with?
The most recent appears to be crosstool-ng-1.26.0 (Sep 2023), is that what you folks are using? Craig https://github.com/crosstool-ng/crosstool-ng/releases/tag/crosstool-ng-1.26....
On 9/19/24 14:36, Craig Leres via OvmsDev wrote:
On 9/15/24 16:08, Craig Leres via OvmsDev wrote:
What version of the toolchain are you building with?
The most recent appears to be crosstool-ng-1.26.0 (Sep 2023), is that what you folks are using?
Poking around I see my ubuntu 22.04 ssd, that I use to compile esp32 arduino sketches, is using gcc 8.4.0 (crosstool-NG esp-2021r2-patch5). (This version is bundled somehow with arduino 1.8.19.) What I'm expecting is that we would pick one specific, recent version of the toolchain to (eventually) be supported for building ovms. For references purposes the "latest" crosstool-ng is 1.26.0 (gcc 13.2). Craig
I'm using esp-idf from https://github.com/espressif/esp-idf.git The previous was some months ago on release/v5.2 a36d8bc742 which ran on an ovms unit not connected to a car. I've just done a compile off v5.3 .. with a series of patches that I've got from Ludovic and some extras from myself. I've also used the latest mongoose which is annoyingly problematic as they re-did the whole way it was put together making it impossible just to apply patches. I've re-implemented some of the good things they used to have wrt defines and there are a couple of things that ovms team added that would be required. One of those is that the 'certificate' is meant to be a filename and there was a patch to the ovms version that allowed that filename string to be the certificate itself. //.ichael On Fri, 20 Sept 2024 at 10:29, Craig Leres via OvmsDev < ovmsdev@lists.openvehicles.com> wrote:
On 9/19/24 14:36, Craig Leres via OvmsDev wrote:
On 9/15/24 16:08, Craig Leres via OvmsDev wrote:
What version of the toolchain are you building with?
The most recent appears to be crosstool-ng-1.26.0 (Sep 2023), is that what you folks are using?
Poking around I see my ubuntu 22.04 ssd, that I use to compile esp32 arduino sketches, is using gcc 8.4.0 (crosstool-NG esp-2021r2-patch5). (This version is bundled somehow with arduino 1.8.19.)
What I'm expecting is that we would pick one specific, recent version of the toolchain to (eventually) be supported for building ovms.
For references purposes the "latest" crosstool-ng is 1.26.0 (gcc 13.2).
Craig _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
On 9/21/24 02:26, Michael Geddes via OvmsDev wrote:
I'm using esp-idf from https://github.com/espressif/esp-idf.git <https://github.com/espressif/esp-idf.git> The previous was some months ago on release/v5.2 a36d8bc742 which ran on an ovms unit not connected to a car.
I've just done a compile off v5.3 .. with a series of patches that I've got from Ludovic and some extras from myself.
I've also used the latest mongoose which is annoyingly problematic as they re-did the whole way it was put together making it impossible just to apply patches. I've re-implemented some of the good things they used to have wrt defines and there are a couple of things that ovms team added that would be required. One of those is that the 'certificate' is meant to be a filename and there was a patch to the ovms version that allowed that filename string to be the certificate itself.
The work on supporting the latest version of esp-idf sounds great. However I'm working on a earlier step -- the toolchain. crosstool-ng-1.22.0-97-gc752ad5 is based around gcc 5.2.0, I'd like to move to esp-13.2.0_20240530 (gcc 13.2.0) or esp-14.2.0_20240906 (gcc 14.2.0). What does "xtensa-esp32-elf-cc --version" say for you? Craig [On my FreeBSD system] ice 10 % xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG crosstool-ng-1.22.0-97-gc752ad5) 5.2.0 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [On my ubuntu test box] sea 15 % bin/xtensa-esp32-elf-cc --version xtensa-esp-elf-cc (crosstool-NG esp-13.2.0_20240530) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Hi Craig, I understand that you're trying to setup the toolchain, but I'm wondering it you're doing it the way ESP-IDF "wants" you to ( https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linu... ). There is a relationship between ESP-IDF's version, and the toolchain. And during the install of a specific ESP-IDF version it will (locally) install the proper toolchain. (Note: I don't think they deliver a FreeBSD version... and I don't have an Ubuntu box to test. But at least on my macos setup it works like this - and the env variable IDF_TOOLS_PATH points to the toolchain). You can find the proper versions here : https://github.com/espressif/esp-idf/blob/master/tools/tools.json (change `master` with the ESP-IDF version you need) - Only Linux, MacOS, Windows. Maybe you could start with a proper Linux install with the `|install.sh`| command, then replicate the version on FreeBSD. Oh, and I encourage you to isolate your multiple builds / versions so that you do not mixup crosstool version and non-corresponding ESP-IDF version. Now to strictly answer your question :-) , here are the version I'm using: $ switch-idf 3.3 $ xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG crosstool-ng-1.22.0-97-gc752ad5) 5.2.0 Copyright (C) 2015 Free Software Foundation, Inc. $ switch-idf 5.0 $ xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG esp-2022r1) 11.2.0 Copyright (C) 2021 Free Software Foundation, Inc. $ switch-idf 5.0.1 $ xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG esp-2022r1) 11.2.0 Copyright (C) 2021 Free Software Foundation, Inc. $ switch-idf 5.0.2 $ xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG esp-2022r1) 11.2.0 Copyright (C) 2021 Free Software Foundation, Inc. $ switch-idf 5.0.6 $ xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG esp-2022r1) 11.2.0 Copyright (C) 2021 Free Software Foundation, Inc. $ switch-idf 5.1.4 $ xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG esp-12.2.0_20230208) 12.2.0 Copyright (C) 2022 Free Software Foundation, Inc. $ switch-idf 5.2.2 $ xtensa-esp32-elf-cc --version xtensa-esp-elf-cc (crosstool-NG esp-13.2.0_20230928) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. $ switch-idf 5.3.0 $ xtensa-esp32-elf-cc --version xtensa-esp-elf-cc (crosstool-NG esp-13.2.0_20240530) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. Let me know if it helps. Regards, Le 21/09/2024 à 21:34, Craig Leres via OvmsDev a écrit :
On 9/21/24 02:26, Michael Geddes via OvmsDev wrote:
I'm using esp-idf from https://github.com/espressif/esp-idf.git <https://github.com/espressif/esp-idf.git> The previous was some months ago on release/v5.2 a36d8bc742 which ran on an ovms unit not connected to a car.
I've just done a compile off v5.3 .. with a series of patches that I've got from Ludovic and some extras from myself.
I've also used the latest mongoose which is annoyingly problematic as they re-did the whole way it was put together making it impossible just to apply patches. I've re-implemented some of the good things they used to have wrt defines and there are a couple of things that ovms team added that would be required. One of those is that the 'certificate' is meant to be a filename and there was a patch to the ovms version that allowed that filename string to be the certificate itself.
The work on supporting the latest version of esp-idf sounds great. However I'm working on a earlier step -- the toolchain. crosstool-ng-1.22.0-97-gc752ad5 is based around gcc 5.2.0, I'd like to move to esp-13.2.0_20240530 (gcc 13.2.0) or esp-14.2.0_20240906 (gcc 14.2.0).
What does "xtensa-esp32-elf-cc --version" say for you?
Craig
[On my FreeBSD system] ice 10 % xtensa-esp32-elf-cc --version xtensa-esp32-elf-cc (crosstool-NG crosstool-ng-1.22.0-97-gc752ad5) 5.2.0 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[On my ubuntu test box] sea 15 % bin/xtensa-esp32-elf-cc --version xtensa-esp-elf-cc (crosstool-NG esp-13.2.0_20240530) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
On 9/21/24 12:52, Ludovic LANGE via OvmsDev wrote:
I understand that you're trying to setup the toolchain, but I'm wondering it you're doing it the way ESP-IDF "wants" you to ( https:// docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linux- macos-setup.html#get-started-set-up-tools ).
There is a relationship between ESP-IDF's version, and the toolchain. And during the install of a specific ESP-IDF version it will (locally) install the proper toolchain. (Note: I don't think they deliver a FreeBSD version... and I don't have an Ubuntu box to test. But at least on my macos setup it works like this - and the env variable IDF_TOOLS_PATH points to the toolchain).
You can find the proper versions here : https://github.com/espressif/ esp-idf/blob/master/tools/tools.json (change `master` with the ESP-IDF version you need) - Only Linux, MacOS, Windows.
Maybe you could start with a proper Linux install with the `| install.sh`| command, then replicate the version on FreeBSD.
[...]
$ switch-idf 5.2.2 $ xtensa-esp32-elf-cc --version xtensa-esp-elf-cc (crosstool-NG esp-13.2.0_20230928) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc.
$ switch-idf 5.3.0 $ xtensa-esp32-elf-cc --version xtensa-esp-elf-cc (crosstool-NG esp-13.2.0_20240530) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc.
Let me know if it helps.
No, espressif does not provide FreeBSD versions of their toolchains. And yes, that is helpful; looking at tools/tools.json for 5.3.1 I see it's still using 13.2.0_20240530 so I focus on that version. Here's some info about what I'm upgrading from: https://www.freshports.org/devel/xtensa-esp32-elf/ It is a lot of work to make this guy build within the FreeBSD ports system, I have to identify and pre-download all of the source blobs prior to the build because ports aren't allowed to use the network during the build... (And it doesn't help that bootstrapping a cross compiler takes a long time.) Craig
participants (3)
-
Craig Leres -
Ludovic LANGE -
Michael Geddes