Hello List, Just a little status update on the conversion of wolfssh / wolfssl into submodules, and their subsequent version upgrade: * WolfSSH v4.7.0-stable has been converted from inline git to submodule in master o and has been upgraded to v1.4.10-stable in master just after that. Thanks !! Pending is the WolfSSL conversion itself: 1. Submodule conversion: https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/887 needs review and merge, BUT first needs the 2 following PRs: 1. https://github.com/openvehicles/wolfssl/pull/1 2. https://github.com/openvehicles/wolfssl/pull/2 2. WolfSSL upgrade to v5.3.0-stable : https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/890 (draft, needs review and merge and previous PRs) Then we would be able to upgrade WolfSSH to latest v1.4.13-stable with (but first we need upgrade of WolfSSL) : 3. https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/891 4. https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/893 Another pending with no link to the preceding: * https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/889 Thanks if some of you can take the time to review these in this order :-) Best regards, Ludovic PS: I still stop at WolfSSL v5.3.0-stable as I have crashes with SSH console for later versions. Will keep on testing to see if the recent changes on WolfSSL-master can fix that. Le 30/04/2023 à 21:39, Ludovic LANGE a écrit :
Thanks Mark, it's perfect.
I've just reapplied the original patches from Stephen here : https://github.com/openvehicles/wolfssl/pull/1
Once those are reviewed and merged, I'll upgrade the baseline PR for WolfSSL migration to submodule (and convert it from draft to final PR).
Thanks in advance !
Regards,
Le 30/04/2023 à 10:28, Mark Webb-Johnson a écrit :
I think this is done. Please try, and let me know if ok for you now.
Regards, Mark.
On 29 Apr 2023, at 4:44 AM, Ludovic LANGE <ll-ovmsdev@lange.nom.fr> wrote:
Hi Mark,
Thanks for the 2 repos.
Regarding the versions, I had success with wolfssl until 5.3.0. After that version, source code compatibility is still OK but I had crashes in SSH sessions (stack overflow).
So I did some baseline PRs with the same versions we had before (4.7.0/1.4.6) ; then other ones to increase up to 5.3.0 / 1.4.13:
* Baseline: o WolfSSH 1.4.6 : https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/885 o WolfSSL : (draft in-progress, it's missing 2 patches from our repo that are needed - at least one for compilation - with v4.7.0) * Upgrades: o WolfSSH up to 1.4.10 : https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/886 + I'll prepare an upgrade to the latest but some changes are needed that I need to test on all releases. o (For WolfSSL I'll prepare the upgrade later)
That way we can revert the "upgrade" commits without going back to the module / submodule commit + we do not introduce doubts/regressions with the submodule operation.
Please note: in the WolfSSL fork the tags are missing ; could you please fetch them and push them ? Thanks in advance.
Please also create - for WolfSSL still - a dedicated branch `v4.7.0-stable-ovms` so that I can apply the missing patches (mainly a workaround for a define SHA_CTX + some changes from Stephen (WOLFSSL_SMALL_STACK))
Thanks in advance.
Regards,
PS: If you have some time, and want to review my other pendings PRs: https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pulls?q=is%... I believe they can be reviewed / applied independently should you wish so.
Le 28/04/2023 à 07:37, Mark Webb-Johnson a écrit :
I’ve forked the two wolf repos:
* https://github.com/openvehicles/wolfssl * https://github.com/openvehicles/wolfssh
However, those are the latest versions (5.6.0 and 1.4.13 respectively) so I am not certain of compatibility with our code base, or how hard it would be to integrate those. We can always branch+tag earlier versions closer to ours if too hard. Our previous wolfSSH was v1.4.6 (February 3, 2021), and wolfSSL Release 4.7.0 (February 16, 2021). From my understanding, WolfSSH requires the crypt parts of WolfSSL to build.
Doing it this way, and bringing these in as submodules to our code, would be the best way to maintain compatibility with upstream. If it is really too difficult (given the number of changes Stephen made to get this to work), we can always just roll back to committing the existing versions directly to those GitHub repositories and making them submodules.
Could you have a try and see if that is possible?
Regards, Mark.
On 26 Apr 2023, at 2:26 PM, Ludovic LANGE <ll-ovmsdev@lange.nom.fr> wrote:
Hi Mark,
I'm going to add the missing parts for the main tunnel (new config items to a) auto-start + b) to register as a default route if wanted).
For the "manual" commands I'll experiment with these. I like the idea of having a quick remote access for troubleshooting purposes.
Regarding wolf*, yes, I do think that would be the best idea - to have these as submodules.
I intend to "relocate" those modules as subfolders of the component in my patches: components/wolfssh ├── CMakeLists.txt ├── README.md ├── component.mk └── wolfssh ├── ChangeLog.md ├── LICENSING ├── Makefile.am ├── README ├── .... ...
This way, we can manage our "glue" code ( CMakeLists.txt and component.mk ) without interfering with the external component. I believe that it keeps a proper separation between "our code" and "upstream code", making it easy to upstream patches and/or to rebase to a newer release. Especially given that some of these external components now have their own CMakeLists.txt - not easily re-usable for our build structure.
If you're OK with that, you can either:
* take this opportunity to add the submodule at the target place - only dealing with relocating component.mk (you can take inspiration from my tree : wolfssh <https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-build-workflow/vehicle/OVMS.V3/components/wolfssh> (component.mk andREADME.md) and wolfssl <https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-build-workflow/vehicle/OVMS.V3/components/wolfssl> (component.mk andREADME.md and port/ directory that you need to move - careful, in my tree the user_settings.h is already patched, you don't want that for the moment.)) * just replace as-is, and I'll do the relocation with a PR. * Or if you want I can also handle the (b) and (c) parts of your proposal myself with a PR
Let me know.
Regards,
Le 26/04/2023 à 04:10, Mark Webb-Johnson a écrit :
I reviewed your config, and see the issue. Quite a lot of parameters to set.
Perhaps it is sufficient to have just:
1. One single wireguard tunnel in configuration, with an associated auto setting to automatically start it. 2. A wireguiard command to bring up/down other tunnels, manually specified on the command line.
That way, if the user really needs more than one tunnel (or custom tunnels such as I suggest), he can use command shell scripts, or remote commands to bring them up/down.
Regarding wolfssh and other similar external components, I think that they can/should be split off and run as submodules sourced from a GitHub.com/openvehicles repository. We’ve already done that for mongoose, zlib, and libzip. Would you like me to create empty repositories for those, and then you can submit the PRs to (a) add the existing code to the new repository, (b) remove the code from Open-Vehicle-Monitoring-System-3, and (c) add the submodule to Open-Vehicle-Monitoring-System-3? I think candidates for this approach include:
1. wolfssh 2. wolfssl
Regards, Mark
> On 25 Apr 2023, at 3:28 PM, Ludovic LANGE > <ll-ovmsdev@lange.nom.fr> wrote: > > Hello Mark, > > Thanks for the comments. I'll see how we can manage tunnels from > the cli, should be doable. > > Regarding the PR, here is one PR with only WireGuard support : > https://github.com/llange/Open-Vehicle-Monitoring-System-3/pull/1 > - it's just for review, as it is from my fork to itself. > > For the WolfSSH / WolfSSL, I have some pending PRs on master, > and I create new ones as soon as the previous ones are merged, > as to not increase too much the burden on the reviewers. > They are here : > https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pulls?q=is%... > > (wolfssl will be a new one, not created yet, I'm waiting for the > feedback on the wolfssh one - is a new submodule acceptable or > not, or is it better to have a subtree, or a copy, ....) > > Regards, > > Le 25/04/2023 à 02:49, Mark Webb-Johnson a écrit : >> Really glad to see this, and thanks for working on it. >> >> I do think it would be useful to have many wireguard circuits >> configurable. >> >> For my own use case, I would like to be able to bring up a >> wireguard circuit purely from the command line (with no >> configuration set). This is because I am frequently called in >> to help with setup/configuration/diagnostic issues remotely, >> and having a full VPN would be extremely useful for that. If I >> could just send a single command to start the vpn back to me, >> then ssh into the module (or get can bus data over tcp/ip, >> etc), it would help tremendously. >> >> Regarding the PR, can we split this into (a) for >> wolfssh/wolfssl as a module, and (b) for wireguard support. At >> the moment, it is quite hard to review with both in the same PR. >> >> Regards, Mark. >> >>> On 24 Apr 2023, at 5:35 PM, Ludovic LANGE >>> <ll-ovmsdev@lange.nom.fr> wrote: >>> >>> Dear list, >>> >>> A few months ago I created >>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752 >>> to explore WireGuard VPN support ; which leaded me to add >>> ESP-IDFv5 support to OVMS. >>> >>> Now that this ESP-IDFv5 support is added (in my branch, and it >>> is in the progress of getting included in master - with the >>> help and the testing of everybody here), I've resumed my >>> exploration of adding support for WireGuard VPN to OVMS. >>> >>> It's now ready for comments, you can now check: >>> >>> * a new branch here >>> https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/752-wireguar... >>> * a DRAFT PR here >>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/882 >>> >>> if you want to explore and test this VPN support for OVMS. >>> >>> >>> My own use case for this feature is : >>> >>> * Security : I would like my module to be unreachable from >>> the public Internet. This is a first step. >>> * Practicality : I can reach my module with a single IP >>> address / name that is part of my private network. SSH, >>> Web, SCP, ... all work as if my module is local to my servers >>> * Roaming : The idea is to have a single point of contact >>> even if the module changes network, changes IP address, etc... >>> >>> Part of this feature set is already available with a >>> combination of the OVMS Server (v2, v3) and the Hologram.io >>> services, but I wanted to be independent of the mobile >>> connexion provider, and also file transfer is important for my >>> use case (SCP or other), as I'm often wanting to sync the >>> content of the SD card over the network. >>> >>> >>> If you can have a look and give feedback (either here, or on >>> the PR), especially on: >>> >>> * The documentation : is it enough ? properly organized ? >>> should it be split ? etc... >>> * The command set >>> * The configuration items : what's missing ? is the naming OK ? >>> * Other features (should I introduced events ? metrics ?) >>> >>> Also if you have any feature request, please share. >>> >>> Limitations: >>> >>> * Currently limited to 1 tunnel, but should work with >>> multiple - it's just a question of arranging the >>> configuration to support multiple instances >>> * Roaming not tested yet (will report) >>> * Compatibility with mobile network not tested yet (will >>> need help on this) >>> * I'm not really happy with the way I set the configuration >>> items. I'd like to "hide" (write-only) the important bits >>> (private key, shared key), but fear that it would clutter >>> the config namespace - especially if I introduce multiple >>> tunnels. >>> Maybe one solution would be to have a rich configuration >>> per tunnel (like a JSON / YAML), which would be a >>> nightmare to edit by hand and would need support in the >>> web interface. >>> * Tunnel always active as soon as the configuration is >>> correct. May be will need to add an enabled/disabled flag >>> to the configuration, and/or an auto-start flag. >>> >>> Current status: >>> >>> * Builds on GitHub actions (if you can to test, pre-compiled >>> firmwares are available here for example: >>> https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions/runs/4784... >>> - just download a Zip file (v5.0 or v5.0.1), and flash >>> with a command-line like esptool.py --chip esp32 --port >>> /dev/xxxx --baud 921600 write_flash --compress >>> --flash_mode "dio" --flash_freq "40m" --flash_size detect >>> 0x10000 ovms3.bin ) >>> * Works on My Machine (tunnel is UP, SSH is working OK, HTTP >>> is working OK, performances look OK. Ping time (ICMP) is >>> comparable with or without tunnel) >>> >>> >>> Thanks for your comments. >>> >>> Regards, >>> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.openvehicles.com >>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >> >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev