[Ovmsdev] wolfssX submodule transform (Was: Request For Comments - Wireguard VPN)
Ludovic LANGE
ll-ovmsdev at lange.nom.fr
Fri May 5 00:55:40 HKT 2023
Hello List,
Just a little status update on the conversion of wolfssh / wolfssl into
submodules, and their subsequent version upgrade:
* WolfSSH v4.7.0-stable has been converted from inline git to
submodule in master
o and has been upgraded to v1.4.10-stable in master just after that.
Thanks !!
Pending is the WolfSSL conversion itself:
1. Submodule conversion:
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/887
needs review and merge, BUT first needs the 2 following PRs:
1. https://github.com/openvehicles/wolfssl/pull/1
2. https://github.com/openvehicles/wolfssl/pull/2
2. WolfSSL upgrade to v5.3.0-stable :
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/890
(draft, needs review and merge and previous PRs)
Then we would be able to upgrade WolfSSH to latest v1.4.13-stable with
(but first we need upgrade of WolfSSL) :
3. https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/891
4. https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/893
Another pending with no link to the preceding:
* https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/889
Thanks if some of you can take the time to review these in this order :-)
Best regards,
Ludovic
PS: I still stop at WolfSSL v5.3.0-stable as I have crashes with SSH
console for later versions. Will keep on testing to see if the recent
changes on WolfSSL-master can fix that.
Le 30/04/2023 à 21:39, Ludovic LANGE a écrit :
> Thanks Mark, it's perfect.
>
> I've just reapplied the original patches from Stephen here :
> https://github.com/openvehicles/wolfssl/pull/1
>
> Once those are reviewed and merged, I'll upgrade the baseline PR for
> WolfSSL migration to submodule (and convert it from draft to final PR).
>
> Thanks in advance !
>
> Regards,
>
>
> Le 30/04/2023 à 10:28, Mark Webb-Johnson a écrit :
>> I think this is done. Please try, and let me know if ok for you now.
>>
>> Regards, Mark.
>>
>>> On 29 Apr 2023, at 4:44 AM, Ludovic LANGE <ll-ovmsdev at lange.nom.fr>
>>> wrote:
>>>
>>> Hi Mark,
>>>
>>> Thanks for the 2 repos.
>>>
>>> Regarding the versions, I had success with wolfssl until 5.3.0.
>>> After that version, source code compatibility is still OK but I had
>>> crashes in SSH sessions (stack overflow).
>>>
>>> So I did some baseline PRs with the same versions we had before
>>> (4.7.0/1.4.6) ; then other ones to increase up to 5.3.0 / 1.4.13:
>>>
>>> * Baseline:
>>> o WolfSSH 1.4.6 :
>>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/885
>>> o WolfSSL : (draft in-progress, it's missing 2 patches from
>>> our repo that are needed - at least one for compilation -
>>> with v4.7.0)
>>> * Upgrades:
>>> o WolfSSH up to 1.4.10 :
>>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/886
>>> + I'll prepare an upgrade to the latest but some changes
>>> are needed that I need to test on all releases.
>>> o (For WolfSSL I'll prepare the upgrade later)
>>>
>>> That way we can revert the "upgrade" commits without going back to
>>> the module / submodule commit + we do not introduce
>>> doubts/regressions with the submodule operation.
>>>
>>>
>>> Please note: in the WolfSSL fork the tags are missing ; could you
>>> please fetch them and push them ? Thanks in advance.
>>>
>>> Please also create - for WolfSSL still - a dedicated branch
>>> `v4.7.0-stable-ovms` so that I can apply the missing patches (mainly
>>> a workaround for a define SHA_CTX + some changes from Stephen
>>> (WOLFSSL_SMALL_STACK))
>>>
>>>
>>> Thanks in advance.
>>>
>>>
>>> Regards,
>>>
>>>
>>> PS: If you have some time, and want to review my other pendings PRs:
>>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pulls?q=is%3Aopen+is%3Apr+author%3Allange+draft%3Afalse
>>> I believe they can be reviewed / applied independently should you
>>> wish so.
>>>
>>> Le 28/04/2023 à 07:37, Mark Webb-Johnson a écrit :
>>>>
>>>> I’ve forked the two wolf repos:
>>>>
>>>> * https://github.com/openvehicles/wolfssl
>>>> * https://github.com/openvehicles/wolfssh
>>>>
>>>>
>>>> However, those are the latest versions (5.6.0 and 1.4.13
>>>> respectively) so I am not certain of compatibility with our code
>>>> base, or how hard it would be to integrate those. We can always
>>>> branch+tag earlier versions closer to ours if too hard. Our
>>>> previous wolfSSH was v1.4.6 (February 3, 2021), and wolfSSL Release
>>>> 4.7.0 (February 16, 2021). From my understanding, WolfSSH requires
>>>> the crypt parts of WolfSSL to build.
>>>>
>>>> Doing it this way, and bringing these in as submodules to our code,
>>>> would be the best way to maintain compatibility with upstream. If
>>>> it is really too difficult (given the number of changes Stephen
>>>> made to get this to work), we can always just roll back to
>>>> committing the existing versions directly to those GitHub
>>>> repositories and making them submodules.
>>>>
>>>> Could you have a try and see if that is possible?
>>>>
>>>> Regards, Mark.
>>>>
>>>>> On 26 Apr 2023, at 2:26 PM, Ludovic LANGE
>>>>> <ll-ovmsdev at lange.nom.fr> wrote:
>>>>>
>>>>> Hi Mark,
>>>>>
>>>>> I'm going to add the missing parts for the main tunnel (new config
>>>>> items to a) auto-start + b) to register as a default route if wanted).
>>>>>
>>>>> For the "manual" commands I'll experiment with these. I like the
>>>>> idea of having a quick remote access for troubleshooting purposes.
>>>>>
>>>>>
>>>>> Regarding wolf*, yes, I do think that would be the best idea - to
>>>>> have these as submodules.
>>>>>
>>>>> I intend to "relocate" those modules as subfolders of the
>>>>> component in my patches:
>>>>> components/wolfssh
>>>>> ├── CMakeLists.txt
>>>>> ├── README.md
>>>>> ├── component.mk
>>>>> └── wolfssh
>>>>> ├── ChangeLog.md
>>>>> ├── LICENSING
>>>>> ├── Makefile.am
>>>>> ├── README
>>>>> ├── ....
>>>>> ...
>>>>>
>>>>> This way, we can manage our "glue" code ( CMakeLists.txt and
>>>>> component.mk ) without interfering with the external component. I
>>>>> believe that it keeps a proper separation between "our code" and
>>>>> "upstream code", making it easy to upstream patches and/or to
>>>>> rebase to a newer release.
>>>>> Especially given that some of these external components now have
>>>>> their own CMakeLists.txt - not easily re-usable for our build
>>>>> structure.
>>>>>
>>>>> If you're OK with that, you can either:
>>>>>
>>>>> * take this opportunity to add the submodule at the target place
>>>>> - only dealing with relocating component.mk (you can take
>>>>> inspiration from my tree : wolfssh
>>>>> <https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-build-workflow/vehicle/OVMS.V3/components/wolfssh>
>>>>> (component.mk andREADME.md) and wolfssl
>>>>> <https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-build-workflow/vehicle/OVMS.V3/components/wolfssl>
>>>>> (component.mk andREADME.md and port/ directory that you need
>>>>> to move - careful, in my tree the user_settings.h is already
>>>>> patched, you don't want that for the moment.))
>>>>> * just replace as-is, and I'll do the relocation with a PR.
>>>>> * Or if you want I can also handle the (b) and (c) parts of your
>>>>> proposal myself with a PR
>>>>>
>>>>> Let me know.
>>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>> Le 26/04/2023 à 04:10, Mark Webb-Johnson a écrit :
>>>>>> I reviewed your config, and see the issue. Quite a lot of
>>>>>> parameters to set.
>>>>>>
>>>>>> Perhaps it is sufficient to have just:
>>>>>>
>>>>>> 1. One single wireguard tunnel in configuration, with an
>>>>>> associated auto setting to automatically start it.
>>>>>> 2. A wireguiard command to bring up/down other tunnels, manually
>>>>>> specified on the command line.
>>>>>>
>>>>>>
>>>>>> That way, if the user really needs more than one tunnel (or
>>>>>> custom tunnels such as I suggest), he can use command shell
>>>>>> scripts, or remote commands to bring them up/down.
>>>>>>
>>>>>> Regarding wolfssh and other similar external components, I think
>>>>>> that they can/should be split off and run as submodules sourced
>>>>>> from a GitHub.com/openvehicles repository. We’ve already done
>>>>>> that for mongoose, zlib, and libzip. Would you like me to create
>>>>>> empty repositories for those, and then you can submit the PRs to
>>>>>> (a) add the existing code to the new repository, (b) remove the
>>>>>> code from Open-Vehicle-Monitoring-System-3, and (c) add the
>>>>>> submodule to Open-Vehicle-Monitoring-System-3? I think candidates
>>>>>> for this approach include:
>>>>>>
>>>>>> 1. wolfssh
>>>>>> 2. wolfssl
>>>>>>
>>>>>>
>>>>>> Regards, Mark
>>>>>>
>>>>>>> On 25 Apr 2023, at 3:28 PM, Ludovic LANGE
>>>>>>> <ll-ovmsdev at lange.nom.fr> wrote:
>>>>>>>
>>>>>>> Hello Mark,
>>>>>>>
>>>>>>> Thanks for the comments. I'll see how we can manage tunnels from
>>>>>>> the cli, should be doable.
>>>>>>>
>>>>>>> Regarding the PR, here is one PR with only WireGuard support :
>>>>>>> https://github.com/llange/Open-Vehicle-Monitoring-System-3/pull/1
>>>>>>> - it's just for review, as it is from my fork to itself.
>>>>>>>
>>>>>>> For the WolfSSH / WolfSSL, I have some pending PRs on master,
>>>>>>> and I create new ones as soon as the previous ones are merged,
>>>>>>> as to not increase too much the burden on the reviewers.
>>>>>>> They are here :
>>>>>>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pulls?q=is%3Aopen+is%3Apr+author%3Allange+draft%3Afalse
>>>>>>>
>>>>>>> (wolfssl will be a new one, not created yet, I'm waiting for the
>>>>>>> feedback on the wolfssh one - is a new submodule acceptable or
>>>>>>> not, or is it better to have a subtree, or a copy, ....)
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Le 25/04/2023 à 02:49, Mark Webb-Johnson a écrit :
>>>>>>>> Really glad to see this, and thanks for working on it.
>>>>>>>>
>>>>>>>> I do think it would be useful to have many wireguard circuits
>>>>>>>> configurable.
>>>>>>>>
>>>>>>>> For my own use case, I would like to be able to bring up a
>>>>>>>> wireguard circuit purely from the command line (with no
>>>>>>>> configuration set). This is because I am frequently called in
>>>>>>>> to help with setup/configuration/diagnostic issues remotely,
>>>>>>>> and having a full VPN would be extremely useful for that. If I
>>>>>>>> could just send a single command to start the vpn back to me,
>>>>>>>> then ssh into the module (or get can bus data over tcp/ip,
>>>>>>>> etc), it would help tremendously.
>>>>>>>>
>>>>>>>> Regarding the PR, can we split this into (a) for
>>>>>>>> wolfssh/wolfssl as a module, and (b) for wireguard support. At
>>>>>>>> the moment, it is quite hard to review with both in the same PR.
>>>>>>>>
>>>>>>>> Regards, Mark.
>>>>>>>>
>>>>>>>>> On 24 Apr 2023, at 5:35 PM, Ludovic LANGE
>>>>>>>>> <ll-ovmsdev at lange.nom.fr> wrote:
>>>>>>>>>
>>>>>>>>> Dear list,
>>>>>>>>>
>>>>>>>>> A few months ago I created
>>>>>>>>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752
>>>>>>>>> to explore WireGuard VPN support ; which leaded me to add
>>>>>>>>> ESP-IDFv5 support to OVMS.
>>>>>>>>>
>>>>>>>>> Now that this ESP-IDFv5 support is added (in my branch, and it
>>>>>>>>> is in the progress of getting included in master - with the
>>>>>>>>> help and the testing of everybody here), I've resumed my
>>>>>>>>> exploration of adding support for WireGuard VPN to OVMS.
>>>>>>>>>
>>>>>>>>> It's now ready for comments, you can now check:
>>>>>>>>>
>>>>>>>>> * a new branch here
>>>>>>>>> https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/752-wireguard
>>>>>>>>> * a DRAFT PR here
>>>>>>>>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/882
>>>>>>>>>
>>>>>>>>> if you want to explore and test this VPN support for OVMS.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> My own use case for this feature is :
>>>>>>>>>
>>>>>>>>> * Security : I would like my module to be unreachable from
>>>>>>>>> the public Internet. This is a first step.
>>>>>>>>> * Practicality : I can reach my module with a single IP
>>>>>>>>> address / name that is part of my private network. SSH,
>>>>>>>>> Web, SCP, ... all work as if my module is local to my servers
>>>>>>>>> * Roaming : The idea is to have a single point of contact
>>>>>>>>> even if the module changes network, changes IP address, etc...
>>>>>>>>>
>>>>>>>>> Part of this feature set is already available with a
>>>>>>>>> combination of the OVMS Server (v2, v3) and the Hologram.io
>>>>>>>>> services, but I wanted to be independent of the mobile
>>>>>>>>> connexion provider, and also file transfer is important for my
>>>>>>>>> use case (SCP or other), as I'm often wanting to sync the
>>>>>>>>> content of the SD card over the network.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> If you can have a look and give feedback (either here, or on
>>>>>>>>> the PR), especially on:
>>>>>>>>>
>>>>>>>>> * The documentation : is it enough ? properly organized ?
>>>>>>>>> should it be split ? etc...
>>>>>>>>> * The command set
>>>>>>>>> * The configuration items : what's missing ? is the naming OK ?
>>>>>>>>> * Other features (should I introduced events ? metrics ?)
>>>>>>>>>
>>>>>>>>> Also if you have any feature request, please share.
>>>>>>>>>
>>>>>>>>> Limitations:
>>>>>>>>>
>>>>>>>>> * Currently limited to 1 tunnel, but should work with
>>>>>>>>> multiple - it's just a question of arranging the
>>>>>>>>> configuration to support multiple instances
>>>>>>>>> * Roaming not tested yet (will report)
>>>>>>>>> * Compatibility with mobile network not tested yet (will
>>>>>>>>> need help on this)
>>>>>>>>> * I'm not really happy with the way I set the configuration
>>>>>>>>> items. I'd like to "hide" (write-only) the important bits
>>>>>>>>> (private key, shared key), but fear that it would clutter
>>>>>>>>> the config namespace - especially if I introduce multiple
>>>>>>>>> tunnels.
>>>>>>>>> Maybe one solution would be to have a rich configuration
>>>>>>>>> per tunnel (like a JSON / YAML), which would be a
>>>>>>>>> nightmare to edit by hand and would need support in the
>>>>>>>>> web interface.
>>>>>>>>> * Tunnel always active as soon as the configuration is
>>>>>>>>> correct. May be will need to add an enabled/disabled flag
>>>>>>>>> to the configuration, and/or an auto-start flag.
>>>>>>>>>
>>>>>>>>> Current status:
>>>>>>>>>
>>>>>>>>> * Builds on GitHub actions (if you can to test, pre-compiled
>>>>>>>>> firmwares are available here for example:
>>>>>>>>> https://github.com/llange/Open-Vehicle-Monitoring-System-3/actions/runs/4784405668
>>>>>>>>> - just download a Zip file (v5.0 or v5.0.1), and flash
>>>>>>>>> with a command-line like esptool.py --chip esp32 --port
>>>>>>>>> /dev/xxxx --baud 921600 write_flash --compress
>>>>>>>>> --flash_mode "dio" --flash_freq "40m" --flash_size detect
>>>>>>>>> 0x10000 ovms3.bin )
>>>>>>>>> * Works on My Machine (tunnel is UP, SSH is working OK, HTTP
>>>>>>>>> is working OK, performances look OK. Ping time (ICMP) is
>>>>>>>>> comparable with or without tunnel)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for your comments.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> OvmsDev mailing list
>>>>>>>>> OvmsDev at lists.openvehicles.com
>>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OvmsDev mailing list
>>>>>>>> OvmsDev at lists.openvehicles.com
>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OvmsDev mailing list
>>>>>>> OvmsDev at lists.openvehicles.com
>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OvmsDev mailing list
>>>>>> OvmsDev at lists.openvehicles.com
>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.openvehicles.com
>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>>
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.openvehicles.com
>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>
>>>
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.openvehicles.com
>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20230504/427c0eea/attachment-0001.htm>
More information about the OvmsDev
mailing list