[Ovmsdev] Urgent TLS root certificate issue (Let's Encrypt)
Mark Webb-Johnson
mark at webb-johnson.net
Wed Sep 29 15:56:12 HKT 2021
The cross-signing bit seems to be on the X1 cert provided in the certificate chain by the server. The one provided there is signed by X3. The problem comes because a different X1 certificate may be provided as a trusted CA (and that X1 is self-signed with a long expiration date - 2035). So the verification process has two possible paths to verify the cert, and the result depends on which one it chooses (and how it handles a trusted CA cert supplementing / replacing one provided by the server).
Overall, a mess. I’m fighting this same issue now with a bunch of systems in my day job, and likely a large part of the Internet is going to start to fail tomorrow. Particularly a lot of small embedded devices.
Regards, Mark.
> On 29 Sep 2021, at 2:51 PM, Michael Balzer <dexter at expeedo.de> wrote:
>
> Signed PGP part
> SNI was pretty much intact already, I took care of a remaining Mongoose SNI bug in commit 9e4f31249856a67317f99920e47efa34f6734c73 / mongoose 8dc9012b57b85e062974fbdec17db30a501bf68f (April 2020).
>
> I understand the WolfSSL chain management uses a supplied root cert over one delivered by the server:
> https://www.wolfssl.com/docs/wolfssl-manual/ch7/ <https://www.wolfssl.com/docs/wolfssl-manual/ch7/> → section 7.3
>
>> Our problem could be either the wolfssl not recognising the X1 cert, or not supporting the cross-signing arrangement (where there are two paths to verify - the expired X3 and the provided X1). I suspect the latter. In an ideal world, having both X1 and X3 trusted shouldn’t be a problem.
>
> Is cross signing supported at all by WolfSSL? The only place I've found it mentioned on wolfssl is this page: https://www.wolfssl.com/certificate-chain-chain-trust/ <https://www.wolfssl.com/certificate-chain-chain-trust/>
> …where they say "(We’ll cover cross-signing below)" … but I don't see them doing that.
>
> There is also no commit in their git history regarding cross-signed certificates or multiple signatures, so I don't think updating to 4.8.1 would help.
>
> The intermediate "R3" has issuer "C=US, O=Internet Security Research Group, CN=ISRG Root X1", which should match the new ISRG certificate, but not the DST certificate. Maybe that's identified by a fingerprint in the signature though (?), and maybe the DST signature comes first and WolfSSL just doesn't check further signatures if the first fails?
>
> I'll look into the code later on, but need to take care of my daily job duties first.
>
> If we can't fix this today, I think our best option is to switch back to mbedTLS for the final 3.2 release and try to fix this for the initial 3.3 release.
>
> Regards,
> Michael
>
>
> Am 29.09.21 um 03:40 schrieb Mark Webb-Johnson:
>>
>> It seems the dexter server returns different certificates depending on the presence of SNI extension in the TSL negotiation. That is pretty normal (where a single IP is serving multiple different websites), but I am not sure if our client library supports that?
>>
>> Without SNI, we get:
>>
>> Serial Number:
>> 04:ab:25:e5:50:75:49:7b:9f:1b:39:8f:ed:3e:53:44:b4:54
>> Subject: CN=ns34.expeedo.de <http://ns34.expeedo.de/>
>> X509v3 Subject Alternative Name:
>> DNS:ns34.expeedo.de <http://ns34.expeedo.de/>
>>
>> With SNI dexters-web.de <http://dexters-web.de/>, we get:
>>
>> Serial Number:
>> 03:be:45:05:45:aa:e3:da:cb:4c:38:ae:f6:90:2f:65:65:e0
>> Subject: CN=dexter.shopdriver.de <http://dexter.shopdriver.de/>
>> X509v3 Subject Alternative Name:
>> DNS:dexter.shopdriver.de <http://dexter.shopdriver.de/>, DNS:dexters-web.de <http://dexters-web.de/>, DNS:ovms.dexters-web.de <http://ovms.dexters-web.de/>, DNS:www.dexter.shopdriver.de <http://www.dexter.shopdriver.de/>, DNS:www.dexters-web.de <http://www.dexters-web.de/>
>>
>> Assuming the SNI is set correctly by our library, looking at the dexter.shopdriver.de <http://dexter.shopdriver.de/> chain, we have:
>>
>> Subject: CN=dexter.shopdriver.de <http://dexter.shopdriver.de/>
>> Issuer: C=US, O=Let's Encrypt, CN=R3
>>
>> Subject: C=US, O=Let's Encrypt, CN=R3
>> Issuer: C=US, O=Internet Security Research Group, CN=ISRG Root X1
>>
>> Subject: C=US, O=Internet Security Research Group, CN=ISRG Root X1
>> Issuer: O=Digital Signature Trust Co., CN=DST Root CA X3
>>
>> and the components/ovms_tls/trustedca/dst.crt we have is:
>>
>> Subject: O=Digital Signature Trust Co., CN=DST Root CA X3
>> Issuer: O=Digital Signature Trust Co., CN=DST Root CA X3
>> (Which expires Sep 30 14:01:15 2021 GMT and was issued just a year earlier - ridiculous for a root CA)
>>
>> The components/ovms_tls/trustedca/isrg_x1.crt is:
>>
>> Subject: C=US, O=Internet Security Research Group, CN=ISRG Root X1
>> Issuer: C=US, O=Internet Security Research Group, CN=ISRG Root X1
>> Validity
>> Not Before: Jun 4 11:04:38 2015 GMT
>> Not After : Jun 4 11:04:38 2035 GMT
>> sha256WithRSAEncryption 4096 bit
>>
>> Our problem could be either the wolfssl not recognising the X1 cert, or not supporting the cross-signing arrangement (where there are two paths to verify - the expired X3 and the provided X1). I suspect the latter. In an ideal world, having both X1 and X3 trusted shouldn’t be a problem.
>>
>> There is a later 4.8.1 version of wolfssl (we seem to use 4.7.0 released February 2021). Perhaps we can try to update to that?
>>
>> Mark
>>
>>> On 29 Sep 2021, at 4:11 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>
>>> Signed PGP part
>>> I can now confirm it's a WolfSSL issue :-(
>>>
>>> I've switched back to release 3.2.016, i.e. before changing to WolfSSL, and the ISRG Root X1 certificate works perfectly, just as it should.
>>>
>>> Steve, I remember you included a config option to enable using WolfSSL, but cannot find it now. Can you give me a pointer, or did you remove that option later on?
>>>
>>>
>>>
>>> Am 28.09.21 um 21:53 schrieb Stephen Casner:
>>>> I wonder if we are hitting a key size limit in the WolfSSL code. I
>>>> know there are configuration parameters in user_settings.h for
>>>> different SHA sizes, but I don't recall one for RSA keys.
>>>>
>>>> -- Steve
>>>>
>>>> On Tue, 28 Sep 2021, Michael Balzer wrote:
>>>>
>>>>> Still no luck, and getting clueless... :-/
>>>>>
>>>>> Let's Encrypt have a live test server:
>>>>> https://valid-isrgrootx1.letsencrypt.org/ <https://valid-isrgrootx1.letsencrypt.org/>
>>>>>
>>>>> Here's a simple test script to access that server:
>>>>>
>>>>> (function(){
>>>>> HTTP.request({
>>>>> url: "https://valid-isrgrootx1.letsencrypt.org/" <https://valid-isrgrootx1.letsencrypt.org/>,
>>>>> always: function() { JSON.print(this); }
>>>>> });
>>>>> })();
>>>>>
>>>>> Just copy this into the editor and evaluate.
>>>>>
>>>>> After commenting out the DST certificate from the TLS source, the http request
>>>>> terminates with an SSL error (-3) with installed...
>>>>>
>>>>> * self-signed ISRG certificate
>>>>> * cross-signed ISRG / DST root certificate
>>>>> * LE R3 certificate
>>>>> * any combination of these three.
>>>>>
>>>>> But the access works immediately after reinstalling the old DST root
>>>>> certificate. There is no DST reference in the chain of that server...
>>>>>
>>>>>
>>>>> I found another ESP32 project -- not using WolfSSL but the esp-idf SSL & MQTT
>>>>> lib --; apparently all they needed to do was to add the very same self-signed
>>>>> ISRG X1 certificate I added:
>>>>>
>>>>> *
>>>>> https://github.com/LibreSolar/esp32-edge-firmware/commit/d6b6307cbdb60feb118355fda973eba11d52f8f5 <https://github.com/LibreSolar/esp32-edge-firmware/commit/d6b6307cbdb60feb118355fda973eba11d52f8f5>
>>>>>
>>>>>
>>>>> So: why would WolfSSL not use the supplied ISRG certificate? Why would it use
>>>>> the DST cert without DST being present as an issuer?
>>>>>
>>>>> The ISRG certificate is the only one having...
>>>>>
>>>>> signed using : RSA with SHA-256
>>>>> RSA key size : 4096 bits
>>>>>
>>>>> But that would surely be supported by WolfSSL, wouldn't it?
>>>>>
>>>>> I'm running out of ideas, and this really isn't my primary area. Any help is
>>>>> appreciated.
>>>>>
>>>>> Regards,
>>>>> Michael
>>>>>
>>>>>
>>>>> Am 28.09.21 um 19:15 schrieb Stephen Casner:
>>>>>> Michael,
>>>>>>
>>>>>> Certainly within our code as it stands there is not any mechanism to
>>>>>> tell the WolfSSL code to adjust its certificate validation procedure.
>>>>>> The browser certificate substitution that you hypothesize is not clear
>>>>>> to me. I would expect the validation to simply follow the chain.
>>>>>>
>>>>>> -- Steve
>>>>>>
>>>>>> On Tue, 28 Sep 2021, Michael Balzer wrote:
>>>>>>
>>>>>>> More info:
>>>>>>>
>>>>>>> All my browsers already have a builtin ISRG X1 certificate signed by ISRG
>>>>>>> only, that's the new version:
>>>>>>>
>>>>>>> https://crt.sh/?id=9314791 <https://crt.sh/?id=9314791>
>>>>>>>
>>>>>>> My server still sends the ISRG X1 certificate cross signed / issued by DST
>>>>>>> Root CA X3. That's the chain it got from Let's Encrypt (via certbot) on
>>>>>>> the
>>>>>>> last renewal (last month!):
>>>>>>>
>>>>>>> https://crt.sh/?id=3958242236 <https://crt.sh/?id=3958242236>
>>>>>>>
>>>>>>> Without the DST root cert, WolfSSL then fails validating the DST signed X1
>>>>>>> root certificate (I assume):
>>>>>>> https://www.wolfssl.com/docs/wolfssl-manual/ch7/ <https://www.wolfssl.com/docs/wolfssl-manual/ch7/>
>>>>>>>
>>>>>>> My servers will continue sending that chain including the outdated root
>>>>>>> cert
>>>>>>> probably until the next renewal, so it's possible having added the new X1
>>>>>>> root
>>>>>>> certificate didn't solve the issue.
>>>>>>>
>>>>>>> The browsers seem to know how to substitute the DST signed certificate by
>>>>>>> the
>>>>>>> builtin self-signed (?). Is there a similar option in WolfSSL, and do we
>>>>>>> need
>>>>>>> to enable that?
>>>>>>>
>>>>>>> Steve, can you confirm this, do you know a solution?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Michael
>>>>>>>
>>>>>>>
>>>>>>> Am 28.09.21 um 15:34 schrieb Michael Balzer:
>>>>>>>> I've tried adding the intermediate cert ("R3") and then also my site
>>>>>>>> certificate, that didn't help.
>>>>>>>>
>>>>>>>> Only adding the DST cert again fixes the connection.
>>>>>>>>
>>>>>>>> Any ideas?
>>>>>>>>
>>>>>>>>
>>>>>>>> OVMS# tls trust list
>>>>>>>> ...
>>>>>>>> ISRG Root X1 length 1939 bytes
>>>>>>>> 1939 byte certificate: ISRG Root X1
>>>>>>>> cert. version : 3
>>>>>>>> serial number : 82:10:CF:B0:D2:40:E3:59:44:63:E0:BB:63:82:8B:00
>>>>>>>> issuer name : C=US, O=Internet Security Research Group, CN=ISRG
>>>>>>>> Root
>>>>>>>> X1
>>>>>>>> subject name : C=US, O=Internet Security Research Group, CN=ISRG
>>>>>>>> Root
>>>>>>>> X1
>>>>>>>> issued on : 2015-06-04 11:04:38
>>>>>>>> expires on : 2035-06-04 11:04:38
>>>>>>>> signed using : RSA with SHA-256
>>>>>>>> RSA key size : 4096 bits
>>>>>>>> basic constraints : CA=true
>>>>>>>> key usage : Key Cert Sign, CRL Sign
>>>>>>>> ...
>>>>>>>> dexter length 1972 bytes
>>>>>>>> 1972 byte certificate: dexter
>>>>>>>> cert. version : 3
>>>>>>>> serial number :
>>>>>>>> 04:55:1D:F4:27:A3:7D:E9:E4:A8:5C:37:F6:A1:61:87:3C:E5
>>>>>>>> issuer name : C=US, O=Let's Encrypt, CN=R3
>>>>>>>> subject name : CN=dexter.shopdriver.de <http://dexter.shopdriver.de/>
>>>>>>>> issued on : 2021-08-07 05:47:57
>>>>>>>> expires on : 2021-11-05 05:47:55
>>>>>>>> signed using : RSA with SHA-256
>>>>>>>> RSA key size : 2048 bits
>>>>>>>> basic constraints : CA=false
>>>>>>>> subject alt name : dexter.shopdriver.de <http://dexter.shopdriver.de/>, dexters-web.de <http://dexters-web.de/>,
>>>>>>>> ovms.dexters-web.de <http://ovms.dexters-web.de/>, www.dexter.shopdriver.de <http://www.dexter.shopdriver.de/>, www.dexters-web.de <http://www.dexters-web.de/>
>>>>>>>> key usage : Digital Signature, Key Encipherment
>>>>>>>> ext key usage : TLS Web Server Authentication, TLS Web Client
>>>>>>>> Authentication
>>>>>>>> ...
>>>>>>>> r3 length 1826 bytes
>>>>>>>> 1826 byte certificate: r3
>>>>>>>> cert. version : 3
>>>>>>>> serial number : 91:2B:08:4A:CF:0C:18:A7:53:F6:D6:2E:25:A7:5F:5A
>>>>>>>> issuer name : C=US, O=Internet Security Research Group, CN=ISRG
>>>>>>>> Root
>>>>>>>> X1
>>>>>>>> subject name : C=US, O=Let's Encrypt, CN=R3
>>>>>>>> issued on : 2020-09-04 00:00:00
>>>>>>>> expires on : 2025-09-15 16:00:00
>>>>>>>> signed using : RSA with SHA-256
>>>>>>>> RSA key size : 2048 bits
>>>>>>>> basic constraints : CA=true, max_pathlen=0
>>>>>>>> key usage : Digital Signature, Key Cert Sign, CRL Sign
>>>>>>>> ext key usage : TLS Web Client Authentication, TLS Web Server
>>>>>>>> Authentication
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Am 28.09.21 um 15:07 schrieb Michael Balzer:
>>>>>>>>> We would need to bypass / shortcut the "eap" test phase.
>>>>>>>>>
>>>>>>>>> But I agree, "master" is stable, I haven't had any issues or reports,
>>>>>>>>> so I
>>>>>>>>> think we could do that. The FreeRTOS timer issue I'm working on only
>>>>>>>>> affects very specific conditions, so not necessary to wait for that.
>>>>>>>>>
>>>>>>>>> Should we remove the expiring DST certificate in that release then?
>>>>>>>>>
>>>>>>>>> ...uh oh: just tried removing the DST certificate: the module cannot
>>>>>>>>> connect to my server anymore...!?
>>>>>>>>>
>>>>>>>>> I (490213) ovms-server-v2: Connection is ovms.dexters-web.de:6870 <http://ovms.dexters-web.de:6870/>
>>>>>>>>> TEST1
>>>>>>>>> I (490213) ovms-server-v2: Status: Connecting...
>>>>>>>>> V (490723) ovms-server-v2:
>>>>>>>>> OvmsServerV2MongooseCallback(MG_EV_CONNECT=-3)
>>>>>>>>> W (490723) ovms-server-v2: Connection failed
>>>>>>>>> E (490723) ovms-server-v2: Status: Error: Connection failed
>>>>>>>>> V (490723) ovms-server-v2: OvmsServerV2MongooseCallback(MG_EV_CLOSE)
>>>>>>>>> I (490723) ovms-server-v2: Status: Disconnected
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Am 28.09.21 um 14:32 schrieb Mark Webb-Johnson:
>>>>>>>>>> Shall we release a full update? The last 3.2?
>>>>>>>>>>
>>>>>>>>>> What we have now in master seems stable.
>>>>>>>>>>
>>>>>>>>>> Mark
>>>>>>>>>>
>>>>>>>>>>> On 28 Sep 2021, at 5:39 PM, Michael Balzer <dexter at expeedo.de> <mailto:dexter at expeedo.de>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Everyone,
>>>>>>>>>>>
>>>>>>>>>>> the DST root certificate we include (DST Root CA X3) expires on
>>>>>>>>>>> September 30, i.e. in two days.
>>>>>>>>>>>
>>>>>>>>>>> OVMS# tls trust list
>>>>>>>>>>> DST Root CA X3 length 1200 bytes
>>>>>>>>>>> 1200 byte certificate: DST Root CA X3
>>>>>>>>>>> cert. version : 3
>>>>>>>>>>> serial number :
>>>>>>>>>>> 44:AF:B0:80:D6:A3:27:BA:89:30:39:86:2E:F8:40:6B
>>>>>>>>>>> issuer name : O=Digital Signature Trust Co., CN=DST Root
>>>>>>>>>>> CA X3
>>>>>>>>>>> subject name : O=Digital Signature Trust Co., CN=DST Root
>>>>>>>>>>> CA X3
>>>>>>>>>>> issued on : 2000-09-30 21:12:19
>>>>>>>>>>> * expires on : 2021-09-30 14:01:15*
>>>>>>>>>>> signed using : RSA with SHA1
>>>>>>>>>>> RSA key size : 2048 bits
>>>>>>>>>>> basic constraints : CA=true
>>>>>>>>>>> key usage : Key Cert Sign, CRL Sign
>>>>>>>>>>>
>>>>>>>>>>> AFAICT, this root certificate is currently used by the OVMS to
>>>>>>>>>>> validate Let's Encrypt certificates.
>>>>>>>>>>>
>>>>>>>>>>> *
>>>>>>>>>>> https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/ <https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/>
>>>>>>>>>>> * https://letsencrypt.org/docs/certificate-compatibility/ <https://letsencrypt.org/docs/certificate-compatibility/>
>>>>>>>>>>>
>>>>>>>>>>> Unfortunately, we missed adding the followup LE root certificate
>>>>>>>>>>> "ISRG
>>>>>>>>>>> Root X1" in time.
>>>>>>>>>>>
>>>>>>>>>>> I've just added that certificate to our builtin certificate
>>>>>>>>>>> repository, but it's too late now to roll out a "main" update in
>>>>>>>>>>> time
>>>>>>>>>>> (isn't it?).
>>>>>>>>>>>
>>>>>>>>>>> So, to prevent losing TLS connectivity with LE servers, users need
>>>>>>>>>>> to
>>>>>>>>>>> manually add the ISRG Root X1 certificate to their TLS
>>>>>>>>>>> repositories.
>>>>>>>>>>>
>>>>>>>>>>> I've added a section on this to our user manual:
>>>>>>>>>>>
>>>>>>>>>>> * https://docs.openvehicles.com/en/latest/userguide/ssltls.html <https://docs.openvehicles.com/en/latest/userguide/ssltls.html>
>>>>>>>>>>>
>>>>>>>>>>> If users contact you, point them to that page.
>>>>>>>>>>>
>>>>>>>>>>> We probably should also remove the expired DST root certificate
>>>>>>>>>>> after
>>>>>>>>>>> September 30.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Michael
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>>>>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> OvmsDev mailing list
>>>>>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> OvmsDev mailing list
>>>>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>>>> --
>>>>>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> OvmsDev mailing list
>>>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>>> --
>>>>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OvmsDev mailing list
>>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>> --
>>>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OvmsDev mailing list
>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>> --
>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>
>>> --
>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>
> --
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20210929/cb2c3b99/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20210929/cb2c3b99/attachment-0001.sig>
More information about the OvmsDev
mailing list