[Ovmsdev] SSL Support
Michael Balzer
dexter at expeedo.de
Fri Feb 14 20:18:01 HKT 2020
Mark,
it's really a memory issue, but I wouldn't have expected that, given…
OVMS# mo me
Free 8-bit 58944/271108, 32-bit 1900/27848, SPIRAM 3761072/4194252
Switching to vehicle NONE gave me
OVMS# mo me
Free 8-bit 69124/271108, 32-bit 1900/27848, SPIRAM 3802648/4194252
… and a working HTTP.Request with SSL (but it's sloooooow, see below).
I now did some more tests and could get it working with…
> CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y
… so I assume we need to add that change to our default sdkconfig as well.
Regarding the performance, SSL connections are somewhere beteen awful & unusable now. Test:
(function(){
var t0 = performance.now();
HTTP.Request({
url: "http://ovms.dexters-web.de/f/test.json",
always: function() {
var t1 = performance.now();
print(this.url + " => " + (t1-t0).toFixed(0) + " ms, body.length=" + this.response.body.length);
}
});
})();
This gives me:
I (520500) script: [eval:7:] http://ovms.dexters-web.de/f/test.json => 267 ms, body.length=13
I (550250) script: [eval:7:] https://ovms.dexters-web.de/f/test.json => 5347 ms, body.length=13
… up to …
I (68270) script: [eval:7:] https://ovms.dexters-web.de/f/test.json => 11930 ms, body.length=13
… regardless of actually using the CA verification or reverting to "*".
I don't know how much of this is TLS session init overhead. Do you have better performance with the persistent V2 SSL connection? Haven't had time to test that yet.
Regards,
Michael
Am 14.02.20 um 10:08 schrieb Mark Webb-Johnson:
> Michael,
>
> I am getting:
>
> OVMS# script eval 'HTTP.Request({url: "http://dexters-web.de/f/test.json",always: function() { JSON.print(this, false); } });'
> I (140781) script: [eval:1:] {"url":"https://dexters-web.de/f/test.json","always":function () { [ecmascript code]
> },"redirectCount":1,"error":"","response":{"statusCode":200,"statusText":"OK","body":"{\"foo\":\"bar\"}","headers":[{"Date":"Fri, 14 Feb 2020 08:45:05
> GMT"},{"Server":"Apache/2.2.15 (CentOS)"},{"Last-Modified":"Sun, 01 Dec 2019 20:36:31
> GMT"},{"ETag":"\"73806cb-d-598aa6b0f7003\""},{"Accept-Ranges":"bytes"},{"Content-Length":"13"},{"Cache-Control":"max-age=0"},{"Expires":"Fri, 14 Feb 2020
> 08:45:05 GMT"},{"Content-Type":"application/json"}]}}
>
> OVMS# script eval 'HTTP.Request({url: "https://dexters-web.de/f/test.json",always: function() { JSON.print(this, false); } });'
> I (169991) script: [eval:1:] {"url":"https://dexters-web.de/f/test.json","always":function () { [ecmascript code]
> },"redirectCount":0,"error":"","response":{"statusCode":200,"statusText":"OK","body":"{\"foo\":\"bar\"}","headers":[{"Date":"Fri, 14 Feb 2020 08:45:34
> GMT"},{"Server":"Apache/2.2.15 (CentOS)"},{"Last-Modified":"Sun, 01 Dec 2019 20:36:31
> GMT"},{"ETag":"\"73806cb-d-598aa6b0f7003\""},{"Accept-Ranges":"bytes"},{"Content-Length":"13"},{"Cache-Control":"max-age=0"},{"Expires":"Fri, 14 Feb 2020
> 08:45:34 GMT"},{"Content-Type":"application/json"}]}}
>
> OVMS# metrics list version
> m.version 3.2.010-15-g931ca3d3-dirty/factory/edge (build idf v3.3-beta3-770-ge97f72ea2 Feb 13 2020 13:01:16)
>
>
> That seems correct to me.
>
> Maybe increased ram usage for you? I your memory status ok?
>
> The change I made to mongoose would only affect if the ca_cert started with “-----“. Nothing else should have changed (at least in that part). Maybe the
> increase of fragment size to 16384 has impacted something?
>
>> I've seen you set opts.ssl_server_name in server V2 & V3. This could be left NULL for using the hostname before, is it now necessary to set this?
>
> I think you are right. I hadn’t noticed that default ca_cert != NULL, but server_name is null. I don’t think it is necessary to set explicitly, but should not
> matter.
>
> However, I have just noticed:
>
> void mg_ssl_if_conn_free(struct mg_connection *nc) {
> struct mg_ssl_if_ctx *ctx = (struct mg_ssl_if_ctx *) nc->ssl_if_data;
> if (ctx == NULL) return;
> nc->ssl_if_data = NULL;
> MG_FREE(ctx->ssl_cert);
> MG_FREE(ctx->ssl_key);
> MG_FREE(ctx->ssl_ca_cert);
> MG_FREE(ctx->ssl_server_name);
> memset(ctx, 0, sizeof(*ctx));
> MG_FREE(ctx);
> }
>
>
> Which is scary. But that seems to be the ’simplessl’ version of that function. There are two other ones, for different SSL implementations, which seem to
> behave differently. That monolithic mongoose.{h,c} is really confusing to work through. Maybe I should look at their python mechanism to pack/unpack from
> individual scripts and then try to build a mongoose.{h,c} specific to our needs (without the crud we don’t use such as openssl, simplessl)?
>
> Regards, Mark.
>
>> On 14 Feb 2020, at 6:25 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>
>> Mark,
>>
>> something's wrong now with the Mongoose SSL support. I could do this javascript call before (example from scripting API doc):
>>
>> HTTP.Request({
>> url: "https://dexters-web.de/f/test.json",
>> always: function() { JSON.print(this, false); }
>> });
>>
>> This now no longer works, neither with the previous opts.ssl_ca_cert = "*":
>>
>> E (37350) mongoose: mg_ssl_if_mbed_err 0x3f85befc SSL error: -17040
>> I (37410) script: [eval:3:] {"url":"https://dexters-web.de/f/test.json","always":function () { [ecmascript code] },"redirectCount":0,"error":"SSL error"}
>>
>> …nor with opts.ssl_ca_cert = MyOvmsTLS.GetTrustedList(), it then sometimes fails with…
>>
>> {"url":"https://dexters-web.de/f/test.json","always":function () { [ecmascript code] },"redirectCount":0,"error":"Failed to create SSL session"}
>>
>> …but mostly causes some kind of lockup (USB console not responding) and / or wifi / mongoose restart:
>>
>> I (55980) wifi: bcn_timout,ap_probe_send_start
>> I (58480) wifi: ap_probe_send over, resett wifi status to disassoc
>> I (58480) wifi: state: run -> init (c800)
>> I (58490) wifi: pm stop, total sleep time: 35296441 us / 53655320 us
>> I (58490) wifi: new:<11,0>, old:<11,2>, ap:<11,2>, sta:<11,0>, prof:11
>> E (58490) mongoose: mg_ssl_if_mbed_err 0x3f85dd64 SSL error: -1
>> D (58520) events: Signal(server.web.socket.closed)
>> I (58580) script: [eval:3:] {"url":"https://dexters-web.de/f/test.json","always":function () { [ecmascript code] },"redirectCount":0,"error":"SSL error"}
>> W (58600) wifi: Haven't to connect to a suitable AP now!
>> W (58630) wifi: Haven't to connect to a suitable AP now!
>> W (58640) wifi: Haven't to connect to a suitable AP now!
>> W (58660) wifi: Haven't to connect to a suitable AP now!
>> W (58680) wifi: Haven't to connect to a suitable AP now!
>> W (58700) wifi: Haven't to connect to a suitable AP now!
>> W (58720) wifi: Haven't to connect to a suitable AP now!
>> W (58740) wifi: Haven't to connect to a suitable AP now!
>> D (58770) events: Signal(system.event)
>> D (58780) events: Signal(system.wifi.sta.disconnected)
>>
>>
>> I've seen you set opts.ssl_server_name in server V2 & V3. This could be left NULL for using the hostname before, is it now necessary to set this?
>>
>> Regards,
>> Michael
>>
>>
>> Am 13.02.20 um 08:26 schrieb Mark Webb-Johnson:
>>>
>>> And #2 now done. To add to the trusted CA list we can now add the CA into firmware (like the 3 already there), or put the PEM formatted certificate in
>>> /store/trustedca on the module itself. A set of commands (like ’tls trusted reload’, ’tls trusted list’, etc) are also now provided to help management.
>>>
>>> I’ve also added a short document to the user guide to explain this.
>>>
>>> Regards, Mark.
>>>
>>>> On 12 Feb 2020, at 3:25 PM, Mark Webb-Johnson <mark at webb-johnson.net <mailto:mark at webb-johnson.net>> wrote:
>>>>
>>>> I managed to get quite a lot done last night, and just committed my changes. Current status:
>>>>
>>>> #1 is done. An extension had to be made to mongoose to support this, so please update submodules. I’ve added the trusted CAs for openvehicles, dexters
>>>> (letsencrypt) and pushover.
>>>>
>>>> #3 is done. A couple of lines of code. It is enabled by ‘config set server.v2 tls yes’.
>>>>
>>>> #4 is done. A couple of lines of code. It is enabled by ‘config set server.v3 tls yes’.
>>>>
>>>> #5 is done, and live on api.openvehicles.com <http://api.openvehicles.com/>.
>>>>
>>>> #8 can start when ready. Have a look at the ovms_server_v[23] code for an example - it is literally a couple of lines to be added.
>>>>
>>>> I did find a few servers that didn’t support the segment size negotiation option in SSL that we were relying on (including mosquitto!). So had to change my
>>>> sdkconfig to:
>>>>
>>>> CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN=16384
>>>>
>>>>
>>>> I suggest you do the same for best compatibility.
>>>>
>>>> The rest is pending. I will work on #2 next, then move on to the iOS App.
>>>>
>>>> Regards, Mark.
>>>>
>>>>> On 10 Feb 2020, at 11:24 AM, Mark Webb-Johnson <mark at webb-johnson.net <mailto:mark at webb-johnson.net>> wrote:
>>>>>
>>>>>
>>>>> Given that we use the mongoose library for most of our stuff, adding SSL support should not be hard. This would finally bring strong encryption and server
>>>>> side authentication. Given the number of attacks now on IoT devices that would not be a bad thing.
>>>>>
>>>>> I think what we need is:
>>>>>
>>>>> 1. A set of helper functions to make it easier for components to use SSL. Build on top of mongoose.
>>>>>
>>>>> 2. A way to manage a list of trusted Certificate Authorities, including adding to the trusted list via:
>>>>> * Components providing Certificate Authorities in firmware.
>>>>> * Certificate Authorities in configuration.
>>>>>
>>>>> 3. Extensions to ovms_server_v2 to support an SSL connection option.
>>>>>
>>>>> 4. Extensions to ovms_server_v3 to support an SSL connection option.
>>>>>
>>>>> 5. Extensions to the Ovms Server v2 code to support an SSL connection listener.
>>>>>
>>>>> 6. Extensions to the iOS App to support an SSL connection option.
>>>>>
>>>>> 7. Extensions to the Android App to support an SSL connection option.
>>>>>
>>>>> 8. Migration of any components already supporting SSL to this new standardised approach.
>>>>>
>>>>> 9. Then we can open up the discussion of the whole thing of passwords. We have far too many of these at the moment (user account+password, vehicle ID,
>>>>> server password, module password, hologram account+password, etc). Once we have an encrypted connection, we don’t need to use the password for
>>>>> encryption, but merely for authentication. That simplifies things, as we can perhaps just use the user account+password for most things (giving access
>>>>> to all vehicles registered under that user account - in a similar way to MQTT does it already for ovms_server_v3).
>>>>>
>>>>>
>>>>> I will take on the majority of this project. I can do #1, #2, #3, #4, #5, and #6).
>>>>>
>>>>> If anyone has any feedback on requirements, please let me know.
>>>>>
>>>>> Regards, Mark.
>>>>>
>>>>> _______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>
>>>
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.openvehicles.com
>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>> --
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20200214/7b407e4b/attachment.htm>
More information about the OvmsDev
mailing list