[Ovmsdev] Anyone using HTTP API?
Mark Webb-Johnson
mark at webb-johnson.net
Sun Feb 23 20:54:41 HKT 2020
This is turning into a bigger job than I imagined. "Give a mouse a cookie", and all that.
The ovms_server.pl has gotten horrendous over the years. Almost 3,000 monolithic lines of code, 4 server listeners, three different types of server, push notifications, database synchronisation, etc. I tried turning on ‘use strict’ and it showed up a bunch of bugs and errors.
So, I am refactoring it to a plugin architecture. That should make it more maintainable, and also provide a foundation for it to work better with the v3 MQTT. I suggest people hold off from making any changes to the server code in the next few days.
Regards, Mark
> On 21 Feb 2020, at 9:48 PM, Mark Webb-Johnson <mark at webb-johnson.net> wrote:
>
> Ok. I will rework a modular approach. Should be able to get this done over the weekend.
>
> Yes, strict and warn would help.
>
> Mark
>
> P.S. Explains why nobody used the http api on my server :-)
>
>> On 21 Feb 2020, at 9:35 PM, Michael Balzer <dexter at expeedo.de> wrote:
>>
>> Thanks Mark,
>>
>> I must have been blind… but perl also never fails to amaze me in terms of "compiles fine, but won't run" -- $ph isn't defined anywhere else. Maybe "strict" mode would have told me about that.
>>
>> And I didn't think about meta data in the hash. You're right, we need to pass both values to the function. And I need to rework my password hashing…
>>
>> A modular solution seems to be best, easy to add custom implementations and to provide some standard modules.
>>
>> Regards,
>> Michael
>>
>>
>> Am 21.02.20 um 12:15 schrieb Mark Webb-Johnson:
>>>
>>> An alternative would be to implement a server authentication module, and to ‘require’ that into the system at startup:
>>>
>>> require $config->val('db',’pw_module’,’auth_none.pl’);
>>>
>>> …
>>>
>>> If (&auth_password_check($passwordhash, $password))
>>> ...
>>>
>>> Provide a ‘auto_none.pl’:
>>>
>>> #!/usr/bin/perl
>>>
>>> sub auth_password_check
>>> {
>>> my ($hash,$password) = @_;
>>>
>>> return 0;
>>> }
>>>
>>> Then a ‘auth_drupal7.pl’, ‘auth_sha1.pl’, etc.
>>>
>>> This could also be done in a perl modular fashion by having the module provided as an object (using ‘use …’). Probably cleaner than the old-style require.
>>>
>>> That is much more extendable and standardised. In particular, there is also code that syncs Drupal users to ovms_owners (svr_tim) and if we have a separate module that drupal-dependant code could be removed from ovms_server.pl.
>>>
>>> Is that a better solution?
>>>
>>> Regards, Mark.
>>>
>>>> On 21 Feb 2020, at 11:37 AM, Mark Webb-Johnson <mark at webb-johnson.net <mailto:mark at webb-johnson.net>> wrote:
>>>>
>>>> Michael,
>>>>
>>>> Just before your commit, the server code was:
>>>>
>>>> my $passwordhash = $row->{'pass'};
>>>> if (&drupal_password_check($passwordhash, $password))
>>>>
>>>> …
>>>>
>>>> sub drupal_password_check
>>>> {
>>>> my ($ph,$password) = @_;
>>>>
>>>> my $iter_log2 = index($itoa64,substr($ph,3,1));
>>>> my $iter_count = 1 << $iter_log2;
>>>>
>>>> my $phash = substr($ph,0,12);
>>>> my $salt = substr($ph,4,8);
>>>>
>>>> my $hash = sha512($salt.$password);
>>>> do
>>>> {
>>>> $hash = sha512($hash.$password);
>>>> $iter_count--;
>>>> } while ($iter_count > 0);
>>>>
>>>> my $encoded = substr($phash . &drupal_password_base64_encode($hash,length($hash)),0,55);
>>>>
>>>> return ($encoded eq $ph);
>>>> }
>>>>
>>>> Your change was:
>>>>
>>>> # User password encoding function:
>>>> my $pw_encode = $config->val('db','pw_encode','drupal_password($password)’);
>>>>
>>>> …
>>>>
>>>> my $passwordhash = $row->{'pass'};
>>>> my $encoded = eval $pw_encode;
>>>> if ($encoded eq $passwordhash)
>>>>
>>>> …
>>>>
>>>> sub drupal_password
>>>> {
>>>> my ($password) = @_;
>>>>
>>>> my $iter_log2 = index($itoa64,substr($ph,3,1));
>>>> my $iter_count = 1 << $iter_log2;
>>>>
>>>> my $phash = substr($ph,0,12);
>>>> my $salt = substr($ph,4,8);
>>>>
>>>> my $hash = sha512($salt.$password);
>>>> do
>>>> {
>>>> $hash = sha512($hash.$password);
>>>> $iter_count--;
>>>> } while ($iter_count > 0);
>>>>
>>>> my $encoded = substr($phash . &drupal_password_base64_encode($hash,length($hash)),0,55);
>>>>
>>>> return $encoded;
>>>> }
>>>>
>>>> You changed the parameters from ($ph,$password) to just ($password), but the drupal_password function still needs to use $ph (the hash) to extract meta data to set the encoding parameters.
>>>>
>>>> The problem is that Drupal (and others) has a strong hashing function with multiple iterations. The meta data for that is stored in the password hash itself. The unix crypt library does something similar (with the encoding method and salt stored as meta data in the hash). Just storing passwords as straight hashes (md5, sha1, etc) is fundamentally not secure, as it is trivial to use rainbow tables to break the hashes - so most modern systems use iterations, salts, or other techniques to limit the effectiveness of rainbow tables and make brute force approaches computationally unfeasible.
>>>>
>>>> For many systems, we can only encode a password in the same way as a previous encoding if we know the meta data of the previous encoding (and that is stored in the hash). Hence we need the hash as a parameter, to extract the meta data to be able to encode the new password in the same way.
>>>>
>>>> This won’t just affect drupal, but any system with a non-trivial password hashing function.
>>>>
>>>> So, pw_encode() needs both the old hash as well as the plaintext password to encode. At which point, I think it becomes easier to make it simply pw_check() returning a boolean. It also seems easier to me to do that as a plugin function (pw_check vs pw_encode) as it will allow other non-trivial hashing comparisons if required. For example, say you needed to check the password against an external lookup (ldap, etc).
>>>>
>>>> Regards, Mark.
>>>>
>>>>> On 21 Feb 2020, at 1:12 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>>>
>>>>> Mark,
>>>>>
>>>>> I did 1b73a7f8 to split the "create & compare password" function into separate "create" & "compare" steps, and introduced the "pw_encode" config hook to be able to supply just a custom "create" operation. That simplifies the config (see example).
>>>>>
>>>>> That change has been working since 2016 on my server. I see you reintroduced the "create & compare" function as a separate function for the MQTT auth, but don't see why that was needed. I also don't see why the separated function was broken on your server. Can you please elaborate? I'd like to understand what was going wrong.
>>>>>
>>>>> With reverting to the "create & compare", this breaks the configuration of servers not using Drupal. Essentially, the new "pw_check" hook does just the previous "pw_encode" and adds the comparison to that, so I'd rather opt for adding a default function here that simply reuses the existing "pw_encode" hook.
>>>>>
>>>>> Regards,
>>>>> Michael
>>>>>
>>>>>
>>>>> Am 20.02.20 um 04:09 schrieb Mark Webb-Johnson:
>>>>>> Even stranger. This conversation obviously triggered someone to try it and then raise a support ticket that HTTP API authentication didn’t work.
>>>>>>
>>>>>> It seems a change was made back in 2016-02-01 23:59:22 (1b73a7f8) that broke the pw_encode function (drupal_password). It was also weird because we had drupal_password and drupal_password_check functions, doing pretty much the same thing (one used by HTTP API and the other by MQ authentication).
>>>>>>
>>>>>> I standardised to use a new pw_check (overridable in the config) parameter, which defaults to:
>>>>>>
>>>>>> drupal_password_check($passwordhash,$password)
>>>>>>
>>>>>> and stopped using the pw_encode config value. I also changed the MQ authentication stuff to use the same pw_check parameter (so both authentication uses are now able to be changed in the same config). If using something other than drupal, just need to change the pw_check parameter in the config.
>>>>>>
>>>>>> I realise that this may break other users of the server, but it doesn’t seem a difficult fix to make, and is a much better approach.
>>>>>>
>>>>>> Regards, Mark
>>>>>>
>>>>>>> On 19 Feb 2020, at 1:53 PM, Mark Webb-Johnson <mark at webb-johnson.net <mailto:mark at webb-johnson.net>> wrote:
>>>>>>>
>>>>>>> Strange. I have zero using mine. Must be a EU thing?
>>>>>>>
>>>>>>> I’ll keep it in mind and try not to break anything.
>>>>>>>
>>>>>>> Regards, Mark.
>>>>>>>
>>>>>>>> On 18 Feb 2020, at 8:41 PM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>>>>>>
>>>>>>>> Mark,
>>>>>>>>
>>>>>>>> grep "main: http" in the log: yes, I've got some users accessing the API frequently.
>>>>>>>>
>>>>>>>> Usage is mostly /api/charge followed by /api/status & /api/historical, but almost all calls have been used during the last days.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Michael
>>>>>>>>
>>>>>>>>
>>>>>>>> Am 18.02.20 um 04:28 schrieb Mark Webb-Johnson:
>>>>>>>>> Is anyone here using the HTTP API at all?
>>>>>>>>>
>>>>>>>>> It seems so tied to the v2 protocol, as to not be much use.
>>>>>>>>>
>>>>>>>>> Regards, Mark.
>>>>>>>>> _______________________________________________
>>>>>>>>> OvmsDev mailing list
>>>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OvmsDev mailing list
>>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OvmsDev mailing list
>>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OvmsDev mailing list
>>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>>
>>>>> --
>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>> _______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>
>> --
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20200223/7264f6a9/attachment-0001.html>
More information about the OvmsDev
mailing list