[Ovmsdev] Time for 3.2.003? / Issue #241

Fri Sep 6 01:55:19 HKT 2019

I've pushed the nasty workaround: the v2 server checks for no RX over 15 minutes, then restarts the network (wifi & modem) as configured for autostart.

Rolled out on my server in edge as 3.2.002-237-ge075f655.

Please test.

Regards,
Michael

Am 05.09.19 um 01:58 schrieb Mark Webb-Johnson:
>> Mark, you can check your server logs for history messages with ridiculous time offsets:
>>
>>     [sddexter at ns27 server]$ cat log-20190903 | egrep "rx msg h [0-9]+,-[0-9]{4}" | wc -l
>>     455283
>>
>
> I checked my logs and see 12 vehicles showing this. But, 2 only show this for a debugcrash log (which is expected, I guess, if the time is not synced at
> report time). I’ve got 4 cars with the offset > 10,000.
>
> Regards, Mark.
>
>> On 4 Sep 2019, at 4:45 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>
>> Everyone,
>>
>> I've pushed a change that needs some testing.
>>
>> I had the issue myself now parking at a certain distance from my garage wifi AP, i.e. on the edge of "in", after wifi had been disconnected for some hours,
>> and with the module still connected via modem. The wifi blob had been trying to connect to the AP for about two hours.
>>
>> As seen before, the module saw no error, just the server responses and commands stopped coming in. I noticed the default interface was still "st1" despite
>> wifi having been disconnected and modem connected. The DNS was also still configured for my wifi network, and the interface seemed to have an IP address --
>> but wasn't pingable from the wifi network.
>>
>> A power cycle of the modem solved the issue without reboot. So the cause may be in the modem/ppp subsystem, or it may be related (in some weird way) to the
>> default interface / DNS setup.
>>
>> More tests showed the default interface again/still got set by the wifi blob itself at some point, overriding our modem prioritization. The events we didn't
>> handle up to now were "sta.connected" and "sta.lostip", so I added these, and the bug didn't show up again since then. That doesn't mean anything, so we need
>> to test this.
>>
>> The default interface really shouldn't affect inbound packet routing of an established connection, but there always may be strange bugs lurking in those libs.
>>
>> The change also reimplements the wifi signal strength reading, as the tests also showed that still wasn't working well using the CSI callback. It now seems
>> to be much more reliable.
>>
>> Please test & report. The single module will be hard to test, as the bug isn't reproducable easily, but you can still try if wifi / modem transitions work well.
>>
>> Mark, you can check your server logs for history messages with ridiculous time offsets:
>>
>>     [sddexter at ns27 server]$ cat log-20190903 | egrep "rx msg h [0-9]+,-[0-9]{4}" | wc -l
>>     455283
>>
>> The bug now severely affects the V2 server performance, as the server is single threaded and doesn't scale very well to this kind of bulk data bursts,
>> especially when coming from multiple modules in parallel. So we really need to solve this now. Slow reactions or connection drops from my server lately have
>> been due to this bug. If this change doesn't solve it, we'll need to add some reboot trigger on "too many server v2 notification retransmissions" -- or maybe
>> a modem power cycle will do, that wouldn't discard the data.
>>
>> Thanks,
>> Michael
>>
>>
>> Am 03.09.19 um 07:46 schrieb Mark Webb-Johnson:
>>> No problem. We can hold. I won’t commit anything for the next few days (and agree to hold-off on Markos’s pull). Let me know when you are ready.
>>>
>>> Regards, Mark.
>>>
>>>> On 3 Sep 2019, at 1:58 AM, Michael Balzer <dexter at expeedo.de> wrote:
>>>>
>>>> Mark, please wait.
>>>>
>>>> I may just have found the cause for issue #241, or at least something I need to investigate before releasing.
>>>>
>>>> I need to dig into my logs first, and try something.
>>>>
>>>> Regards,
>>>> Michael
>>>>
>>>>
>>>> Am 02.09.19 um 12:23 schrieb Michael Balzer:
>>>>> Nothing open from my side at the moment.
>>>>>
>>>>> I haven't had the time to look in to Markos pull request, but from a first check also think that's going too deep to be included in this release.
>>>>>
>>>>> Regards,
>>>>> Michael
>>>>>
>>>>>
>>>>> Am 02.09.19 um 04:15 schrieb Mark Webb-Johnson:
>>>>>> I think it is well past time for a 3.2.003 release. Things seems table in edge (although some things only partially implemented).
>>>>>>
>>>>>> Anything people want to include at the last minute, or can we go ahead and build?
>>>>>>
>>>>>> Regards, Mark.
>>>>>>
>>>>>> _______________________________________________
>>>>>> OvmsDev mailing list
>>>>>> OvmsDev at lists.openvehicles.com
>>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>>> -- 
>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.openvehicles.com
>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.openvehicles.com
>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>> -- 
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20190905/a3afa93c/attachment.html>