[Ovmsdev] Time for 3.2.003? / Issue #241

Michael Balzer dexter at expeedo.de
Thu Sep 5 02:18:17 HKT 2019 

Steve, everyone,

master is identical, spiram-fix-test only differs in the changes necessary for the new toolchain.

Bad news: I see issue #241 in today's v2 server log from at least two vehicles having already updated to my edge release.

So, sadly, that didn't fix it. That means the bug is most probably in the pppos stack or in our modem driver. There is that open issue about the very slow ppp
data transfer, the modem code definitely needs some refinement. But finding that specific bug will take time, except someone has a direct idea where to look.

I'll implement that nasty workaround now for the release…

Regards,
Michael


Am 04.09.19 um 19:16 schrieb Stephen Casner:
> Michael,
>
> I saw the comments introducing -98 and followup.  I think I'd rather
> not be on the bleeding edge unless that's where your fix for 241 is
> that you need us to test.  Is it on master, and if so, what toolchain
> to use for that?
>
>                                                         -- Steve
>
> On Wed, 4 Sep 2019, Michael Balzer wrote:
>
>> Steve,
>>
>> I meanwhile use toolkit -98 along with the new libs provided here:  https://github.com/espressif/esp-idf/issues/2892
>>
>> ...plus the (not yet checked in) volatile patch described here: https://github.com/espressif/esp-idf/issues/2892#issuecomment-525697255
>>
>> To use the new libs I replaced (symlinked) the respective files in xtensa-esp32-elf/sysroot/lib/esp32-psram by those from the zip.
>>
>> Then checkout and build on our branch "spiram-fix-test".
>>
>> Regards,
>> Michael
>>
>>
>> Am 04.09.19 um 07:04 schrieb Stephen Casner:
>>> Michael,
>>>
>>> I have not built the OVMS software in a while, so I need to update to
>>> the current esp-idf, which I have done by the steps you prescribed a
>>> while back.  But you also recommend the -93 toolchain; I currently
>>> have -80 installed.  Where can I find the -93 tarball?  Google didn't
>>> answer that question.
>>>
>>>                                                         -- Steve
>>>
>>> On Tue, 3 Sep 2019, Michael Balzer wrote:
>>>
>>>> Everyone,
>>>>
>>>> I've pushed a change that needs some testing.
>>>>
>>>> I had the issue myself now parking at a certain distance from my garage wifi AP, i.e. on the edge of "in", after wifi had been disconnected for some hours, and
>>>> with the module still connected via modem. The wifi blob had been trying to connect to the AP for about two hours.
>>>>
>>>> As seen before, the module saw no error, just the server responses and commands stopped coming in. I noticed the default interface was still "st1" despite wifi
>>>> having been disconnected and modem connected. The DNS was also still configured for my wifi network, and the interface seemed to have an IP address -- but
>>>> wasn't pingable from the wifi network.
>>>>
>>>> A power cycle of the modem solved the issue without reboot. So the cause may be in the modem/ppp subsystem, or it may be related (in some weird way) to the
>>>> default interface / DNS setup.
>>>>
>>>> More tests showed the default interface again/still got set by the wifi blob itself at some point, overriding our modem prioritization. The events we didn't
>>>> handle up to now were "sta.connected" and "sta.lostip", so I added these, and the bug didn't show up again since then. That doesn't mean anything, so we need to
>>>> test this.
>>>>
>>>> The default interface really shouldn't affect inbound packet routing of an established connection, but there always may be strange bugs lurking in those libs.
>>>>
>>>> The change also reimplements the wifi signal strength reading, as the tests also showed that still wasn't working well using the CSI callback. It now seems to
>>>> be much more reliable.
>>>>
>>>> Please test & report. The single module will be hard to test, as the bug isn't reproducable easily, but you can still try if wifi / modem transitions work well.
>>>>
>>>> Mark, you can check your server logs for history messages with ridiculous time offsets:
>>>>
>>>>     [sddexter at ns27 server]$ cat log-20190903 | egrep "rx msg h [0-9]+,-[0-9]{4}" | wc -l
>>>>     455283
>>>>
>>>> The bug now severely affects the V2 server performance, as the server is single threaded and doesn't scale very well to this kind of bulk data bursts,
>>>> especially when coming from multiple modules in parallel. So we really need to solve this now. Slow reactions or connection drops from my server lately have
>>>> been due to this bug. If this change doesn't solve it, we'll need to add some reboot trigger on "too many server v2 notification retransmissions" -- or maybe a
>>>> modem power cycle will do, that wouldn't discard the data.
>>>>
>>>> Thanks,
>>>> Michael
>>>>
>>>>
>>>> Am 03.09.19 um 07:46 schrieb Mark Webb-Johnson:
>>>>> No problem. We can hold. I won't commit anything for the next few days (and agree to hold-off on Markos's pull). Let me know when you are ready.
>>>>>
>>>>> Regards, Mark.
>>>>>
>>>>>> On 3 Sep 2019, at 1:58 AM, Michael Balzer <dexter at expeedo.de> wrote:
>>>>>>
>>>>>> Mark, please wait.
>>>>>>
>>>>>> I may just have found the cause for issue #241, or at least something I need to investigate before releasing.
>>>>>>
>>>>>> I need to dig into my logs first, and try something.
>>>>>>
>>>>>> Regards,
>>>>>> Michael
>>>>>>
>>>>>>
>>>>>> Am 02.09.19 um 12:23 schrieb Michael Balzer:
>>>>>>> Nothing open from my side at the moment.
>>>>>>>
>>>>>>> I haven't had the time to look in to Markos pull request, but from a first check also think that's going too deep to be included in this release.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Michael
>>>>>>>
>>>>>>>
>>>>>>> Am 02.09.19 um 04:15 schrieb Mark Webb-Johnson:
>>>>>>>> I think it is well past time for a 3.2.003 release. Things seems table in edge (although some things only partially implemented).
>>>>>>>>
>>>>>>>> Anything people want to include at the last minute, or can we go ahead and build?
>>>>>>>>
>>>>>>>> Regards, Mark.
>>>>>>>>
>> --
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

More information about the OvmsDev mailing list