[Ovmsdev] Investigating crashes. And, OVMS stopped collecting data

Fri Jan 1 20:55:01 HKT 2021

Hi,

Thanks for the advice - managed to get backtraces but looks like I need to
keep old ovms3.elfs.

I didn't realise there is a full gdb - I found an option in 'make
menuconfig" to write coredump to "flash" - will that work?  Is Is there
enough room?

If I can get a full core file I can scratch in the stack frames and perhaps
better understand that went wrong.

Steve

On Fri, 1 Jan 2021 at 12:54, Michael Balzer <dexter at expeedo.de> wrote:

> Steve,
>
> decode your backtraces using the "a2l" script (see support folder) and the
> .elf file of the build the crash occured with. The script by default uses
> the latest .elf file.
>
> You can find the crash backtraces of previous crashes on the server in
> table "*-OVM-DebugCrash" (kept for 30 days).
>
> Known crash issues:
> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues?q=is%3Aissue+is%3Aopen+crash
>
> Any help with finding crash reasons is appreciated. I still am under the
> impression the PSRAM bug isn't solved for all edge cases, but also am
> certain we can find fatal bugs in our code as well.
>
> Regards,
> Michael
>
>
> Am 01.01.21 um 08:47 schrieb Steve Davies:
>
> Hi Ethan,
>
> Thanks for the info.  Naively doesn't seem like a javascript program
> should be able to crash the whole system.
>
> I did think about rather posting to ABRP from the server side - IE as data
> is sent to the server then code on the server sends the update to ABRP.
>
> But maybe I buried the lead, which really was to find out how I can get a
> lower level view of what crashed the board, and how to turn those memory
> addresses into a better idea as to where in the code the crash occurred.
>
> Do you know, or Michael can you help?
>
> Thanks,
> Steve
>
>
>
> On Fri, 1 Jan 2021 at 03:11, Ethan Rose <ethan at ethanrose.co.nz> wrote:
>
>> I too have crashes often when running the ABRP plugin on my NL ‘13. I’ve
>> resorted to using the LeafSpy/BT option instead on long drives as it’s very
>> unreliable.
>>
>> Regards,
>>
>> Ethan Rose
>>
>> On 1/01/2021, at 08:52, Steve Davies <steve at telviva.co.za> wrote:
>>
>> 
>> Well something had obviously gone off the rails since eventually the OVMS
>> seemed to crash - couldn't ping it - and didn't come back.
>>
>> I power-cycled it and it came back and got a GPS lock.
>>
>> Was there ever discussion of a watchdog to reset things if it goes wonky?
>>
>> Steve
>>
>> On Thu, 31 Dec 2020 at 19:37, Steve Davies <steve at telviva.co.za> wrote:
>>
>>> Hi,
>>>
>>> I went for a scenic drive partly to test my OVMS I3 stuff and feeding
>>> data to ABRP.
>>>
>>> When I got home I found that ABRP had my drive ending say 15km from
>>> home.  And when I checked OVMS it also thought I was in the same spot.
>>>
>>> [6315S] v.p.altitude                             65.4m
>>> [6315S] v.p.direction                            97.5°
>>> [6315S] v.p.gpshdop                              1
>>> [9999S] v.p.gpslock                              yes
>>> [6315S] v.p.gpsmode                              AN
>>> [6315S] v.p.gpsspeed                             18.7052km/h
>>> [6315S] v.p.latitude                             -34.0808
>>> [-----] v.p.location
>>> [6315S] v.p.longitude                            18.4327
>>> [5282S] v.p.odometer                             14901km
>>> [6315S] v.p.satcount                             9
>>> [5195S] v.p.speed                                0km/h
>>> [5195S] v.p.trip                                 85km
>>>
>>> You can see that the metrics are very old (I show more digits in my
>>> version).
>>>
>>> I then looked further and no metrics seemed to be updating - for
>>> instance the m.time.utc wasn't updating.
>>>
>>> I was starting to look at it when my OVMS rebooted (which seems to
>>> happen sometimes.  I've also had the logging stop - I noticed one less task
>>> so I presume that's related).
>>>
>>>
>>> Last boot was 127 second(s) ago
>>> Time at boot: 2020-12-31 19:10:09 SAST
>>> This is reset #8 since last power cycle
>>> Detected boot reason: Crash (12/12)
>>> Reset reason: Exception/panic (4)
>>> Crash counters: 1 total, 0 early
>>>
>>> Last crash: LoadProhibited exception on core 1
>>> Registers:
>>> PC : 0x400f0161 PS : 0x00060a30 A0 : 0x800f023a A1 : 0x3ffe5920
>>> A2 : 0x3ffb5738 A3 : 0x00000000 A4 : 0x3f8566a8 A5 : 0x00000001
>>> A6 : 0x3ffe5990 A7 : 0x3f8567d0 A8 : 0x800f015c A9 : 0x3ffe5910
>>> A10 : 0x00000000 A11 : 0x3f414cb1 A12 : 0x00000002 A13 : 0x00000001
>>> A14 : 0x000000fe A15 : 0x3ffd4f90 SAR : 0x00000004 EXCCAUSE: 0x0000001c
>>> EXCVADDR: 0x00000000 LBEG : 0x4008bcf1 LEND : 0x4008bd01 LCOUNT :
>>> 0xfffffffd
>>> Backtrace:
>>> 0x400f0161 0x400f0237 0x400fe87f 0x40103d4b 0x40103f02 0x400fe8af
>>> 0x4013a432
>>> Version: 3.2.015-267-g39dba375-dirty/ota_1/main (build idf
>>> v3.3.4-845-gd59ed8bba-dirty Dec 26 2020 15:37:52)
>>>
>>> After this crash and restart m.time.utc was empty. I rebooted and its
>>> still missing - does it only appear once NTP syncs or something?
>>>
>>> Also I have no GPS position, gpsmode or anything like that (but I have a
>>> 3G session with the hologram sim and roaming on one of the local cellular
>>> operators).
>>>
>>> I've made sure the GPS puck is out in the open and I guess I'll wait -
>>> does the time come from the GPS?  And any ideas why I'm not getting a GPS
>>> lock suddenly?
>>>
>>> Apart from that - how do I debug crashes?  I went looking for some sort
>>> of loader memory map in the build directory to try turn the PC and
>>> backtrace addresses into something more useful but didn't pick up anything.
>>>
>>> Thanks
>>> Steve
>>>
>>>
>>>
>>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>
> _______________________________________________
> OvmsDev mailing listOvmsDev at lists.openvehicles.comhttp://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
>
> --
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20210101/e766e77a/attachment-0003.htm>