[Ovmsdev] OVMS v2 Memory/Settings Corruption

[Michael Balzer]

Got the buffer overrun effect yesterday with the new release.

As I now...

> - Keep volatile features over reset

...it now also shows up with random data in features 0-7. Annoying.

Julien, Pierre, any crash debug data collected yet?

Regards,
Michael


Am 29.11.2016 um 22:13 schrieb Julien Banchet:
> Thank you Michael,
> Will flash it this weekend!
>
> MfG,
> JaXX./.
>
> On Tue, Nov 29, 2016 at 10:10 PM, Michael Balzer <dexter at expeedo.de
> <mailto:dexter at expeedo.de>> wrote:
>
>     Julien, Pierre,
>
>     here's a firmware build with crash debug enabled:
>     https://dexters-web.de/f/tw-beta/OVMS-Twizy-3.8.2-crashdebug.zip
>     <https://dexters-web.de/f/tw-beta/OVMS-Twizy-3.8.2-crashdebug.zip>
>
>     It excludes the diag mode (serial interface command shell) but is
>     otherwise complete.
>
>     The crash debug logs to the server (when it is/becomes available),
>     historical record type "*-OVM-DebugCrash". These records expire
>     after 30 days, so you can collect some days.
>
>     The crash count and data from the most recent crash will also be
>     shown by the SMS/text command "DIAG".
>
>     The crash records contain a crash count, crash reasons (flags from
>     RCON) and the last checkpoint seen before the crash.
>
>     Regards,
>     Michael
>
>
>
>     Am 28.11.2016 um 22:37 schrieb Julien Banchet:
>>     Sure Michael,
>>
>>     I've never got around compiling it right, and I'm out of a
>>     Windows since, wow, I don't even know if grub would boot the
>>     partition anymore... If you have it under hand (i hope), I'd be
>>     happy to give it a shot, it would also be the occasion for me
>>     next weekend to pull a ribbon cable out once and for all (or find
>>     a clever way to keep it hidden though programmable)
>>
>>     How does crash recording work ? over Diag ? (PS: 50M/month data
>>     if needed)
>>
>>     Julien
>>     ./.
>>
>>     On Mon, Nov 28, 2016 at 10:16 PM, Michael Balzer
>>     <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>
>>         There's no memory management in the module, there's too few
>>         RAM available to do dynamic allocations.
>>
>>         All buffers are static, the error can be some pointer
>>         corruption or missing length check / end of string check.
>>
>>         A stack overflow could also be possible, the software stack
>>         is 256 bytes (1 page). As there's not enough ROM left to
>>         support the crash debug framework anymore in the Twizy build,
>>         I cannot tell if that happens again.
>>
>>         Julien, you could try a reduced build with crash debug
>>         recording enabled...
>>
>>         Regards,
>>         Michael
>>
>>
>>
>>         Am 28.11.2016 um 22:00 schrieb Greg D.:
>>>         Ah!  That sounds much more likely.
>>>
>>>         Is there an error (timeout) path that might be freeing the
>>>         buffer when it's still in use?
>>>
>>>         Greg.
>>>
>>>
>>>         Michael Balzer wrote:
>>>>         Hi Greg, welcome :)
>>>>
>>>>         Juliens parameter #0 has been overwritten with "Topping o",
>>>>         which is part of the charge status message ("Topping off").
>>>>
>>>>         So it's much more likely we've got a buffer corruption bug
>>>>         that's triggered by (maybe) bad connectivity situations.
>>>>
>>>>         Regards,
>>>>         Michael
>>>>
>>>>
>>>>         Am 28.11.2016 um 06:38 schrieb Greg D.:
>>>>>         Hi all,
>>>>>
>>>>>         New here, so this may be totally off the wall.
>>>>>
>>>>>         I notice that the first example, with a "5" going to a
>>>>>         "k", it may be more easily explained as a bit */shift/*
>>>>>         instead of a set of flips.  Is there anywhere in the
>>>>>         system where the bytes are clocked serially between
>>>>>         devices, where there may be some marginal timing?
>>>>>
>>>>>         Just a thought.
>>>>>
>>>>>         Greg.
>>>>>
>>>>>
>>>>>         Mark Webb-Johnson wrote:
>>>>>>         $ perl -e 'printf "%08b\n",ord("5");'
>>>>>>         00110101
>>>>>>         $ perl -e 'printf "%08b\n",ord("k");'
>>>>>>         01101011
>>>>>>         $ perl -e 'printf "%08b\n",ord("8");'
>>>>>>         00111000
>>>>>>         $ perl -e 'printf "%08b\n",ord("Z");'
>>>>>>         01011010
>>>>>>
>>>>>>         Strange. Seems more than a simple bit flip. I haven’t seen this myself, or heard of it much from other users.
>>>>>>
>>>>>>         Is the Twizy code using EEPROM for anything other than settings storage? I know that there are a very limit number of write cycles into that EEPROM space.
>>>>>>
>>>>>>         I did some googling, but all I can find is corruption during writing (either power down, or interrupt during write). But, for our case of very very rare EEPROM writes, that shouldn’t be an issue.
>>>>>>
>>>>>>         Regards, Mark.
>>>>>>
>>>>>>>         On 25 Nov 2016, at 5:47 AM, Michael Balzer <dexter at expeedo.de> <mailto:dexter at expeedo.de> wrote:
>>>>>>>
>>>>>>>         The EEPROM data loss is a known issue, and I'm afraid there's nothing we can do about that. It seems to be a bug in the PIC18.
>>>>>>>
>>>>>>>         Regards
>>>>>>>         Michael
>>>>>>>
>>>>>>>         Am 24.11.2016 um 22:10 schrieb Julien Banchet:
>>>>>>>>         Hi all,
>>>>>>>>
>>>>>>>>         Thanks to Michaels last message to remind me to ask a question here:
>>>>>>>>
>>>>>>>>         Since a few months, maybe a bit more because it happens only once in a while, I have noticed I wasn't receiving the SMS alerts (feature on SMSIP mode) , and sending a "Stats?" only gave me the nasty access denied answer.
>>>>>>>>
>>>>>>>>         It turns out, using the application, I noticed that the phone number was getting corrupted (afaik, it's the only setting affected)
>>>>>>>>
>>>>>>>>         Normally: "+33651886877 <tel:%2B33651886877>" it became "+336k1886877" or "+33651886Z77" etc... like a solar flare bit flip !_______________________________________________
>>>>>>         _______________________________________________
>>>>>>         OvmsDev mailing list
>>>>>>         OvmsDev at lists.teslaclub.hk
>>>>>>         <mailto:OvmsDev at lists.teslaclub.hk>
>>>>>>         http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>>>>         <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>>>
>>>>>         _______________________________________________
>>>>>         OvmsDev mailing list
>>>>>         OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>>>         http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>>>         <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>>         -- 
>>>>         Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>         Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>
>>>>         _______________________________________________
>>>>         OvmsDev mailing list
>>>>         OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>>         http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>>         <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>
>>>         _______________________________________________
>>>         OvmsDev mailing list
>>>         OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>         http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>         <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>
>>         -- 
>>         Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>         Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>>         _______________________________________________ OvmsDev
>>         mailing list OvmsDev at lists.teslaclub.hk
>>         <mailto:OvmsDev at lists.teslaclub.hk>
>>         http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>         <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> 
>>
>>     _______________________________________________
>>     OvmsDev mailing list
>>     OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>     http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>     <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>     -- 
>     Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>     Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>
>     _______________________________________________ OvmsDev mailing
>     list OvmsDev at lists.teslaclub.hk
>     <mailto:OvmsDev at lists.teslaclub.hk>
>     http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>     <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> 
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.teslaclub.hk
> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20161211/9bfe5778/attachment.htm>