[Ovmsdev] Crashing on restart

[Michael Balzer]

Everyone,

I've just checked in my changes, so this is now included in the edge 
build (3.3.004-331-g4a7b966d, already available on dexters-web).

I had no issues with my modules, but depending on the vehicle & 
configuration, we may need to raise the new default queue size beyond 100.

Please test & report. If the queue is too small, the module will crash 
with a "queue overflow" log entry.

Regards,
Michael


Am 06.01.25 um 20:49 schrieb Michael Balzer via OvmsDev:
> Tamás,
>
> I've got a test build that should fix your reboot issue, if my 
> diagnosis was correct. Can you please test this firmware image?
>
> http://ovms.dexters-web.de/f/firmware/test/ovms3.bin
>
> The build works in my test & live module. The changes are not yet 
> checked in, patch attached. This consists of a) delegating registry 
> changes to the event task and b) invalidating callbacks in the list on 
> deregistration before actually deleting them, so the list processing 
> cannot run into deleted entries, when deregistration is done from 
> within a callback execution.
>
> The changes need another mutex and a larger event queue. I had queue 
> size 40 before, minimum now is 80, build uses 100 to be on the safe 
> side. That means another 720 bytes of 8-bit RAM are used for the queue 
> unfortunately, but that shouldn't be a problem yet.
>
> I also again needed to power off my module for the reboot, but with 
> the new firmware, the reboot works. So chances are, this issue is now 
> hit more often than before, probably because of more stuff going on in 
> the shutdown.
>
> Regards,
> Michael
>
>
> Am 02.01.25 um 15:25 schrieb Michael Balzer via OvmsDev:
>> The log shows it actually crashes during the shutdown process -- but 
>> @Michael, I'm pretty sure this is an old issue, turning up now only 
>> by chance.
>>
>> This looks like a race condition between event handler invocation and 
>> deregistration:
>>
>> Guru Meditation Error: Core  1 panic'ed 
>> (InstrFetchProhibited).Exception was unhandled.
>> …
>> Backtrace: 0x41000000:0x3ffc8c30 0x401208ed:0x3ffc8c60 
>> 0x401209d4:0x3ffc8cc0 0x40120a5d:0x3ffc8d00
>>
>> → Crash at *0x41000000* (invalid address), called from:
>>
>> 0x401208ed is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) 
>> (/home/balzer/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:310).
>> 305           {
>> 306           for (EventCallbackList::iterator itc=el->begin(); 
>> itc!=el->end(); ++itc)
>> 307             {
>> 308             m_current_started = monotonictime;
>> 309             m_current_callback = *itc;
>> *310 m_current_callback->m_callback(m_current_event, 
>> msg->body.signal.data);*
>> 311             m_current_callback = NULL;
>> 312             }
>> 313           }
>> 314         }
>>
>> From there it then enters a crash loop from the crash handler somehow 
>> running into an illegal address when trying to access the RTC 
>> boot_data object (?).
>>
>>
>> I think, when introducing the separate event handling task, we missed 
>> adding a lock to make manipulations & accesses to the event handler 
>> list atomic.
>>
>> The log shows, the event queue grew significantly due to 
>> duktape/scripts lagging during shutdown (as seen in the log), so the 
>> task was quite busy.
>>
>> So probably a `DeregisterEvent()` call from the shutdown process came 
>> in, invalidating a callback entry just while the event task was 
>> processing that callback list.
>>
>> Solving this may need some additional thoughts & measures: locking 
>> the whole event registry with handlers lagging like this could lead 
>> to a deadlock… probably at least the Duktape component needs to 
>> cleanly abort/reject all pending events before/while deregistering.
>>
>> Michael, if you'd like to have a look at this, go ahead.
>>
>> Regards,
>> Michael
>>
>>
>> Am 01.01.25 um 11:47 schrieb Tamás Kovács via OvmsDev:
>>> Now i change the sd card to a smaller, and formatted to fat32 
>>> (16Gb), but on reboot is crashing.
>>> Now connect a computer to the usb port, and create a log in verbose, 
>>> attached to the email.
>>>
>>>
>>> -- 
>>> Üdvözlettel:
>>> Kovács Tamás
>>>
>>>
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.openvehicles.com
>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>> -- 
>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
>> Fon 02330 9104094 * Handy 0176 20698926
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
> -- 
> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
> Fon 02330 9104094 * Handy 0176 20698926
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
Fon 02330 9104094 * Handy 0176 20698926

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20250108/7766466c/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20250108/7766466c/attachment.sig>