[Ovmsdev] Crashing on restart

[Michael Balzer]

Tamás,

I've got a test build that should fix your reboot issue, if my diagnosis 
was correct. Can you please test this firmware image?

http://ovms.dexters-web.de/f/firmware/test/ovms3.bin

The build works in my test & live module. The changes are not yet 
checked in, patch attached. This consists of a) delegating registry 
changes to the event task and b) invalidating callbacks in the list on 
deregistration before actually deleting them, so the list processing 
cannot run into deleted entries, when deregistration is done from within 
a callback execution.

The changes need another mutex and a larger event queue. I had queue 
size 40 before, minimum now is 80, build uses 100 to be on the safe 
side. That means another 720 bytes of 8-bit RAM are used for the queue 
unfortunately, but that shouldn't be a problem yet.

I also again needed to power off my module for the reboot, but with the 
new firmware, the reboot works. So chances are, this issue is now hit 
more often than before, probably because of more stuff going on in the 
shutdown.

Regards,
Michael


Am 02.01.25 um 15:25 schrieb Michael Balzer via OvmsDev:
> The log shows it actually crashes during the shutdown process -- but 
> @Michael, I'm pretty sure this is an old issue, turning up now only by 
> chance.
>
> This looks like a race condition between event handler invocation and 
> deregistration:
>
> Guru Meditation Error: Core  1 panic'ed 
> (InstrFetchProhibited).Exception was unhandled.
> …
> Backtrace: 0x41000000:0x3ffc8c30 0x401208ed:0x3ffc8c60 
> 0x401209d4:0x3ffc8cc0 0x40120a5d:0x3ffc8d00
>
> → Crash at *0x41000000* (invalid address), called from:
>
> 0x401208ed is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) 
> (/home/balzer/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:310).
> 305           {
> 306           for (EventCallbackList::iterator itc=el->begin(); 
> itc!=el->end(); ++itc)
> 307             {
> 308             m_current_started = monotonictime;
> 309             m_current_callback = *itc;
> *310 m_current_callback->m_callback(m_current_event, 
> msg->body.signal.data);*
> 311             m_current_callback = NULL;
> 312             }
> 313           }
> 314         }
>
> From there it then enters a crash loop from the crash handler somehow 
> running into an illegal address when trying to access the RTC 
> boot_data object (?).
>
>
> I think, when introducing the separate event handling task, we missed 
> adding a lock to make manipulations & accesses to the event handler 
> list atomic.
>
> The log shows, the event queue grew significantly due to 
> duktape/scripts lagging during shutdown (as seen in the log), so the 
> task was quite busy.
>
> So probably a `DeregisterEvent()` call from the shutdown process came 
> in, invalidating a callback entry just while the event task was 
> processing that callback list.
>
> Solving this may need some additional thoughts & measures: locking the 
> whole event registry with handlers lagging like this could lead to a 
> deadlock… probably at least the Duktape component needs to cleanly 
> abort/reject all pending events before/while deregistering.
>
> Michael, if you'd like to have a look at this, go ahead.
>
> Regards,
> Michael
>
>
> Am 01.01.25 um 11:47 schrieb Tamás Kovács via OvmsDev:
>> Now i change the sd card to a smaller, and formatted to fat32 (16Gb), 
>> but on reboot is crashing.
>> Now connect a computer to the usb port, and create a log in verbose, 
>> attached to the email.
>>
>>
>> -- 
>> Üdvözlettel:
>> Kovács Tamás
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
> -- 
> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
> Fon 02330 9104094 * Handy 0176 20698926
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
Fon 02330 9104094 * Handy 0176 20698926

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20250106/066a128a/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: events-patch-v1.diff
Type: text/x-patch
Size: 9503 bytes
Desc: not available
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20250106/066a128a/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20250106/066a128a/attachment-0001.sig>