[Ovmsdev] SIMCOM UART interrupt issue & optimization

Mark Webb-Johnson mark at webb-johnson.net
Fri Nov 1 16:43:52 HKT 2019


Initial feedback from me is that this looks good (both for this one and for the subsequent CAN one). Great work!

My remote access via modem seems more ’snappy’ now. I get solid green LIVE on the App quicker, and response to messages is better. Perhaps there was some TCP/IP packet retries I had not been aware of before, or just the async is less delayed. On the async simcom port, I have never seen fifo overflows.

For the past week or two, CAN2 on my car has been locking up on a daily basis, and required a ‘can can2 stop’, ‘can can2 start active 500000’ to recover. I’ve been running the new firmware all day, and it seems stable.

I will try to stress it over the weekend, on my bench, to see how it copes. The can logging to savvycan is fairly high stress (with all 3 can buses active, a lot of wifi, and a bunch of translation code running on the cpus).

Regards, Mark.

> On 1 Nov 2019, at 5:19 AM, Michael Balzer <dexter at expeedo.de> wrote:
> 
> Everyone,
> 
> I've pushed another optimization for the SIMCOM component that reduces the UART hardware
> FIFO overflows (https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274>)
> to nearly zero by changing core associations.
> 
> TL;DR:
> you also should apply the sdkconfig changes from the commit
> we need to optimize interrupt latency
> 
> Analyzing our current core usage I found we've been running nearly all interrupt handlers on
> core #1. That's because interrupts are bound to the core that allocates them, and after system
> init, the housekeeper created all peripheral components on core #1.
> 
> This has been our interrupt allocation up to now:
> 
> Task           Run#  Lvl  ISR#  Usage                    Source
> ----------------------------------------------------------------------------------------
> -              0     1    0     RTC Core                 ETS_RTC_CORE_INTR_SOURCE
> esp_timer      0     1    0     RTOS esp_timer           ETS_TIMER2_INTR_SOURCE
> ipc0           0     1    0     RTOS Scheduler Core0     ETS_FROM_CPU_INTR0_SOURCE
> ipc1           1     1    1     RTOS Scheduler Core1     ETS_FROM_CPU_INTR1_SOURCE
> main           0     1    0     RTOS Watchdog            ETS_TG0_WDT_LEVEL_INTR_SOURCE
> OVMS Events    1     1    1     Peripherals (GPIO ISR)   ETS_GPIO_INTR_SOURCE
> OVMS Events    1     1    1     MAX7317 (spi_nodma)      ETS_SPI3_INTR_SOURCE
> OVMS Events    1     1    1     ESP32 CAN                ETS_CAN_INTR_SOURCE
> OVMS Events    1     1    1     SIMCOM UART              ETS_UART1_INTR_SOURCE
> OVMS Events    1     1    1     USB console (UART)       ETS_UART0_INTR_SOURCE
> OVMS Events    1     1    1     SD card                  ETS_SDIO_HOST_INTR_SOURCE
> ----------------------------------------------------------------------------------------
> 
> I had some advances on the FIFO overflows by allocating a level 3 interrupt for the SIMCOM UART,
> but not much. I then tried moving the SIMCOM interrupt handler to core #0, and that immediately
> reduced the overflows drastically -- most of the time.
> 
> I read through the Espressif forums and issues on interrupt latency optimization and found a
> precious hint here: https://www.esp32.com/viewtopic.php?t=11462#p46437 <https://www.esp32.com/viewtopic.php?t=11462#p46437>
> 
> "Finally, by default the LWIP task is allowed to run on either core, and will sometimes take a
> spinlock which requires disabling interrupts. Try pinning this task to CPU 0, only"
> 
> I did so, and the FIFO overflows disappeared completely on my bench. In my car there still
> are some, but very few. I also tested pinning LWIP to core #1, that also worked and may be
> an alternative.
> 
> I then searched for more config candidates affecting interrupt performance and also disabled
> the interrupt backtrace feature. That saves just 4 instructions on interrupt entry, so has
> little effect and can be re-enabled in case we need it.
> 
> I've reduced the UART IRQ level to 2 again as a support for my second finding regarding
> the CAN controller, will describe that in the next post.
> 
> Regards,
> Michael
> 
> -- 
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20191101/7c09c29d/attachment.html>


More information about the OvmsDev mailing list