[Ovmsdev] SIMCOM UART interrupt issue & optimization

[Mark Webb-Johnson]

Initial feedback from me is that this looks good (both for this one and for the subsequent CAN one). Great work!

My remote access via modem seems more ’snappy’ now. I get solid green LIVE on the App quicker, and response to messages is better. Perhaps there was some TCP/IP packet retries I had not been aware of before, or just the async is less delayed. On the async simcom port, I have never seen fifo overflows.

For the past week or two, CAN2 on my car has been locking up on a daily basis, and required a ‘can can2 stop’, ‘can can2 start active 500000’ to recover. I’ve been running the new firmware all day, and it seems stable.

I will try to stress it over the weekend, on my bench, to see how it copes. The can logging to savvycan is fairly high stress (with all 3 can buses active, a lot of wifi, and a bunch of translation code running on the cpus).

Regards, Mark.

> On 1 Nov 2019, at 5:19 AM, Michael Balzer <dexter at expeedo.de> wrote:
> 
> Everyone,
> 
> I've pushed another optimization for the SIMCOM component that reduces the UART hardware
> FIFO overflows (https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274>)
> to nearly zero by changing core associations.
> 
> TL;DR:
> you also should apply the sdkconfig changes from the commit
> we need to optimize interrupt latency
> 
> Analyzing our current core usage I found we've been running nearly all interrupt handlers on
> core #1. That's because interrupts are bound to the core that allocates them, and after system
> init, the housekeeper created all peripheral components on core #1.
> 
> This has been our interrupt allocation up to now:
> 
> Task           Run#  Lvl  ISR#  Usage                    Source
> ----------------------------------------------------------------------------------------
> -              0     1    0     RTC Core                 ETS_RTC_CORE_INTR_SOURCE
> esp_timer      0     1    0     RTOS esp_timer           ETS_TIMER2_INTR_SOURCE
> ipc0           0     1    0     RTOS Scheduler Core0     ETS_FROM_CPU_INTR0_SOURCE
> ipc1           1     1    1     RTOS Scheduler Core1     ETS_FROM_CPU_INTR1_SOURCE
> main           0     1    0     RTOS Watchdog            ETS_TG0_WDT_LEVEL_INTR_SOURCE
> OVMS Events    1     1    1     Peripherals (GPIO ISR)   ETS_GPIO_INTR_SOURCE
> OVMS Events    1     1    1     MAX7317 (spi_nodma)      ETS_SPI3_INTR_SOURCE
> OVMS Events    1     1    1     ESP32 CAN                ETS_CAN_INTR_SOURCE
> OVMS Events    1     1    1     SIMCOM UART              ETS_UART1_INTR_SOURCE
> OVMS Events    1     1    1     USB console (UART)       ETS_UART0_INTR_SOURCE
> OVMS Events    1     1    1     SD card                  ETS_SDIO_HOST_INTR_SOURCE
> ----------------------------------------------------------------------------------------
> 
> I had some advances on the FIFO overflows by allocating a level 3 interrupt for the SIMCOM UART,
> but not much. I then tried moving the SIMCOM interrupt handler to core #0, and that immediately
> reduced the overflows drastically -- most of the time.
> 
> I read through the Espressif forums and issues on interrupt latency optimization and found a
> precious hint here: https://www.esp32.com/viewtopic.php?t=11462#p46437 <https://www.esp32.com/viewtopic.php?t=11462#p46437>
> 
> "Finally, by default the LWIP task is allowed to run on either core, and will sometimes take a
> spinlock which requires disabling interrupts. Try pinning this task to CPU 0, only"
> 
> I did so, and the FIFO overflows disappeared completely on my bench. In my car there still
> are some, but very few. I also tested pinning LWIP to core #1, that also worked and may be
> an alternative.
> 
> I then searched for more config candidates affecting interrupt performance and also disabled
> the interrupt backtrace feature. That saves just 4 instructions on interrupt entry, so has
> little effect and can be re-enabled in case we need it.
> 
> I've reduced the UART IRQ level to 2 again as a support for my second finding regarding
> the CAN controller, will describe that in the next post.
> 
> Regards,
> Michael
> 
> -- 
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20191101/7c09c29d/attachment.htm>