Initial feedback from me is that this looks good (both for this one and for the subsequent CAN one). Great work! My remote access via modem seems more ’snappy’ now. I get solid green LIVE on the App quicker, and response to messages is better. Perhaps there was some TCP/IP packet retries I had not been aware of before, or just the async is less delayed. On the async simcom port, I have never seen fifo overflows. For the past week or two, CAN2 on my car has been locking up on a daily basis, and required a ‘can can2 stop’, ‘can can2 start active 500000’ to recover. I’ve been running the new firmware all day, and it seems stable. I will try to stress it over the weekend, on my bench, to see how it copes. The can logging to savvycan is fairly high stress (with all 3 can buses active, a lot of wifi, and a bunch of translation code running on the cpus). Regards, Mark.
On 1 Nov 2019, at 5:19 AM, Michael Balzer <dexter@expeedo.de> wrote:
Everyone,
I've pushed another optimization for the SIMCOM component that reduces the UART hardware FIFO overflows (https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274 <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274>) to nearly zero by changing core associations.
TL;DR: you also should apply the sdkconfig changes from the commit we need to optimize interrupt latency
Analyzing our current core usage I found we've been running nearly all interrupt handlers on core #1. That's because interrupts are bound to the core that allocates them, and after system init, the housekeeper created all peripheral components on core #1.
This has been our interrupt allocation up to now:
Task Run# Lvl ISR# Usage Source ---------------------------------------------------------------------------------------- - 0 1 0 RTC Core ETS_RTC_CORE_INTR_SOURCE esp_timer 0 1 0 RTOS esp_timer ETS_TIMER2_INTR_SOURCE ipc0 0 1 0 RTOS Scheduler Core0 ETS_FROM_CPU_INTR0_SOURCE ipc1 1 1 1 RTOS Scheduler Core1 ETS_FROM_CPU_INTR1_SOURCE main 0 1 0 RTOS Watchdog ETS_TG0_WDT_LEVEL_INTR_SOURCE OVMS Events 1 1 1 Peripherals (GPIO ISR) ETS_GPIO_INTR_SOURCE OVMS Events 1 1 1 MAX7317 (spi_nodma) ETS_SPI3_INTR_SOURCE OVMS Events 1 1 1 ESP32 CAN ETS_CAN_INTR_SOURCE OVMS Events 1 1 1 SIMCOM UART ETS_UART1_INTR_SOURCE OVMS Events 1 1 1 USB console (UART) ETS_UART0_INTR_SOURCE OVMS Events 1 1 1 SD card ETS_SDIO_HOST_INTR_SOURCE ----------------------------------------------------------------------------------------
I had some advances on the FIFO overflows by allocating a level 3 interrupt for the SIMCOM UART, but not much. I then tried moving the SIMCOM interrupt handler to core #0, and that immediately reduced the overflows drastically -- most of the time.
I read through the Espressif forums and issues on interrupt latency optimization and found a precious hint here: https://www.esp32.com/viewtopic.php?t=11462#p46437 <https://www.esp32.com/viewtopic.php?t=11462#p46437>
"Finally, by default the LWIP task is allowed to run on either core, and will sometimes take a spinlock which requires disabling interrupts. Try pinning this task to CPU 0, only"
I did so, and the FIFO overflows disappeared completely on my bench. In my car there still are some, but very few. I also tested pinning LWIP to core #1, that also worked and may be an alternative.
I then searched for more config candidates affecting interrupt performance and also disabled the interrupt backtrace feature. That saves just 4 instructions on interrupt entry, so has little effect and can be re-enabled in case we need it.
I've reduced the UART IRQ level to 2 again as a support for my second finding regarding the CAN controller, will describe that in the next post.
Regards, Michael
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev