[Ovmsdev] SIMCOM UART interrupt issue & optimization

Michael Balzer dexter at expeedo.de
Fri Nov 1 17:48:15 HKT 2019


From my logs, the UART overflows mostly happened with NMEA sentences, so maybe you need to enable GPS to see them. But they are rare now.

We're subscribing to sentences 2 + 64 (RMC + GNS), which happen to add up to more than 128 bytes and are sent together once per second.

CAN2 + 3 handling still is done on core 1. I don't know if the optimization of that ISR already has full effect, as the underlying GPIO ISR isn't allocated with
ESP_INTR_FLAG_IRAM. The esp-idf main GPIO ISR is in IRAM, but I think all registered sub handlers need to be as well before we can do that.

Regards,
Michael


Am 01.11.19 um 09:43 schrieb Mark Webb-Johnson:
>
> Initial feedback from me is that this looks good (both for this one and for the subsequent CAN one). Great work!
>
> My remote access via modem seems more ’snappy’ now. I get solid green LIVE on the App quicker, and response to messages is better. Perhaps there was some
> TCP/IP packet retries I had not been aware of before, or just the async is less delayed. On the async simcom port, I have never seen fifo overflows.
>
> For the past week or two, CAN2 on my car has been locking up on a daily basis, and required a ‘can can2 stop’, ‘can can2 start active 500000’ to recover. I’ve
> been running the new firmware all day, and it seems stable.
>
> I will try to stress it over the weekend, on my bench, to see how it copes. The can logging to savvycan is fairly high stress (with all 3 can buses active, a
> lot of wifi, and a bunch of translation code running on the cpus).
>
> Regards, Mark.
>
>> On 1 Nov 2019, at 5:19 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>
>> Everyone,
>>
>> I've pushed another optimization for the SIMCOM component that reduces the UART hardware
>> FIFO overflows (https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274)
>> to nearly zero by changing core associations.
>>
>> TL;DR:
>>
>>   * you also should apply the sdkconfig changes from the commit
>>   * we need to optimize interrupt latency
>>
>>
>> Analyzing our current core usage I found we've been running nearly all interrupt handlers on
>> core #1. That's because interrupts are bound to the core that allocates them, and after system
>> init, the housekeeper created all peripheral components on core #1.
>>
>> This has been our interrupt allocation up to now:
>>
>> Task           Run#  Lvl  ISR#  Usage                    Source
>> ----------------------------------------------------------------------------------------
>> -              0     1    0     RTC Core                 ETS_RTC_CORE_INTR_SOURCE
>> esp_timer      0     1    0     RTOS esp_timer           ETS_TIMER2_INTR_SOURCE
>> ipc0           0     1    0     RTOS Scheduler Core0     ETS_FROM_CPU_INTR0_SOURCE
>> ipc1           1     1    1     RTOS Scheduler Core1     ETS_FROM_CPU_INTR1_SOURCE
>> main           0     1    0     RTOS Watchdog            ETS_TG0_WDT_LEVEL_INTR_SOURCE
>> OVMS Events    1     1    1     Peripherals (GPIO ISR)   ETS_GPIO_INTR_SOURCE
>> OVMS Events    1     1    1     MAX7317 (spi_nodma)      ETS_SPI3_INTR_SOURCE
>> OVMS Events    1     1    1     ESP32 CAN                ETS_CAN_INTR_SOURCE
>> OVMS Events    1     1    1     SIMCOM UART              ETS_UART1_INTR_SOURCE
>> OVMS Events    1     1    1     USB console (UART)       ETS_UART0_INTR_SOURCE
>> OVMS Events    1     1    1     SD card                  ETS_SDIO_HOST_INTR_SOURCE
>> ----------------------------------------------------------------------------------------
>>
>> I had some advances on the FIFO overflows by allocating a level 3 interrupt for the SIMCOM UART,
>> but not much. I then tried moving the SIMCOM interrupt handler to core #0, and that immediately
>> reduced the overflows drastically -- most of the time.
>>
>> I read through the Espressif forums and issues on interrupt latency optimization and found a
>> precious hint here: https://www.esp32.com/viewtopic.php?t=11462#p46437
>>
>> /"Finally, by default the LWIP task is allowed to run on either core, and will sometimes take a//
>> //spinlock which requires disabling interrupts. Try pinning this task to CPU 0, only"/
>>
>> I did so, and the FIFO overflows disappeared completely on my bench. In my car there still
>> are some, but very few. I also tested pinning LWIP to core #1, that also worked and may be
>> an alternative.
>>
>> I then searched for more config candidates affecting interrupt performance and also disabled
>> the interrupt backtrace feature. That saves just 4 instructions on interrupt entry, so has
>> little effect and can be re-enabled in case we need it.
>>
>> I've reduced the UART IRQ level to 2 again as a support for my second finding regarding
>> the CAN controller, will describe that in the next post.
>>
>> Regards,
>> Michael
>>
>> -- 
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20191101/43eb7486/attachment.htm>


More information about the OvmsDev mailing list