<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
From my logs, the UART overflows mostly happened with NMEA
sentences, so maybe you need to enable GPS to see them. But they are
rare now.<br>
<br>
We're subscribing to sentences 2 + 64 (RMC + GNS), which happen to
add up to more than 128 bytes and are sent together once per second.<br>
<br>
CAN2 + 3 handling still is done on core 1. I don't know if the
optimization of that ISR already has full effect, as the underlying
GPIO ISR isn't allocated with ESP_INTR_FLAG_IRAM. The esp-idf main
GPIO ISR is in IRAM, but I think all registered sub handlers need to
be as well before we can do that.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 01.11.19 um 09:43 schrieb Mark
Webb-Johnson:<br>
</div>
<blockquote type="cite"
cite="mid:B47A4140-FA4A-4644-9283-A08B6FB6282B@webb-johnson.net">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class=""><br class="">
</div>
Initial feedback from me is that this looks good (both for this
one and for the subsequent CAN one). Great work!
<div class=""><br class="">
</div>
<div class="">My remote access via modem seems more ’snappy’ now.
I get solid green LIVE on the App quicker, and response to
messages is better. Perhaps there was some TCP/IP packet retries
I had not been aware of before, or just the async is less
delayed. On the async simcom port, I have never seen fifo
overflows.</div>
<div class=""><br class="">
</div>
<div class="">For the past week or two, CAN2 on my car has been
locking up on a daily basis, and required a ‘can can2 stop’,
‘can can2 start active 500000’ to recover. I’ve been running the
new firmware all day, and it seems stable.<br class="">
<div><br class="">
</div>
<div>I will try to stress it over the weekend, on my bench, to
see how it copes. The can logging to savvycan is fairly high
stress (with all 3 can buses active, a lot of wifi, and a
bunch of translation code running on the cpus).</div>
<div><br class="">
</div>
<div>Regards, Mark.</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 1 Nov 2019, at 5:19 AM, Michael Balzer <<a
href="mailto:dexter@expeedo.de" class=""
moz-do-not-send="true">dexter@expeedo.de</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8" class="">
<div class=""> Everyone,<br class="">
<br class="">
I've pushed another optimization for the SIMCOM
component that reduces the UART hardware<br class="">
FIFO overflows
(<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/274</a>)<br
class="">
to nearly zero by changing core associations.<br
class="">
<br class="">
TL;DR:<br class="">
<ul class="">
<li class="">you also should apply the sdkconfig
changes from the commit</li>
<li class="">we need to optimize interrupt latency</li>
</ul>
<br class="">
Analyzing our current core usage I found we've been
running nearly all interrupt handlers on<br class="">
core #1. That's because interrupts are bound to the core
that allocates them, and after system<br class="">
init, the housekeeper created all peripheral components
on core #1.<br class="">
<br class="">
This has been our interrupt allocation up to now:<br
class="">
<br class="">
<tt class="">Task Run# Lvl ISR#
Usage Source</tt><tt class=""><br
class="">
</tt><tt class="">----------------------------------------------------------------------------------------</tt><tt
class=""><br class="">
</tt><tt class="">- 0 1 0 RTC
Core ETS_RTC_CORE_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">esp_timer 0 1 0 RTOS
esp_timer ETS_TIMER2_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">ipc0 0 1 0 RTOS
Scheduler Core0 ETS_FROM_CPU_INTR0_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">ipc1 1 1 1 RTOS
Scheduler Core1 ETS_FROM_CPU_INTR1_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">main 0 1 0 RTOS
Watchdog ETS_TG0_WDT_LEVEL_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">OVMS Events 1 1 1
Peripherals (GPIO ISR) ETS_GPIO_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">OVMS Events 1 1 1
MAX7317 (spi_nodma) ETS_SPI3_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">OVMS Events 1 1 1 ESP32
CAN ETS_CAN_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">OVMS Events 1 1 1 SIMCOM
UART ETS_UART1_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">OVMS Events 1 1 1 USB
console (UART) ETS_UART0_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">OVMS Events 1 1 1 SD
card ETS_SDIO_HOST_INTR_SOURCE</tt><tt
class=""><br class="">
</tt><tt class="">----------------------------------------------------------------------------------------</tt><br
class="">
<br class="">
I had some advances on the FIFO overflows by allocating
a level 3 interrupt for the SIMCOM UART,<br class="">
but not much. I then tried moving the SIMCOM interrupt
handler to core #0, and that immediately<br class="">
reduced the overflows drastically -- most of the time.<br
class="">
<br class="">
I read through the Espressif forums and issues on
interrupt latency optimization and found a<br class="">
precious hint here: <a class="moz-txt-link-freetext"
href="https://www.esp32.com/viewtopic.php?t=11462#p46437"
moz-do-not-send="true">https://www.esp32.com/viewtopic.php?t=11462#p46437</a><br
class="">
<br class="">
<i class="">"Finally, by default the LWIP task is
allowed to run on either core, and will sometimes take
a</i><i class=""><br class="">
</i><i class="">spinlock which requires disabling
interrupts. Try pinning this task to CPU 0, only"</i><br
class="">
<br class="">
I did so, and the FIFO overflows disappeared completely
on my bench. In my car there still<br class="">
are some, but very few. I also tested pinning LWIP to
core #1, that also worked and may be<br class="">
an alternative.<br class="">
<br class="">
I then searched for more config candidates affecting
interrupt performance and also disabled<br class="">
the interrupt backtrace feature. That saves just 4
instructions on interrupt entry, so has<br class="">
little effect and can be re-enabled in case we need it.<br
class="">
<br class="">
I've reduced the UART IRQ level to 2 again as a support
for my second finding regarding<br class="">
the CAN controller, will describe that in the next post.<br
class="">
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<pre class="moz-signature" cols="160">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
</pre>
</div>
_______________________________________________<br
class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class=""
moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br
class="">
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br
class="">
</div>
</blockquote>
</div>
<br class="">
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="160">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
</pre>
</body>
</html>