[Ovmsdev] Duktape heap memory management

[Stephen Casner]

Michael,

Thanks for performing this diagnosis and enhancement.

Has anyone analyzed the work required to move to current esp-idf?  How
many things will break?  I imagine the heap instrumentation I added
would be one of them.  Perhaps we could divide up that work among
several developers.  I would likely need to get another module,
though, because my current v3.0 bench unit without SPIRAM probably
wouldn't be usable.

                                                        -- Steve

On Sat, 4 Sep 2021, Michael Balzer wrote:

> Everyone,
>
> the poor JSON encoding performance reminded me of another issue we've had with
> Duktape since integration: the overall slowly degrading performance of the
> whole system over time.
>
> I did some new analysis on this effect using the task monitoring, which showed
> it's become worse lately up to the point where Duktape needs more than a
> second to perform the garbage collection after just 24 hours of uptime. It
> turned out this effect is also one of the major causes for the very bad JSON
> encoding performance, i.e. needing up to 10 seconds for a simple object of 5 x
> 1440 values, and eating 15-20% of CPU time on a permanent base (instead of the
> tolerable 5-6% right after boot).
>
> Doing a "script reload" would help, but only temporarily and not near the
> point of a freshly booted system.
>
> I didn't find anything about such an effect for Duktape in general, so I
> turned to examining possible ESP32 specific causes, and finally found the
> culprit: the esp-idf memory management, or more precisely, it's issues with
> fragmentation in SPIRAM. Duktape is probably one of the main fragmentation
> drivers due to the nature of Javascript memory management (garbage
> collection).
>
> There has been some discussion on this by neoniousTR, who also was involved in
> the SPIRAM bug hunt: https://www.esp32.com/viewtopic.php?t=8628
>
> The dlmalloc adaptation of neoniousTR unfortunately isn't usable for us, as it
> exceeds our available IRAM. It also was based on a much earlier esp-idf
> release, and doesn't support the extensions and options we use.
>
> Espressif have introduced a new memory manager to address these issues, but
> only in esp-idf release 4.3. We'll need to go there one day, but that's
> nothing we can do short term:
> https://github.com/espressif/esp-idf/releases/tag/v4.3 → "Heap: Switched heap
> algorithm to one based on TLSF, improves performance especially when using a
> high number of allocations in PSRAM"
>
> So I've tried another option: I've removed the Duktape heap from the standard
> system memory management. I've added a separate umm_malloc instance just for
> the heap, which can work very fast because it doesn't need to take care of
> locking or poisoning.
>
> On Duktape startup, a fixed amount of SPIRAM gets allocated for the Javascript
> heap. The amount is 512 KB by default, which should be sufficient for most
> cases. I've added a configuration for this, config module duktape.heapsize,
> and a meminfo command to provide some insight.
>
> The results are:
>
>  * Overall Duktape speedup of at least factor 3, up to factor 8
>  * Garbage collection runs now need below 100 ms even with large
>    scripts running
>  * Duktape average CPU usage dropped to 1-2%
>  * Barely noticeable overall performance degradation after a week of
>    continuous operation
>
> Of course, your mileage may vary, feedback is welcome.
>
> It still degrades, which is kind of expected, as Duktape isn't the only system
> component doing dynamic memory allocations.
>
> I think esp-idf 4.3 may offer a significant overall performance increase for
> us, just by replacing the current memory management by a more sophisticated
> implementation.
>
> As a side effect, JSON encoding is now much faster (by up to factor 8). CBOR
> has become only about 30% faster, which is probably due to CBOR needing much
> less memory allocations. CBOR is still factor 3 faster on average than JSON,
> so still the preferred choice for storage & transmission.
>
> Details & code:
> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/1bcc8dd84405df81e61577a90909e7b43c25baa3
>
> Note: there are three new Kconfig variables that need to be set to enable the
> new Duktape memory management. The sdkconfig.default.hw31 includes the new
> configuration, if doing a "make", simply accept all defaults.
>
> Regards,
> Michael
>
> --
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26