[Ovmsdev] Duktape heap memory management

Sun Sep 5 04:39:31 HKT 2021

Craig, you're probably qualified best to answer this, as you already did 
some steps to port to idf 4.0?

https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360

But I guess there have been quite some changes since 4.0 also. We'll 
have to try and solve the issues one by one.

Probably best to get for-v3.3 including the SIM7600 support stable and 
merged into master first. Mark, I'll merge the heap change into for-v3.3 
next.

Regards,
Michael

Am 04.09.21 um 21:52 schrieb Stephen Casner:
> Michael,
>
> Thanks for performing this diagnosis and enhancement.
>
> Has anyone analyzed the work required to move to current esp-idf?  How
> many things will break?  I imagine the heap instrumentation I added
> would be one of them.  Perhaps we could divide up that work among
> several developers.  I would likely need to get another module,
> though, because my current v3.0 bench unit without SPIRAM probably
> wouldn't be usable.
>
>                                                          -- Steve
>
> On Sat, 4 Sep 2021, Michael Balzer wrote:
>
>> Everyone,
>>
>> the poor JSON encoding performance reminded me of another issue we've had with
>> Duktape since integration: the overall slowly degrading performance of the
>> whole system over time.
>>
>> I did some new analysis on this effect using the task monitoring, which showed
>> it's become worse lately up to the point where Duktape needs more than a
>> second to perform the garbage collection after just 24 hours of uptime. It
>> turned out this effect is also one of the major causes for the very bad JSON
>> encoding performance, i.e. needing up to 10 seconds for a simple object of 5 x
>> 1440 values, and eating 15-20% of CPU time on a permanent base (instead of the
>> tolerable 5-6% right after boot).
>>
>> Doing a "script reload" would help, but only temporarily and not near the
>> point of a freshly booted system.
>>
>> I didn't find anything about such an effect for Duktape in general, so I
>> turned to examining possible ESP32 specific causes, and finally found the
>> culprit: the esp-idf memory management, or more precisely, it's issues with
>> fragmentation in SPIRAM. Duktape is probably one of the main fragmentation
>> drivers due to the nature of Javascript memory management (garbage
>> collection).
>>
>> There has been some discussion on this by neoniousTR, who also was involved in
>> the SPIRAM bug hunt: https://www.esp32.com/viewtopic.php?t=8628
>>
>> The dlmalloc adaptation of neoniousTR unfortunately isn't usable for us, as it
>> exceeds our available IRAM. It also was based on a much earlier esp-idf
>> release, and doesn't support the extensions and options we use.
>>
>> Espressif have introduced a new memory manager to address these issues, but
>> only in esp-idf release 4.3. We'll need to go there one day, but that's
>> nothing we can do short term:
>> https://github.com/espressif/esp-idf/releases/tag/v4.3 → "Heap: Switched heap
>> algorithm to one based on TLSF, improves performance especially when using a
>> high number of allocations in PSRAM"
>>
>> So I've tried another option: I've removed the Duktape heap from the standard
>> system memory management. I've added a separate umm_malloc instance just for
>> the heap, which can work very fast because it doesn't need to take care of
>> locking or poisoning.
>>
>> On Duktape startup, a fixed amount of SPIRAM gets allocated for the Javascript
>> heap. The amount is 512 KB by default, which should be sufficient for most
>> cases. I've added a configuration for this, config module duktape.heapsize,
>> and a meminfo command to provide some insight.
>>
>> The results are:
>>
>>   * Overall Duktape speedup of at least factor 3, up to factor 8
>>   * Garbage collection runs now need below 100 ms even with large
>>     scripts running
>>   * Duktape average CPU usage dropped to 1-2%
>>   * Barely noticeable overall performance degradation after a week of
>>     continuous operation
>>
>> Of course, your mileage may vary, feedback is welcome.
>>
>> It still degrades, which is kind of expected, as Duktape isn't the only system
>> component doing dynamic memory allocations.
>>
>> I think esp-idf 4.3 may offer a significant overall performance increase for
>> us, just by replacing the current memory management by a more sophisticated
>> implementation.
>>
>> As a side effect, JSON encoding is now much faster (by up to factor 8). CBOR
>> has become only about 30% faster, which is probably due to CBOR needing much
>> less memory allocations. CBOR is still factor 3 faster on average than JSON,
>> so still the preferred choice for storage & transmission.
>>
>> Details & code:
>> https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/1bcc8dd84405df81e61577a90909e7b43c25baa3
>>
>> Note: there are three new Kconfig variables that need to be set to enable the
>> new Duktape memory management. The sdkconfig.default.hw31 includes the new
>> configuration, if doing a "make", simply accept all defaults.
>>
>> Regards,
>> Michael
>>
>> --
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20210904/cece225f/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20210904/cece225f/attachment-0001.sig>