I see there are a couple of tasks that allocated memory but did not free it. These are the tasks identified by TCB address in the module memory listing: OVMS# mo m Free 8-bit 32968/283136, 32-bit 27588/55864, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5276 0 0 0 +5276 +0 +0 +0 esp_timer 55300 0 644 0 +55300 +0 +644 +0 main 40072 0 0 0 +40072 +0 +0 +0 ipc0 10848 0 0 0 +10848 +0 +0 +0 Housekeeping 17588 44112 0 0 +17588 +44112 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 tiT 136 1676 0 0 +136 +1676 +0 +0 NetManTask 8 2196 0 0 +8 +2196 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 12960 0 0 +0 +12960 +0 +0 3FFE3E78 0 1396 0 0 +0 +1396 +0 +0 Tmr Svc 0 27808 0 0 +0 +27808 +0 +0 3FFEAFE4 0 88 0 0 +0 +88 +0 +0 AsyncConsole 0 20 27488 0 +0 +20 +27488 +0 OVMS# mo t Number of Tasks = 16 Stack: Now Max Total Heap 32-bit SPIRAM Task 3FFAFAB0 1 esp_timer 0 61770 65278 55300 644 0 Task 3FFBDFC4 2 eventTask 0 63886 65278 12960 0 0 Task 3FFC6630 3 CanRxTask 0 61610 65278 0 0 0 Task 3FFCDA54 4 ipc0 0 64702 65278 10848 0 0 Task 3FFCE054 5 ipc1 0 64702 65278 12 0 0 Task 3FFCFE7C 8 IDLE 0 64742 65278 0 0 0 Task 3FFD0410 9 IDLE 0 64746 65278 0 0 0 Task 3FFD1DA4 10 Tmr Svc 0 61678 65278 27808 0 0 Task 3FFD9DC8 14 Housekeeping 0 62478 65278 61700 0 0 Task 3FFCE2C0 16 tiT 0 63506 65278 1812 0 0 Task 3FFE327C 17 SIMCOMTask 0 61638 65278 0 0 0 Task 3FFF0654 22 AsyncConsole 0 62510 65278 20 27488 0 Task 3FFE8464 25 wifi 0 63474 65278 1600 0 0 Task 3FFF1828 26 pmT 0 64398 65278 0 0 0 Task 3FFE82CC 29 NetManTask 0 60254 65278 2204 0 0 Task 3FFEC408 30 mdns 0 62906 65278 0 0 0 OVMS# -- Steve
These are tasks that were started and then exited (leaving memory unfreed)? Regards, Mark.
On 19 Mar 2018, at 1:37 PM, Stephen Casner <casner@acm.org> wrote:
I see there are a couple of tasks that allocated memory but did not free it. These are the tasks identified by TCB address in the module memory listing:
OVMS# mo m Free 8-bit 32968/283136, 32-bit 27588/55864, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5276 0 0 0 +5276 +0 +0 +0 esp_timer 55300 0 644 0 +55300 +0 +644 +0 main 40072 0 0 0 +40072 +0 +0 +0 ipc0 10848 0 0 0 +10848 +0 +0 +0 Housekeeping 17588 44112 0 0 +17588 +44112 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 tiT 136 1676 0 0 +136 +1676 +0 +0 NetManTask 8 2196 0 0 +8 +2196 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 12960 0 0 +0 +12960 +0 +0 3FFE3E78 0 1396 0 0 +0 +1396 +0 +0 Tmr Svc 0 27808 0 0 +0 +27808 +0 +0 3FFEAFE4 0 88 0 0 +0 +88 +0 +0 AsyncConsole 0 20 27488 0 +0 +20 +27488 +0 OVMS# mo t Number of Tasks = 16 Stack: Now Max Total Heap 32-bit SPIRAM Task 3FFAFAB0 1 esp_timer 0 61770 65278 55300 644 0 Task 3FFBDFC4 2 eventTask 0 63886 65278 12960 0 0 Task 3FFC6630 3 CanRxTask 0 61610 65278 0 0 0 Task 3FFCDA54 4 ipc0 0 64702 65278 10848 0 0 Task 3FFCE054 5 ipc1 0 64702 65278 12 0 0 Task 3FFCFE7C 8 IDLE 0 64742 65278 0 0 0 Task 3FFD0410 9 IDLE 0 64746 65278 0 0 0 Task 3FFD1DA4 10 Tmr Svc 0 61678 65278 27808 0 0 Task 3FFD9DC8 14 Housekeeping 0 62478 65278 61700 0 0 Task 3FFCE2C0 16 tiT 0 63506 65278 1812 0 0 Task 3FFE327C 17 SIMCOMTask 0 61638 65278 0 0 0 Task 3FFF0654 22 AsyncConsole 0 62510 65278 20 27488 0 Task 3FFE8464 25 wifi 0 63474 65278 1600 0 0 Task 3FFF1828 26 pmT 0 64398 65278 0 0 0 Task 3FFE82CC 29 NetManTask 0 60254 65278 2204 0 0 Task 3FFEC408 30 mdns 0 62906 65278 0 0 0 OVMS#
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Awake again now... Yes, the lines in 'module memory' output identified by a hex number are ones where the scan through allocated memory blocks finds blocks owned by a task with that TCB address but that uxTaskGetSystemState() does not include a task with that TCB address. The code keeps a map of TCB address to name so that it is possible to show by name the memory belonging to tasks that have exited, but that map only gets populated two ways: - When 'module memory' or 'module tasks' is run - When a task is created with the TaskBase class - By calling AddTaskToMap() after creating a task, as I added for the main and Housekeeping tasks. Michael has used TaskBase when creating a command task, so that case would be covered. There might be some places where we create other tasks not using TaskBase where a call to AddTaskToMap() could be added. But for tasks created on our behalf by system code, those won't be covered unless we can put the AddTaskToMap() call in our code that causes the task to be created. -- Steve On Mon, 19 Mar 2018, Mark Webb-Johnson wrote:
These are tasks that were started and then exited (leaving memory unfreed)?
Regards, Mark.
On 19 Mar 2018, at 1:37 PM, Stephen Casner <casner@acm.org> wrote:
I see there are a couple of tasks that allocated memory but did not free it. These are the tasks identified by TCB address in the module memory listing:
OVMS# mo m Free 8-bit 32968/283136, 32-bit 27588/55864, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5276 0 0 0 +5276 +0 +0 +0 esp_timer 55300 0 644 0 +55300 +0 +644 +0 main 40072 0 0 0 +40072 +0 +0 +0 ipc0 10848 0 0 0 +10848 +0 +0 +0 Housekeeping 17588 44112 0 0 +17588 +44112 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 tiT 136 1676 0 0 +136 +1676 +0 +0 NetManTask 8 2196 0 0 +8 +2196 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 12960 0 0 +0 +12960 +0 +0 3FFE3E78 0 1396 0 0 +0 +1396 +0 +0 Tmr Svc 0 27808 0 0 +0 +27808 +0 +0 3FFEAFE4 0 88 0 0 +0 +88 +0 +0 AsyncConsole 0 20 27488 0 +0 +20 +27488 +0 OVMS# mo t Number of Tasks = 16 Stack: Now Max Total Heap 32-bit SPIRAM Task 3FFAFAB0 1 esp_timer 0 61770 65278 55300 644 0 Task 3FFBDFC4 2 eventTask 0 63886 65278 12960 0 0 Task 3FFC6630 3 CanRxTask 0 61610 65278 0 0 0 Task 3FFCDA54 4 ipc0 0 64702 65278 10848 0 0 Task 3FFCE054 5 ipc1 0 64702 65278 12 0 0 Task 3FFCFE7C 8 IDLE 0 64742 65278 0 0 0 Task 3FFD0410 9 IDLE 0 64746 65278 0 0 0 Task 3FFD1DA4 10 Tmr Svc 0 61678 65278 27808 0 0 Task 3FFD9DC8 14 Housekeeping 0 62478 65278 61700 0 0 Task 3FFCE2C0 16 tiT 0 63506 65278 1812 0 0 Task 3FFE327C 17 SIMCOMTask 0 61638 65278 0 0 0 Task 3FFF0654 22 AsyncConsole 0 62510 65278 20 27488 0 Task 3FFE8464 25 wifi 0 63474 65278 1600 0 0 Task 3FFF1828 26 pmT 0 64398 65278 0 0 0 Task 3FFE82CC 29 NetManTask 0 60254 65278 2204 0 0 Task 3FFEC408 30 mdns 0 62906 65278 0 0 0 OVMS#
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Steve, I've seen some allocations looking like memory leaks but getting freed after a minute or two. I assume those are mongoose buffers or LWIP socket structs getting freed after some timeout. But if I remember right, those were accounted for on the TCP/IP task and/or the NetManTask, not on dynamic tasks. I start dynamic tasks primarily for web command execution in streaming mode. The "vfs tail" command also runs as a dynamic task. If you had the mDNS component still active, that could also be the cause. Regards, Michael Am 19.03.2018 um 06:37 schrieb Stephen Casner:
I see there are a couple of tasks that allocated memory but did not free it. These are the tasks identified by TCB address in the module memory listing:
OVMS# mo m Free 8-bit 32968/283136, 32-bit 27588/55864, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5276 0 0 0 +5276 +0 +0 +0 esp_timer 55300 0 644 0 +55300 +0 +644 +0 main 40072 0 0 0 +40072 +0 +0 +0 ipc0 10848 0 0 0 +10848 +0 +0 +0 Housekeeping 17588 44112 0 0 +17588 +44112 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 tiT 136 1676 0 0 +136 +1676 +0 +0 NetManTask 8 2196 0 0 +8 +2196 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 12960 0 0 +0 +12960 +0 +0 3FFE3E78 0 1396 0 0 +0 +1396 +0 +0 Tmr Svc 0 27808 0 0 +0 +27808 +0 +0 3FFEAFE4 0 88 0 0 +0 +88 +0 +0 AsyncConsole 0 20 27488 0 +0 +20 +27488 +0 OVMS# mo t Number of Tasks = 16 Stack: Now Max Total Heap 32-bit SPIRAM Task 3FFAFAB0 1 esp_timer 0 61770 65278 55300 644 0 Task 3FFBDFC4 2 eventTask 0 63886 65278 12960 0 0 Task 3FFC6630 3 CanRxTask 0 61610 65278 0 0 0 Task 3FFCDA54 4 ipc0 0 64702 65278 10848 0 0 Task 3FFCE054 5 ipc1 0 64702 65278 12 0 0 Task 3FFCFE7C 8 IDLE 0 64742 65278 0 0 0 Task 3FFD0410 9 IDLE 0 64746 65278 0 0 0 Task 3FFD1DA4 10 Tmr Svc 0 61678 65278 27808 0 0 Task 3FFD9DC8 14 Housekeeping 0 62478 65278 61700 0 0 Task 3FFCE2C0 16 tiT 0 63506 65278 1812 0 0 Task 3FFE327C 17 SIMCOMTask 0 61638 65278 0 0 0 Task 3FFF0654 22 AsyncConsole 0 62510 65278 20 27488 0 Task 3FFE8464 25 wifi 0 63474 65278 1600 0 0 Task 3FFF1828 26 pmT 0 64398 65278 0 0 0 Task 3FFE82CC 29 NetManTask 0 60254 65278 2204 0 0 Task 3FFEC408 30 mdns 0 62906 65278 0 0 0 OVMS#
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
On Mon, 19 Mar 2018, Michael Balzer wrote:
I've seen some allocations looking like memory leaks but getting freed after a minute or two. I assume those are mongoose buffers or LWIP socket structs getting freed after some timeout.
These hang around for at least 5 minutes. But some of these allocations are 88-byte blocks that could be from LWIP allocating mutexes for socket management. Those are explicitly never freed. The following comment precedes the allocation statement: /* one time init and never free */ LWIP appears to use a fresh socket structure each time until all of the sockets in the preconfigured array have been used. The default value is 10 sockets.
I start dynamic tasks primarily for web command execution in streaming mode. The "vfs tail" command also runs as a dynamic task.
That should be covered by the use of TaskBase, as I mentioned in the previous email.
If you had the mDNS component still active, that could also be the cause.
This was after mDNS was disabled. -- Steve
I did a little experiment. I added code in Housekeeping initialization to allocate and close a socket 10 times. That does add storage for 10 88-byte blocks to the Housekeeping task, but it does not change the memory allocated to the two no-longer-existing tasks: Free 8-bit 31148/283136, 32-bit 27588/55864, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM 3FFE5CDC 0 1396 0 0 +0 +1396 +0 +0 3FFEB408 0 264 0 0 +0 +264 +0 +0 This is with the mdns task back in operation and still existing, so those are not it. And I see in our codebase that we only create two tasks with an explict call to xTaskCreatePinnedToCore() rather than through TaskBase. One is Housekeeping, which already has a call to AddTaskToMap() after it. The second is NetManTask. Now I have I added a call to AddTaskToMap() after that one as well, and that lets me see that the task with 264 bytes allocated was an earlier instance of NetManTask that was killed. With that call in place, we see two instances of NetManTask in the module memory output: Free 8-bit 31308/283136, 32-bit 27588/55864, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5276 0 0 0 +0 +0 +0 +0 esp_timer 55112 0 644 0 +0 +0 +0 +0 main 39948 0 0 0 +0 +0 +0 +0 ipc0 10848 0 0 0 +0 +0 +0 +0 Housekeeping 17508 50496 0 0 +0 +0 +0 +0 tiT 148 1624 0 0 +0 +0 +0 +0 ipc1 12 0 0 0 +0 +0 +0 +0 wifi 36 1572 0 0 +0 +0 +0 +0 mdns 4 104 0 0 +0 +0 +0 +0 Tmr Svc 0 28240 0 0 +0 +0 +0 +0 3FFE5D08 0 1396 0 0 +0 +0 +0 +0 NetManTask 0 264 0 0 +0 +0 +0 +0 AsyncConsole 0 20 27488 0 +0 +0 +0 +0 NetManTask 0 2204 0 0 +0 +0 +0 +0 eventTask 0 7620 0 0 +0 +0 +0 +0 The task with 1396 bytes allocated may be something related to the first instance of NetManTask that also gets killed. -- Steve On Mon, 19 Mar 2018, Stephen Casner wrote:
On Mon, 19 Mar 2018, Michael Balzer wrote:
I've seen some allocations looking like memory leaks but getting freed after a minute or two. I assume those are mongoose buffers or LWIP socket structs getting freed after some timeout.
These hang around for at least 5 minutes. But some of these allocations are 88-byte blocks that could be from LWIP allocating mutexes for socket management. Those are explicitly never freed. The following comment precedes the allocation statement: /* one time init and never free */ LWIP appears to use a fresh socket structure each time until all of the sockets in the preconfigured array have been used. The default value is 10 sockets.
I start dynamic tasks primarily for web command execution in streaming mode. The "vfs tail" command also runs as a dynamic task.
That should be covered by the use of TaskBase, as I mentioned in the previous email.
If you had the mDNS component still active, that could also be the cause.
This was after mDNS was disabled.
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
participants (3)
-
Mark Webb-Johnson -
Michael Balzer -
Stephen Casner