We expect production OVMS to be based on release/v3.1 of esp-idf in order to incorporate the support for SPIRAM (PSRAM). I'd like to set a target for all develpers to move to the master branch of openvehicles/esp-idf that contains the finished version of my modifications to support the "module memory" command. At the same time I need to then merge the corresponding changes from my temporary branch in openvehicles/Open-Vehicle-Monitoring-System-3 to the master branch there since the API between the two has changed. In other words, we need a flag day. -- Steve
Better sooner than later. Go ahead. Regards, Michael Am 12.01.2018 um 21:44 schrieb Stephen Casner:
We expect production OVMS to be based on release/v3.1 of esp-idf in order to incorporate the support for SPIRAM (PSRAM).
I'd like to set a target for all develpers to move to the master branch of openvehicles/esp-idf that contains the finished version of my modifications to support the "module memory" command. At the same time I need to then merge the corresponding changes from my temporary branch in openvehicles/Open-Vehicle-Monitoring-System-3 to the master branch there since the API between the two has changed.
In other words, we need a flag day.
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
I think we are pretty much ready to go. I just need to make the sdkconfig.default for v3.0 and v3.1. So long as it works for v3.0 developer hardware, that is fine. @Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok? I guess steps for developers will be: 1) pull the openvehicles IDF and switch to master branch. Sub module update. 2) download and install xtensa build chain to match. 3) update OVMS master, make clean, check menu config, then build and play. I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then. Regards, Mark
On 13 Jan 2018, at 4:44 AM, Stephen Casner <casner@acm.org> wrote:
We expect production OVMS to be based on release/v3.1 of esp-idf in order to incorporate the support for SPIRAM (PSRAM).
I'd like to set a target for all develpers to move to the master branch of openvehicles/esp-idf that contains the finished version of my modifications to support the "module memory" command. At the same time I need to then merge the corresponding changes from my temporary branch in openvehicles/Open-Vehicle-Monitoring-System-3 to the master branch there since the API between the two has changed.
In other words, we need a flag day.
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok?
I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play.
Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then.
I'll be sleeping then, so you can go ahead and do the merge. -- Steve
Mark, It looks like you were not able to do the sdkconfig.default and merge as you planned? -- Steve On Fri, 12 Jan 2018, Stephen Casner wrote:
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok?
I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play.
Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then.
I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour. Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok?
I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play.
Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then.
I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
All committed and pushed now. Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported). Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0. I hope this goes smoothly for you. Regards, Mark. commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800 Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800 Merge branch 'for-master'. This requires ESP-IDF v3.0 support: OVMS developers should now: 1] Pull the latest OpenVehicles IDF, and checkout MASTER branch. 2] Update XTENSA tools to match version required by Espressif for MASTER branch. 3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate. 4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update. 5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported. Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok?
I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play.
Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then.
I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using. Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead. -- Steve On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok?
I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play.
Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then.
I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
I can confirm that I am able to mount and read the SD card now with the new esp-idf. -- Steve
On 15 Jan 2018, at 3:30 PM, Stephen Casner <casner@acm.org> wrote:
I can confirm that I am able to mount and read the SD card now with the new esp-idf.
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
I can confirm that (after an hour of messing around trying to remember how to upgrade ESP-IDF and XTENSA and all) I can mount, read and write to SD-card :-D Geir
15. jan. 2018 kl. 08:59 skrev Mark Webb-Johnson <mark@webb-johnson.net>:
<giphy.gif>
On 15 Jan 2018, at 3:30 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
I can confirm that I am able to mount and read the SD card now with the new esp-idf.
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know. And… OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5 → me too (the minion) :) BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM. After loading just the Twizy module & the v2 server, I'm now again down to… OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0 …so it crashes as soon as it needs to actually do anything, like sending a notification. This is without any of the new CAN logging stuff yet, that's stashed away for the transition. I'll now check the config again for any more options to save RAM… Regards, Michael Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok? I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play. Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then. I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
I've found the missing RAM. It's allocated for the SD card: OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0 I (34235) sdcard: SD CARD has been inserted. Auto-mounting... OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0 I (68065) sdcard: SD CARD has been removed. OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0 …again… OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0 I (114985) sdcard: SD CARD has been inserted. Auto-mounting... OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0 E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed. OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0 Btw, I think the RAM usage of the telnet server is also higher than before: OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0 OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 *AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0* NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0 …but I'm not sure about that, haven't tested that in a while. So, SD cards now work, and I'll not be able to use them due to the RAM usage… :( Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: > > @Steve can you update our master clone to latest from Espressif and > make sure your stuff is still ok? I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
> I guess steps for developers will be: > > 1) pull the openvehicles IDF and switch to master branch. Sub module update. > > 2) download and install xtensa build chain to match. > > 3) update OVMS master, make clean, check menu config, then build and play. Sounds right.
> I can get my part done by Sunday 14th night (HKT), so suggest to do > the Merge of the OVMS firmware master branch then. I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Greg, the SD card is good for all kinds of storage that needs to be done independant of the network connection. I.e. buffering of telemetry data, and especially logging high volume telemetry that cannot be transmitted via GSM. I especially wanted to log real time controller & motor performance data to the SD card. The SEVCON (like other controllers I guess) offers very detailed live information about the actual voltages, currents, frequencies, torque and slip levels of the AC motor, which can be used to optimize the power map. Using a GSM connection drops the achievable data rate for this telemetry to a nearly useless level. Regards, Michael Am 15.01.2018 um 20:40 schrieb Greg D.:
Just to ask (don't shoot me - I'm new here!)... Why do we need SD cards on the module? What is the end-user scenario that makes them essential?
Greg
Michael Balzer wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 *AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0* NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote: > > Mark, > > It looks like you were not able to do the sdkconfig.default and merge > as you planned? > > -- Steve > >> On Fri, 12 Jan 2018, Stephen Casner wrote: >> >>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>> >>> @Steve can you update our master clone to latest from Espressif and >>> make sure your stuff is still ok? >> I have just now done this. I had already done a rebase a few days ago >> before committing the improved version of the OS changes so I could >> issue a pull request. There were a few commits since then, which I >> have now merged. My code still runs correctly. >> >>> I guess steps for developers will be: >>> >>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>> >>> 2) download and install xtensa build chain to match. >>> >>> 3) update OVMS master, make clean, check menu config, then build and play. >> Sounds right. >> >>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>> the Merge of the OVMS firmware master branch then. >> I'll be sleeping then, so you can go ahead and do the merge. >> >> -- Steve > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Really, just storage; temporary storage space for log files, scripts, data captures, etc. Also useful for getting things in and out of the system. For example: put a firmware on the root of the SD CARD as ovms3.bin, insert it, and this is what happens: I (47547) sdcard: SD CARD has been inserted. Auto-mounting... W (47737) ota: AutoFlashSD Current running partition is: factory W (47737) ota: AutoFlashSD Target partition is: ota_0 W (47737) ota: AutoFlashSD Source image is 1582464 bytes in size W (47737) ota: AutoFlashSD Preparing flash partition... W (52587) ota: AutoFlashSD Flashing image partition... W (59977) ota: AutoFlashSD Setting boot partition... W (60937) ota: AutoFlashSD unmounting SD CARD W (60937) ota: AutoFlashSD OTA flash successful: Flashed 1582464 bytes, and booting from ‘ota_0' ets Jun 8 2016 00:22:57 rst:0xc (SW_CPU_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT) configsip: 156795334, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:4468 load:0x40078000,len:0 load:0x40078000,len:12988 entry 0x40078d14 I (173) ovms_main: Set default logging level for * to WARN Welcome to the Open Vehicle Monitoring System (OVMS) - Async Console OVMS > ota status Firmware: 3.0.0/ota_0/main build (idf v3.1-dev-217-g5bf85d06) Jan 16 2018 08:03:11 Running partition: ota_0 Boot partition: ota_0 OVMS > metrics list m.version m.version 3.0.0/ota_0/main build (idf v3.1-dev-217-g5bf85d06) Jan 16 2018 08:03:11 OVMS > vfs ls /sd ovms3.done That (a) detected a SD CARD was inserted, (b) found the /sd/ovms3.bin firmware file, (c) flashed that to a free ota partition, (d) set the ota partition as the one to boot from, and (e) rebooted into it. For users without wifi, that is probably how they are going to update firmware. P.S. For those of you testing this, to switch back to factory: OVMS > ota boot factory Boot from factory at 0x00010000 (size 0x00400000) OVMS > ota status Firmware: 3.0.0/ota_0/main build (idf v3.1-dev-217-g5bf85d06) Jan 16 2018 08:03:11 Running partition: ota_0 Boot partition: factory OVMS > module reset Resetting system... OVMS > ota status Firmware: 3.0.0/factory/main build (idf v3.1-dev-217-g5bf85d06) Jan 16 2018 08:03:11 Running partition: factory Boot partition: factory OVMS > metrics list m.version m.version 3.0.0/factory/main build (idf v3.1-dev-217-g5bf85d06) Jan 16 2018 08:03:11 This can all be done with simple SD CARDs. No complex driver installs, or firmware flashing software. Regards, Mark.
On 16 Jan 2018, at 3:40 AM, Greg D. <gregd2350@gmail.com> wrote:
Just to ask (don't shoot me - I'm new here!)... Why do we need SD cards on the module? What is the end-user scenario that makes them essential?
Greg
Michael Balzer wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0 NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> wrote: > > Mark, > > It looks like you were not able to do the sdkconfig.default and merge > as you planned? > > -- Steve > >> On Fri, 12 Jan 2018, Stephen Casner wrote: >> >>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>> >>> @Steve can you update our master clone to latest from Espressif and >>> make sure your stuff is still ok? >> I have just now done this. I had already done a rebase a few days ago >> before committing the improved version of the OS changes so I could >> issue a pull request. There were a few commits since then, which I >> have now merged. My code still runs correctly. >> >>> I guess steps for developers will be: >>> >>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>> >>> 2) download and install xtensa build chain to match. >>> >>> 3) update OVMS master, make clean, check menu config, then build and play. >> Sounds right. >> >>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>> the Merge of the OVMS firmware master branch then. >> I'll be sleeping then, so you can go ahead and do the merge. >> >> -- Steve > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
I’m seeing this: OVMS > module memory Free 8-bit 79812/282268, 32-bit 17816/44988, SPIRAM 0/0 OVMS > sd unmount Unmounted SD CARD OVMS > module memory Free 8-bit 107768/282268, 32-bit 17816/44988, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 20 0 0 -4 -27780 +0 +0 I guess it is FAT fs stuff, but we use that for /store as well so not sure why so much. I will look into it. With wifi client, ovms server v2, tesla roadster vehicle module, I get this: OVMS > module memory Free 8-bit 54224/282268, 32-bit 17816/44988, SPIRAM 0/0 With the OBDII HUD, that drops to 45KB. Do you have bluetooth enabled (in menuconfig)? Regards, Mark.
On 16 Jan 2018, at 3:35 AM, Michael Balzer <dexter@expeedo.de> wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0 NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
> On Fri, 12 Jan 2018, Stephen Casner wrote: > >> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >> >> @Steve can you update our master clone to latest from Espressif and >> make sure your stuff is still ok? > I have just now done this. I had already done a rebase a few days ago > before committing the improved version of the OS changes so I could > issue a pull request. There were a few commits since then, which I > have now merged. My code still runs correctly. > >> I guess steps for developers will be: >> >> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >> >> 2) download and install xtensa build chain to match. >> >> 3) update OVMS master, make clean, check menu config, then build and play. > Sounds right. > >> I can get my part done by Sunday 14th night (HKT), so suggest to do >> the Merge of the OVMS firmware master branch then. > I'll be sleeping then, so you can go ahead and do the merge. > > -- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Bluetooth is disabled. Has been for a while, I don't see a chance to enable it with the 3.0 module. I've also disabled Javascript support (duktape), Wolf, the web server and MDNS and the OBD2ECU. Telnet is enabled as I sometimes use that parallel to the async console. That config gives me… OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0 …after reboot (without SD card), and with Wifi, modem, Twizy & server v2 running & connected… OVMS > module memory Free 8-bit 23436/286352, 32-bit 17972/45144, SPIRAM 0/0 …when idle, will drop further with transmissions pending etc. Opening a telnet session then results in: OVMS > module memory Free 8-bit 18372/286352, 32-bit 17972/45144, SPIRAM 0/0 …and still stable. Regards, Michael Am 16.01.2018 um 01:11 schrieb Mark Webb-Johnson:
I’m seeing this:
OVMS > module memory Free 8-bit 79812/282268, 32-bit 17816/44988, SPIRAM 0/0
OVMS > sd unmount Unmounted SD CARD
OVMS > module memory Free 8-bit 107768/282268, 32-bit 17816/44988, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 20 0 0 -4 -27780 +0 +0
I guess it is FAT fs stuff, but we use that for /store as well so not sure why so much. I will look into it.
With wifi client, ovms server v2, tesla roadster vehicle module, I get this:
OVMS > module memory Free 8-bit 54224/282268, 32-bit 17816/44988, SPIRAM 0/0
With the OBDII HUD, that drops to 45KB.
Do you have bluetooth enabled (in menuconfig)?
Regards, Mark.
On 16 Jan 2018, at 3:35 AM, Michael Balzer <dexter@expeedo.de> wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0 NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> wrote: > > Mark, > > It looks like you were not able to do the sdkconfig.default and merge > as you planned? > > -- Steve > >> On Fri, 12 Jan 2018, Stephen Casner wrote: >> >>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>> >>> @Steve can you update our master clone to latest from Espressif and >>> make sure your stuff is still ok? >> I have just now done this. I had already done a rebase a few days ago >> before committing the improved version of the OS changes so I could >> issue a pull request. There were a few commits since then, which I >> have now merged. My code still runs correctly. >> >>> I guess steps for developers will be: >>> >>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>> >>> 2) download and install xtensa build chain to match. >>> >>> 3) update OVMS master, make clean, check menu config, then build and play. >> Sounds right. >> >>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>> the Merge of the OVMS firmware master branch then. >> I'll be sleeping then, so you can go ahead and do the merge. >> >> -- Steve > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Note also that the memory debugging option adds a bit of overhead. With debugging enabled (heap: light impact): OVMS > module memory Free 8-bit 79868/282496, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5312 0 0 0 +5312 +0 +0 +0 esp_timer 49568 0 644 0 +49568 +0 +644 +0 main 39444 0 0 0 +39444 +0 +0 +0 ipc0 11096 0 0 0 +11096 +0 +0 +0 Housekeeping 24740 19956 0 0 +24740 +19956 +0 +0 tiT 132 0 0 0 +132 +0 +0 +0 Tmr Svc 4 27800 0 0 +4 +27800 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 AsyncConsole 0 36 26404 0 +0 +36 +26404 +0 OVMS > metrics list m.freeram m.freeram 79868 With debugging enabled (heap: comprehensive): OVMS > module memory Free 8-bit 79728/282320, 32-bit 17968/45140, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5312 0 0 0 +5312 +0 +0 +0 esp_timer 49568 0 644 0 +49568 +0 +644 +0 main 39444 0 0 0 +39444 +0 +0 +0 ipc0 11096 0 0 0 +11096 +0 +0 +0 Housekeeping 24644 20056 0 0 +24644 +20056 +0 +0 tiT 132 0 0 0 +132 +0 +0 +0 Tmr Svc 4 27800 0 0 +4 +27800 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 AsyncConsole 0 20 26404 0 +0 +20 +26404 +0 I (15614) simcom: State timeout, transition to 13 I (15614) simcom: State: Enter PoweredOff state OVMS > metrics list m.freeram m.freeram 79728 With debugging disabled: OVMS > module memory To use these debugging tools, must set CONFIG_HEAP_TASK_TRACKING=y and have updated openvehicles/esp-idf OVMS > metrics list m.freeram m.freeram 99956 I think we’ve still got some stuff we can look at to lighten the impact on internal memory. Once we can get the 3.1 hardware out, I can focus on that. The memory tools Steve has built are really complete and usable now. They let us see exactly what is going on. If we had known going in that the 520KB SRAM of the ESP32 would be 80% used by the system, and only 20% for us, perhaps things would have been done differently. Anyway, we are where we are, and the WROVER module should be able to move a bunch of stuff off to PSRAM. Regards, Mark.
On 17 Jan 2018, at 12:59 AM, Michael Balzer <dexter@expeedo.de> wrote:
Bluetooth is disabled. Has been for a while, I don't see a chance to enable it with the 3.0 module.
I've also disabled Javascript support (duktape), Wolf, the web server and MDNS and the OBD2ECU. Telnet is enabled as I sometimes use that parallel to the async console.
That config gives me… OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0 …after reboot (without SD card), and with Wifi, modem, Twizy & server v2 running & connected…
OVMS > module memory Free 8-bit 23436/286352, 32-bit 17972/45144, SPIRAM 0/0
…when idle, will drop further with transmissions pending etc.
Opening a telnet session then results in:
OVMS > module memory Free 8-bit 18372/286352, 32-bit 17972/45144, SPIRAM 0/0
…and still stable.
Regards, Michael
Am 16.01.2018 um 01:11 schrieb Mark Webb-Johnson:
I’m seeing this:
OVMS > module memory Free 8-bit 79812/282268, 32-bit 17816/44988, SPIRAM 0/0
OVMS > sd unmount Unmounted SD CARD
OVMS > module memory Free 8-bit 107768/282268, 32-bit 17816/44988, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 20 0 0 -4 -27780 +0 +0
I guess it is FAT fs stuff, but we use that for /store as well so not sure why so much. I will look into it.
With wifi client, ovms server v2, tesla roadster vehicle module, I get this:
OVMS > module memory Free 8-bit 54224/282268, 32-bit 17816/44988, SPIRAM 0/0
With the OBDII HUD, that drops to 45KB.
Do you have bluetooth enabled (in menuconfig)?
Regards, Mark.
On 16 Jan 2018, at 3:35 AM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0 NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
> On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote: > > Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour. > > Regards, Mark > >> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> <mailto:casner@acm.org> <mailto:casner@acm.org> wrote: >> >> Mark, >> >> It looks like you were not able to do the sdkconfig.default and merge >> as you planned? >> >> -- Steve >> >>> On Fri, 12 Jan 2018, Stephen Casner wrote: >>> >>>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>>> >>>> @Steve can you update our master clone to latest from Espressif and >>>> make sure your stuff is still ok? >>> I have just now done this. I had already done a rebase a few days ago >>> before committing the improved version of the OS changes so I could >>> issue a pull request. There were a few commits since then, which I >>> have now merged. My code still runs correctly. >>> >>>> I guess steps for developers will be: >>>> >>>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>>> >>>> 2) download and install xtensa build chain to match. >>>> >>>> 3) update OVMS master, make clean, check menu config, then build and play. >>> Sounds right. >>> >>>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>>> the Merge of the OVMS firmware master branch then. >>> I'll be sleeping then, so you can go ahead and do the merge. >>> >>> -- Steve >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> >> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Michael: Can you try menuconfig Component / FAT Filesystem support / Use separate cache for each file -> OFF. Documentation shows that controlling the FS_TINY flag (which sounds like something we want). With wifi, DEMO vehicle module, sd mounted, and server v2 running, I get: Free 8-bit 70136/282488, 32-bit 17660/44832, SPIRAM 0/0 OVMS > sd unmount OVMS > module memory Free 8-bit 77604/282488, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 356 1088 0 0 +16 -20 +0 +0 Tmr Svc 20 528 0 0 -860 -6444 +0 +0 Using “separate cache for each file”, I get: Free 8-bit 29156/282496, 32-bit 17660/44832, SPIRAM 0/0 OVMS > sd unmount OVMS > module memory Free 8-bit 57096/282496, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 552 0 0 -4 -27780 +0 +0 Seems to be 27KB vs 7KB per filesystem mount, and we have two (/sd and /store). Overall win with FS_TINY for FAT is close to 41KB. Regards, Mark.
On 17 Jan 2018, at 9:23 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Note also that the memory debugging option adds a bit of overhead.
With debugging enabled (heap: light impact):
OVMS > module memory Free 8-bit 79868/282496, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5312 0 0 0 +5312 +0 +0 +0 esp_timer 49568 0 644 0 +49568 +0 +644 +0 main 39444 0 0 0 +39444 +0 +0 +0 ipc0 11096 0 0 0 +11096 +0 +0 +0 Housekeeping 24740 19956 0 0 +24740 +19956 +0 +0 tiT 132 0 0 0 +132 +0 +0 +0 Tmr Svc 4 27800 0 0 +4 +27800 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 AsyncConsole 0 36 26404 0 +0 +36 +26404 +0
OVMS > metrics list m.freeram m.freeram 79868
With debugging enabled (heap: comprehensive):
OVMS > module memory Free 8-bit 79728/282320, 32-bit 17968/45140, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5312 0 0 0 +5312 +0 +0 +0 esp_timer 49568 0 644 0 +49568 +0 +644 +0 main 39444 0 0 0 +39444 +0 +0 +0 ipc0 11096 0 0 0 +11096 +0 +0 +0 Housekeeping 24644 20056 0 0 +24644 +20056 +0 +0 tiT 132 0 0 0 +132 +0 +0 +0 Tmr Svc 4 27800 0 0 +4 +27800 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 AsyncConsole 0 20 26404 0 +0 +20 +26404 +0 I (15614) simcom: State timeout, transition to 13 I (15614) simcom: State: Enter PoweredOff state OVMS > metrics list m.freeram m.freeram 79728
With debugging disabled:
OVMS > module memory To use these debugging tools, must set CONFIG_HEAP_TASK_TRACKING=y and have updated openvehicles/esp-idf OVMS > metrics list m.freeram m.freeram 99956
I think we’ve still got some stuff we can look at to lighten the impact on internal memory. Once we can get the 3.1 hardware out, I can focus on that. The memory tools Steve has built are really complete and usable now. They let us see exactly what is going on.
If we had known going in that the 520KB SRAM of the ESP32 would be 80% used by the system, and only 20% for us, perhaps things would have been done differently. Anyway, we are where we are, and the WROVER module should be able to move a bunch of stuff off to PSRAM.
Regards, Mark.
On 17 Jan 2018, at 12:59 AM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
Bluetooth is disabled. Has been for a while, I don't see a chance to enable it with the 3.0 module.
I've also disabled Javascript support (duktape), Wolf, the web server and MDNS and the OBD2ECU. Telnet is enabled as I sometimes use that parallel to the async console.
That config gives me… OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0 …after reboot (without SD card), and with Wifi, modem, Twizy & server v2 running & connected…
OVMS > module memory Free 8-bit 23436/286352, 32-bit 17972/45144, SPIRAM 0/0
…when idle, will drop further with transmissions pending etc.
Opening a telnet session then results in:
OVMS > module memory Free 8-bit 18372/286352, 32-bit 17972/45144, SPIRAM 0/0
…and still stable.
Regards, Michael
Am 16.01.2018 um 01:11 schrieb Mark Webb-Johnson:
I’m seeing this:
OVMS > module memory Free 8-bit 79812/282268, 32-bit 17816/44988, SPIRAM 0/0
OVMS > sd unmount Unmounted SD CARD
OVMS > module memory Free 8-bit 107768/282268, 32-bit 17816/44988, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 20 0 0 -4 -27780 +0 +0
I guess it is FAT fs stuff, but we use that for /store as well so not sure why so much. I will look into it.
With wifi client, ovms server v2, tesla roadster vehicle module, I get this:
OVMS > module memory Free 8-bit 54224/282268, 32-bit 17816/44988, SPIRAM 0/0
With the OBDII HUD, that drops to 45KB.
Do you have bluetooth enabled (in menuconfig)?
Regards, Mark.
On 16 Jan 2018, at 3:35 AM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0 NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
> All committed and pushed now. > > Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported). > > Check > https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> > to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0. > > I hope this goes smoothly for you. > > Regards, Mark. > > commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) > Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> > Date: Mon Jan 15 07:56:45 2018 +0800 > > Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware > > commit 003592d553c881f735fb228b08836a13c306cec3 > Merge: e58c5b6 b28db5e > Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> > Date: Mon Jan 15 07:49:13 2018 +0800 > > Merge branch 'for-master'. This requires ESP-IDF v3.0 support: > > OVMS developers should now: > > 1] Pull the latest OpenVehicles IDF, and checkout MASTER branch. > > 2] Update XTENSA tools to match version required by Espressif for MASTER branch. > > 3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has > been provided. That can be merged/copied to sdkconfig as appropriate. > > 4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will > been provided in the next update. > > 5] It is recommended that developers perform at least one full 'make flash' > with this version, to update the bootloader to latest. Note that the > sdkconfig.default.* files are now set to require bootloader at least v2.1 > (or later), and bootloader v2.0 is no longer supported. > > Enjoy. > > >> On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote: >> >> Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour. >> >> Regards, Mark >> >>> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> <mailto:casner@acm.org> <mailto:casner@acm.org> wrote: >>> >>> Mark, >>> >>> It looks like you were not able to do the sdkconfig.default and merge >>> as you planned? >>> >>> -- Steve >>> >>>> On Fri, 12 Jan 2018, Stephen Casner wrote: >>>> >>>>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>>>> >>>>> @Steve can you update our master clone to latest from Espressif and >>>>> make sure your stuff is still ok? >>>> I have just now done this. I had already done a rebase a few days ago >>>> before committing the improved version of the OS changes so I could >>>> issue a pull request. There were a few commits since then, which I >>>> have now merged. My code still runs correctly. >>>> >>>>> I guess steps for developers will be: >>>>> >>>>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>>>> >>>>> 2) download and install xtensa build chain to match. >>>>> >>>>> 3) update OVMS master, make clean, check menu config, then build and play. >>>> Sounds right. >>>> >>>>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>>>> the Merge of the OVMS firmware master branch then. >>>> I'll be sleeping then, so you can go ahead and do the merge. >>>> >>>> -- Steve >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> >>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> >> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Tried & kept :) With vehicle module running, connectivity _and_ SD card mounted, I now have OVMS > module memory Free 8-bit 46200/285768, 32-bit 17948/45120, SPIRAM 0/0 Great find. I hope that won't affect SD write performance too badly. While testing I found a bug: after a manual "sd unmount" I get lots of E (121106) sdmmc_req: handle_idle_state_events unhandled: 00000008 00000002 …and new mounts afterwards won't work. Regards, Michael Am 19.01.2018 um 01:39 schrieb Mark Webb-Johnson:
Michael: Can you try menuconfig Component / FAT Filesystem support / Use separate cache for each file -> OFF. Documentation shows that controlling the FS_TINY flag (which sounds like something we want).
With wifi, DEMO vehicle module, sd mounted, and server v2 running, I get:
Free 8-bit 70136/282488, 32-bit 17660/44832, SPIRAM 0/0
OVMS > sd unmount OVMS > module memory Free 8-bit 77604/282488, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 356 1088 0 0 +16 -20 +0 +0 Tmr Svc 20 528 0 0 -860 -6444 +0 +0
Using “separate cache for each file”, I get:
Free 8-bit 29156/282496, 32-bit 17660/44832, SPIRAM 0/0
OVMS > sd unmount OVMS > module memory Free 8-bit 57096/282496, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 552 0 0 -4 -27780 +0 +0
Seems to be 27KB vs 7KB per filesystem mount, and we have two (/sd and /store). Overall win with FS_TINY for FAT is close to 41KB.
Regards, Mark.
On 17 Jan 2018, at 9:23 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote:
Note also that the memory debugging option adds a bit of overhead.
With debugging enabled (heap: light impact):
OVMS > module memory Free 8-bit 79868/282496, 32-bit 17660/44832, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5312 0 0 0 +5312 +0 +0 +0 esp_timer 49568 0 644 0 +49568 +0 +644 +0 main 39444 0 0 0 +39444 +0 +0 +0 ipc0 11096 0 0 0 +11096 +0 +0 +0 Housekeeping 24740 19956 0 0 +24740 +19956 +0 +0 tiT 132 0 0 0 +132 +0 +0 +0 Tmr Svc 4 27800 0 0 +4 +27800 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 AsyncConsole 0 36 26404 0 +0 +36 +26404 +0
OVMS > metrics list m.freeram m.freeram 79868
With debugging enabled (heap: comprehensive):
OVMS > module memory Free 8-bit 79728/282320, 32-bit 17968/45140, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM no task 5312 0 0 0 +5312 +0 +0 +0 esp_timer 49568 0 644 0 +49568 +0 +644 +0 main 39444 0 0 0 +39444 +0 +0 +0 ipc0 11096 0 0 0 +11096 +0 +0 +0 Housekeeping 24644 20056 0 0 +24644 +20056 +0 +0 tiT 132 0 0 0 +132 +0 +0 +0 Tmr Svc 4 27800 0 0 +4 +27800 +0 +0 ipc1 12 0 0 0 +12 +0 +0 +0 AsyncConsole 0 20 26404 0 +0 +20 +26404 +0 I (15614) simcom: State timeout, transition to 13 I (15614) simcom: State: Enter PoweredOff state OVMS > metrics list m.freeram m.freeram 79728
With debugging disabled:
OVMS > module memory To use these debugging tools, must set CONFIG_HEAP_TASK_TRACKING=y and have updated openvehicles/esp-idf OVMS > metrics list m.freeram m.freeram 99956
I think we’ve still got some stuff we can look at to lighten the impact on internal memory. Once we can get the 3.1 hardware out, I can focus on that. The memory tools Steve has built are really complete and usable now. They let us see exactly what is going on.
If we had known going in that the 520KB SRAM of the ESP32 would be 80% used by the system, and only 20% for us, perhaps things would have been done differently. Anyway, we are where we are, and the WROVER module should be able to move a bunch of stuff off to PSRAM.
Regards, Mark.
On 17 Jan 2018, at 12:59 AM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
Bluetooth is disabled. Has been for a while, I don't see a chance to enable it with the 3.0 module.
I've also disabled Javascript support (duktape), Wolf, the web server and MDNS and the OBD2ECU. Telnet is enabled as I sometimes use that parallel to the async console.
That config gives me… OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0 …after reboot (without SD card), and with Wifi, modem, Twizy & server v2 running & connected…
OVMS > module memory Free 8-bit 23436/286352, 32-bit 17972/45144, SPIRAM 0/0
…when idle, will drop further with transmissions pending etc.
Opening a telnet session then results in:
OVMS > module memory Free 8-bit 18372/286352, 32-bit 17972/45144, SPIRAM 0/0
…and still stable.
Regards, Michael
Am 16.01.2018 um 01:11 schrieb Mark Webb-Johnson:
I’m seeing this:
OVMS > module memory Free 8-bit 79812/282268, 32-bit 17816/44988, SPIRAM 0/0
OVMS > sd unmount Unmounted SD CARD
OVMS > module memory Free 8-bit 107768/282268, 32-bit 17816/44988, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 20 0 0 -4 -27780 +0 +0
I guess it is FAT fs stuff, but we use that for /store as well so not sure why so much. I will look into it.
With wifi client, ovms server v2, tesla roadster vehicle module, I get this:
OVMS > module memory Free 8-bit 54224/282268, 32-bit 17816/44988, SPIRAM 0/0
With the OBDII HUD, that drops to 45KB.
Do you have bluetooth enabled (in menuconfig)?
Regards, Mark.
On 16 Jan 2018, at 3:35 AM, Michael Balzer <dexter@expeedo.de> wrote:
I've found the missing RAM. It's allocated for the SD card:
OVMS > module memory Free 8-bit 117364/286352, 32-bit 17972/45144, SPIRAM 0/0
I (34235) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27804 +0 +0
I (68065) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
…again…
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
I (114985) sdcard: SD CARD has been inserted. Auto-mounting...
OVMS > module memory Free 8-bit 89348/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 4 27804 0 0 +4 +27792 +0 +0
E (141205) sdmmc_cmd: sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x107 E (141205) diskio_sdmmc: sdmmc_read_blocks failed (263) I (141515) sdcard: SD CARD has been removed.
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM Tmr Svc 0 12 0 0 -4 -27792 +0 +0
Btw, I think the RAM usage of the telnet server is also higher than before:
OVMS > module memory Free 8-bit 117332/286352, 32-bit 17972/45144, SPIRAM 0/0
OVMS > wifi mode client Starting WIFI as a client for any defined SSID I (380175) wifi: wifi firmware version: 403db1d I (380175) wifi: config NVS flash: enabled I (380175) wifi: config nano formating: disabled I (380185) wifi: Init dynamic tx buffer num: 16 I (380185) wifi: Init data frame dynamic rx buffer num: 16 I (380185) wifi: Init management frame dynamic rx buffer num: 16 I (380185) wifi: wifi driver task: 3ffe3db4, prio:23, stack:4096 I (380185) wifi: Init static rx buffer num: 4 I (380185) wifi: Init dynamic rx buffer num: 16 I (380195) wifi: wifi power manager task: 0x3ffe846c prio: 21 stack: 2560 I (380195) wifi: mode : sta (30:ae:a4:37:25:88) W (390205) wifi: incorrect scan type: 1073534024 I (392615) esp32wifi: Found SSID devolo-f4068d73a03e - trying to connect I (393945) wifi: n:11 2, o:1 0, ap:255 255, sta:11 2, prof:1 I (394595) wifi: state: init -> auth (b0) I (394605) wifi: state: auth -> assoc (0) I (394605) wifi: state: assoc -> run (10) I (394625) wifi: connected with devolo-f4068d73a03e, channel 11 I (394665) esp32wifi: WiFi UP with SSID: devolo-f4068d73a03e, MAC: 30:ae:a4:37:25:88, IP: 192.168.2.101, mask: 255.255.255.0, gw: 192.168.2.1 I (394665) telnet: Launching Telnet Server I (397615) wifi: pm start, type:0
OVMS > module memory Free 8-bit 89700/286352, 32-bit 17972/45144, SPIRAM 0/0 --Task-- Total DRAM D/IRAM IRAM SPIRAM +/- DRAM D/IRAM IRAM SPIRAM tiT 128 716 0 0 +0 +484 +0 +0 AsyncConsole 0 16552 26404 0 +0 +16356 +0 +0 NetManTask 8 488 0 0 +8 +488 +0 +0 wifi 0 1600 0 0 +0 +1600 +0 +0 eventTask 0 7564 0 0 +0 +7564 +0 +0
…but I'm not sure about that, haven't tested that in a while.
So, SD cards now work, and I'll not be able to use them due to the RAM usage… :(
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner: > In order for the the "module memory" and "module tasks" commands to > work, the configuration needs to be changed from the sdk.default.hw30 > that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to > be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or > COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have > attached a diff for the sdkconfig. That diff also includes a section > for enabling telnet, which some of you may be using. > > Please also note that in this revised implementation of "module > memory" the block address is now the address that is returned from > malloc, not the address of the header that precedes it, and the length > of the block is now exclusive of the debugging overhead. > > -- Steve > > On Mon, 15 Jan 2018, Mark Webb-Johnson wrote: > >> All committed and pushed now. >> >> Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported). >> >> Check >> https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> >> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0. >> >> I hope this goes smoothly for you. >> >> Regards, Mark. >> >> commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) >> Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> >> Date: Mon Jan 15 07:56:45 2018 +0800 >> >> Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware >> >> commit 003592d553c881f735fb228b08836a13c306cec3 >> Merge: e58c5b6 b28db5e >> Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> >> Date: Mon Jan 15 07:49:13 2018 +0800 >> >> Merge branch 'for-master'. This requires ESP-IDF v3.0 support: >> >> OVMS developers should now: >> >> 1] Pull the latest OpenVehicles IDF, and checkout MASTER branch. >> >> 2] Update XTENSA tools to match version required by Espressif for MASTER branch. >> >> 3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has >> been provided. That can be merged/copied to sdkconfig as appropriate. >> >> 4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will >> been provided in the next update. >> >> 5] It is recommended that developers perform at least one full 'make flash' >> with this version, to update the bootloader to latest. Note that the >> sdkconfig.default.* files are now set to require bootloader at least v2.1 >> (or later), and bootloader v2.0 is no longer supported. >> >> Enjoy. >> >> >>> On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote: >>> >>> Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour. >>> >>> Regards, Mark >>> >>>> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> wrote: >>>> >>>> Mark, >>>> >>>> It looks like you were not able to do the sdkconfig.default and merge >>>> as you planned? >>>> >>>> -- Steve >>>> >>>>> On Fri, 12 Jan 2018, Stephen Casner wrote: >>>>> >>>>>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>>>>> >>>>>> @Steve can you update our master clone to latest from Espressif and >>>>>> make sure your stuff is still ok? >>>>> I have just now done this. I had already done a rebase a few days ago >>>>> before committing the improved version of the OS changes so I could >>>>> issue a pull request. There were a few commits since then, which I >>>>> have now merged. My code still runs correctly. >>>>> >>>>>> I guess steps for developers will be: >>>>>> >>>>>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>>>>> >>>>>> 2) download and install xtensa build chain to match. >>>>>> >>>>>> 3) update OVMS master, make clean, check menu config, then build and play. >>>>> Sounds right. >>>>> >>>>>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>>>>> the Merge of the OVMS firmware master branch then. >>>>> I'll be sleeping then, so you can go ahead and do the merge. >>>>> >>>>> -- Steve >>>> _______________________________________________ >>>> OvmsDev mailing list >>>> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> >>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> >>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Am 19.01.2018 um 15:08 schrieb Michael Balzer:
Great find. I hope that won't affect SD write performance too badly.
Doesn't seem so. I've added message count/drop statistics to the CAN logger: no drops within 3 minutes of full speed logging. I think unless we really stress the filesystem with high volume parallel writes we won't notice any difference. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
I don’t think the setting changes much for the limited usage we have. We would typically have zero files open on flash, and perhaps one file when tracing, etc. It made zero impact to the simplistic benchmark test I did (one file write/read). Regards, Mark
On 19 Jan 2018, at 11:41 PM, Michael Balzer <dexter@expeedo.de> wrote:
Am 19.01.2018 um 15:08 schrieb Michael Balzer:
Great find. I hope that won't affect SD write performance too badly.
Doesn't seem so. I've added message count/drop statistics to the CAN logger: no drops within 3 minutes of full speed logging.
I think unless we really stress the filesystem with high volume parallel writes we won't notice any difference.
Regards, Michael
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before. Gory details are here: https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1): https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET. It sure broke us. I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already. It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority. Regards, Mark.
On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com> wrote:
But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
On Tue, 16 Jan 2018, Greg D. wrote:
I notice that the 3.0 code base is also a lot less chatty than the 2.1. Almost spooky...
Are you referring to the fact that the default log level has changed from INFO to WARN so all those many INFO logs are not longer shown? That is a config setting you can change. -- Steve
Stephen Casner wrote:
On Tue, 16 Jan 2018, Greg D. wrote:
I notice that the 3.0 code base is also a lot less chatty than the 2.1. Almost spooky... Are you referring to the fact that the default log level has changed from INFO to WARN so all those many INFO logs are not longer shown? That is a config setting you can change.
-- Steve
Yes, I understand. But, even the boot-up dialog is short, and until you change things, it's like, hello? Did it just hang? It just caught me by surprise, I guess. Greg
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE. It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things. I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why. Regards, Mark.
On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before.
Gory details are here:
https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>
I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1):
https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>
LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET.
It sure broke us.
I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already.
It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority.
Regards, Mark.
On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com>> wrote:
But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
Mark, The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll. -- Steve On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE.
It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things.
I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why.
Regards, Mark.
On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before.
Gory details are here:
https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>
I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1):
https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>
LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET.
It sure broke us.
I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already.
It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority.
Regards, Mark.
On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com>> wrote:
But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
Oops. Sorry. That change broke MQTT. I couldn’t understand what was going on (as mg_send was sending immediately). MQTT works like this: void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len; uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id); mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len); mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); } It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life. It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-( I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well. How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten? Regards, Mark.
On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org> wrote:
Mark,
The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE.
It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things.
I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why.
Regards, Mark.
On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before.
Gory details are here:
https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>
I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1):
https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>
LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET.
It sure broke us.
I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already.
It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority.
Regards, Mark.
On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com>> wrote:
But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Mark, Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code. As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be. Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose. -- Steve On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
Oops. Sorry. That change broke MQTT. I couldn’t understand what was going on (as mg_send was sending immediately). MQTT works like this:
void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len;
uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id);
mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len);
mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); }
It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life.
It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-(
I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well.
How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten?
Regards, Mark.
On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org> wrote:
Mark,
The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE.
It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things.
I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why.
Regards, Mark.
On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before.
Gory details are here:
https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>
I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1):
https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>
LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET.
It sure broke us.
I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already.
It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority.
Regards, Mark.
On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com>> wrote:
But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
It doesn’t seem as if there is a good solution. I can see two approaches: Use a MG_F_USER_? flag to mean ‘immediate write’ and extend mg_send to honour that. Add a separate mg_flush() call (used after mg_send) to flush the fd. That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c? All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this? For console_ssh, I think this is where it is: int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; } Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can. Regards, Mark.
On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code.
As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be.
Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
Oops. Sorry. That change broke MQTT. I couldn’t understand what was going on (as mg_send was sending immediately). MQTT works like this:
void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len;
uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id);
mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len);
mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); }
It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life.
It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-(
I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well.
How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten?
Regards, Mark.
On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org> wrote:
Mark,
The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE.
It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things.
I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why.
Regards, Mark.
On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before.
Gory details are here:
https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>
I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1):
https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>
LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET.
It sure broke us.
I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already.
It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority.
Regards, Mark.
On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com>> wrote:
But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
To be hopefully clear(er): For event driven systems sending a big file, the usual approach is to send a block, wait for SENT callback, then send the next block. Repeat until no more file left. That approach minimises the buffer usage. We are trying to shoe-horn SSH into this event driven system, but the wolfssh system is expecting a normal blocking API. But, we are running in a separate task, so with a semaphore / poll we can convert the events into a blocking API. Two approaches I can see: Simple is to just check if ctx->send_mbuf has enough space. If not, sleep for a while, and check again. Rely on the mongoose task emptying the buffer. More complex is to use the mongoose MG_EV_SEND callback (which signifies that some data has been sent), and a semaphore to signal data has been sent. The OvmsSSH::EventHandler and SendCallback could then use that to co-ordinate and avoid sleeping. This is the preferred approach. Perhaps this is general enough to be put into a library? An object that could be used to encapsulate the semaphore (initialised to indicate data has been sent), method to indicate data has been sent, and method to wait for that indication. I have a similar problem (although the reverse - receive rather than transmit) in ovms_ota now, and perhaps a generic solution could solve both our problems? Regards, Mark.
On 22 Jan 2018, at 12:34 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
It doesn’t seem as if there is a good solution. I can see two approaches:
Use a MG_F_USER_? flag to mean ‘immediate write’ and extend mg_send to honour that.
Add a separate mg_flush() call (used after mg_send) to flush the fd.
That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c?
All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this?
For console_ssh, I think this is where it is:
int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; }
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
Regards, Mark.
On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code.
As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be.
Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
Oops. Sorry. That change broke MQTT. I couldn’t understand what was going on (as mg_send was sending immediately). MQTT works like this:
void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len;
uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id);
mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len);
mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); }
It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life.
It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-(
I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well.
How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten?
Regards, Mark.
On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE.
It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things.
I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why.
Regards, Mark.
On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote:
This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before.
Gory details are here:
https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> <https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>>
I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1):
https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>>
LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET.
It sure broke us.
I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already.
It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority.
Regards, Mark.
> On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com> <mailto:gregd2350@gmail.com <mailto:gregd2350@gmail.com>>> wrote: > > But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Umm, Mark, we're not running a separate task. If we were, this would not be a problem. With one task, sleeping would do no good because that would be causing Mongoose to sleep. The only way we can save the state of the "user" code that is doing the output would be to recurse into Mongoose so it can actually send, but that is likely to lead to a stack overflow due to repeated recursion. After pondering many ideas, that's why I made the change to mg_send. It is still not difficult to use one of several options make a solution that allows SSH to send immediately while MQTT does not, but it's just that some mods to mongoose.c will be required. -- Steve On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
To be hopefully clear(er):
For event driven systems sending a big file, the usual approach is to send a block, wait for SENT callback, then send the next block. Repeat until no more file left. That approach minimises the buffer usage.
We are trying to shoe-horn SSH into this event driven system, but the wolfssh system is expecting a normal blocking API.
But, we are running in a separate task, so with a semaphore / poll we can convert the events into a blocking API.
Two approaches I can see:
Simple is to just check if ctx->send_mbuf has enough space. If not, sleep for a while, and check again. Rely on the mongoose task emptying the buffer.
More complex is to use the mongoose MG_EV_SEND callback (which signifies that some data has been sent), and a semaphore to signal data has been sent. The OvmsSSH::EventHandler and SendCallback could then use that to co-ordinate and avoid sleeping. This is the preferred approach.
Perhaps this is general enough to be put into a library? An object that could be used to encapsulate the semaphore (initialised to indicate data has been sent), method to indicate data has been sent, and method to wait for that indication. I have a similar problem (although the reverse - receive rather than transmit) in ovms_ota now, and perhaps a generic solution could solve both our problems?
Regards, Mark.
On 22 Jan 2018, at 12:34 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
It doesn’t seem as if there is a good solution. I can see two approaches:
Use a MG_F_USER_? flag to mean ‘immediate write’ and extend mg_send to honour that.
Add a separate mg_flush() call (used after mg_send) to flush the fd.
That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c?
All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this?
For console_ssh, I think this is where it is:
int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; }
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
Regards, Mark.
On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code.
As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be.
Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
Oops. Sorry. That change broke MQTT. I couldn’t understand what was going on (as mg_send was sending immediately). MQTT works like this:
void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len;
uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id);
mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len);
mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); }
It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life.
It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-(
I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well.
How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten?
Regards, Mark.
On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE.
It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things.
I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why.
Regards, Mark.
> On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote: > > > This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before. > > Gory details are here: > > https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> <https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>> > > I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1): > > https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>> > > LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET. > > It sure broke us. > > I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already. > > It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority. > > Regards, Mark. > >> On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com> <mailto:gregd2350@gmail.com <mailto:gregd2350@gmail.com>>> wrote: >> >> But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem. >
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Ok, understood. I thought it was running under console task. I guess the issue is primarily SCP? A general command output should be reasonably small, but a file could by MB in size? What happens if the send_mbuf is full? Regards, Mark.
On 22 Jan 2018, at 12:53 PM, Stephen Casner <casner@acm.org> wrote:
Umm, Mark, we're not running a separate task. If we were, this would not be a problem. With one task, sleeping would do no good because that would be causing Mongoose to sleep. The only way we can save the state of the "user" code that is doing the output would be to recurse into Mongoose so it can actually send, but that is likely to lead to a stack overflow due to repeated recursion. After pondering many ideas, that's why I made the change to mg_send.
It is still not difficult to use one of several options make a solution that allows SSH to send immediately while MQTT does not, but it's just that some mods to mongoose.c will be required.
-- Steve
On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
To be hopefully clear(er):
For event driven systems sending a big file, the usual approach is to send a block, wait for SENT callback, then send the next block. Repeat until no more file left. That approach minimises the buffer usage.
We are trying to shoe-horn SSH into this event driven system, but the wolfssh system is expecting a normal blocking API.
But, we are running in a separate task, so with a semaphore / poll we can convert the events into a blocking API.
Two approaches I can see:
Simple is to just check if ctx->send_mbuf has enough space. If not, sleep for a while, and check again. Rely on the mongoose task emptying the buffer.
More complex is to use the mongoose MG_EV_SEND callback (which signifies that some data has been sent), and a semaphore to signal data has been sent. The OvmsSSH::EventHandler and SendCallback could then use that to co-ordinate and avoid sleeping. This is the preferred approach.
Perhaps this is general enough to be put into a library? An object that could be used to encapsulate the semaphore (initialised to indicate data has been sent), method to indicate data has been sent, and method to wait for that indication. I have a similar problem (although the reverse - receive rather than transmit) in ovms_ota now, and perhaps a generic solution could solve both our problems?
Regards, Mark.
On 22 Jan 2018, at 12:34 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
It doesn’t seem as if there is a good solution. I can see two approaches:
Use a MG_F_USER_? flag to mean ‘immediate write’ and extend mg_send to honour that.
Add a separate mg_flush() call (used after mg_send) to flush the fd.
That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c?
All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this?
For console_ssh, I think this is where it is:
int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; }
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
Regards, Mark.
On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code.
As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be.
Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
Oops. Sorry. That change broke MQTT. I couldn’t understand what was going on (as mg_send was sending immediately). MQTT works like this:
void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len;
uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id);
mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len);
mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); }
It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life.
It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-(
I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well.
How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten?
Regards, Mark.
On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
The update of Mongoose to v6.10 removed the change I had made so that the mg_send() call would transmit on the network immediately if the socket was ready. I needed to make that change because we would otherwise run out of RAM with SSH because mg_send() would just buffer everything until the next poll.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
> I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE. > > It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things. > > I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I’m still trying to find out why. > > Regards, Mark. > >> On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote: >> >> >> This is the issue Michael pointed out. The 'server response is incomplete’ problem with select(). Apologies for this; I am not sure why I didn’t notice it before. >> >> Gory details are here: >> >> https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> <https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>> >> >> I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a ‘breaking change’ (at the bottom of the release notes for 3.0b1): >> >> https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>> >> >> LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET. >> >> It sure broke us. >> >> I’ve made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already. >> >> It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won’t need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority. >> >> Regards, Mark. >> >>> On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com> <mailto:gregd2350@gmail.com <mailto:gregd2350@gmail.com>>> wrote: >>> >>> But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem. >> > _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
My original implementation of both telnet and ssh was with separate tasks. That was straightforward. However, you wanted them merged under mongoose to avoid the cost of the separate stack spaces. That required reworking the control flow to fit into the mongoose event-driven structure. Both telnet and wolfssh are capable of working with non-blocking I/O, so this was possible though it took me a while to work out what I needed to do. The severity of the memory limit depends upon how low we try to go with additional components configured in. Even typing the '?' command does a reasonable chunk of output now, and in SSH that gets amplified a little by the encryption overhead. The send_mbuf does not get full because mongoose just keeps doing realloc. -- Steve On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Ok, understood. I thought it was running under console task.
I guess the issue is primarily SCP? A general command output should be reasonably small, but a file could by MB in size? What happens if the send_mbuf is full?
Regards, Mark.
On 22 Jan 2018, at 12:53 PM, Stephen Casner <casner@acm.org> wrote:
Umm, Mark, we're not running a separate task. If we were, this would not be a problem. With one task, sleeping would do no good because that would be causing Mongoose to sleep. The only way we can save the state of the "user" code that is doing the output would be to recurse into Mongoose so it can actually send, but that is likely to lead to a stack overflow due to repeated recursion. After pondering many ideas, that's why I made the change to mg_send.
It is still not difficult to use one of several options make a solution that allows SSH to send immediately while MQTT does not, but it's just that some mods to mongoose.c will be required.
-- Steve
On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
To be hopefully clear(er):
For event driven systems sending a big file, the usual approach is to send a block, wait for SENT callback, then send the next block. Repeat until no more file left. That approach minimises the buffer usage.
We are trying to shoe-horn SSH into this event driven system, but the wolfssh system is expecting a normal blocking API.
But, we are running in a separate task, so with a semaphore / poll we can convert the events into a blocking API.
Two approaches I can see:
Simple is to just check if ctx->send_mbuf has enough space. If not, sleep for a while, and check again. Rely on the mongoose task emptying the buffer.
More complex is to use the mongoose MG_EV_SEND callback (which signifies that some data has been sent), and a semaphore to signal data has been sent. The OvmsSSH::EventHandler and SendCallback could then use that to co-ordinate and avoid sleeping. This is the preferred approach.
Perhaps this is general enough to be put into a library? An object that could be used to encapsulate the semaphore (initialised to indicate data has been sent), method to indicate data has been sent, and method to wait for that indication. I have a similar problem (although the reverse - receive rather than transmit) in ovms_ota now, and perhaps a generic solution could solve both our problems?
Regards, Mark.
On 22 Jan 2018, at 12:34 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
It doesn't seem as if there is a good solution. I can see two approaches:
Use a MG_F_USER_? flag to mean 'immediate write' and extend mg_send to honour that.
Add a separate mg_flush() call (used after mg_send) to flush the fd.
That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c?
All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this?
For console_ssh, I think this is where it is:
int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; }
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
Regards, Mark.
On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code.
As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be.
Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
Oops. Sorry. That change broke MQTT. I couldn't understand what was going on (as mg_send was sending immediately). MQTT works like this:
void mg_mqtt_publish(struct mg_connection *nc, const char *topic, uint16_t message_id, int flags, const void *data, size_t len) { size_t old_len = nc->send_mbuf.len;
uint16_t topic_len = htons((uint16_t) strlen(topic)); uint16_t message_id_net = htons(message_id);
mg_send(nc, &topic_len, 2); mg_send(nc, topic, strlen(topic)); if (MG_MQTT_GET_QOS(flags) > 0) { mg_send(nc, &message_id_net, 2); } mg_send(nc, data, len);
mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, nc->send_mbuf.len - old_len); }
It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life.
It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-(
I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well.
How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten?
Regards, Mark.
> On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote: > > Mark, > > The update of Mongoose to v6.10 removed the change I had made so that > the mg_send() call would transmit on the network immediately if the > socket was ready. I needed to make that change because we would > otherwise run out of RAM with SSH because mg_send() would just buffer > everything until the next poll. > > -- Steve > > On Fri, 19 Jan 2018, Mark Webb-Johnson wrote: > >> I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE. >> >> It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things. >> >> I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I'm still trying to find out why. >> >> Regards, Mark. >> >>> On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote: >>> >>> >>> This is the issue Michael pointed out. The 'server response is incomplete' problem with select(). Apologies for this; I am not sure why I didn't notice it before. >>> >>> Gory details are here: >>> >>> https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> <https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>> >>> >>> I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a 'breaking change' (at the bottom of the release notes for 3.0b1): >>> >>> https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>> >>> >>> LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET. >>> >>> It sure broke us. >>> >>> I've made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already. >>> >>> It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won't need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority. >>> >>> Regards, Mark. >>> >>>> On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com> <mailto:gregd2350@gmail.com <mailto:gregd2350@gmail.com>>> wrote: >>>> >>>> But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem. >>> >> > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
How does wolfssh deal with the scp issue? Can it send chunk by chunk, based on MG_EV_SEND? I’m not so worried about normal command output (especially with SPI RAM), but scp terrifies me. Regards, Mark.
On 22 Jan 2018, at 1:25 PM, Stephen Casner <casner@acm.org> wrote:
My original implementation of both telnet and ssh was with separate tasks. That was straightforward. However, you wanted them merged under mongoose to avoid the cost of the separate stack spaces. That required reworking the control flow to fit into the mongoose event-driven structure. Both telnet and wolfssh are capable of working with non-blocking I/O, so this was possible though it took me a while to work out what I needed to do.
The severity of the memory limit depends upon how low we try to go with additional components configured in. Even typing the '?' command does a reasonable chunk of output now, and in SSH that gets amplified a little by the encryption overhead.
The send_mbuf does not get full because mongoose just keeps doing realloc.
-- Steve
On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Ok, understood. I thought it was running under console task.
I guess the issue is primarily SCP? A general command output should be reasonably small, but a file could by MB in size? What happens if the send_mbuf is full?
Regards, Mark.
On 22 Jan 2018, at 12:53 PM, Stephen Casner <casner@acm.org> wrote:
Umm, Mark, we're not running a separate task. If we were, this would not be a problem. With one task, sleeping would do no good because that would be causing Mongoose to sleep. The only way we can save the state of the "user" code that is doing the output would be to recurse into Mongoose so it can actually send, but that is likely to lead to a stack overflow due to repeated recursion. After pondering many ideas, that's why I made the change to mg_send.
It is still not difficult to use one of several options make a solution that allows SSH to send immediately while MQTT does not, but it's just that some mods to mongoose.c will be required.
-- Steve
On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
To be hopefully clear(er):
For event driven systems sending a big file, the usual approach is to send a block, wait for SENT callback, then send the next block. Repeat until no more file left. That approach minimises the buffer usage.
We are trying to shoe-horn SSH into this event driven system, but the wolfssh system is expecting a normal blocking API.
But, we are running in a separate task, so with a semaphore / poll we can convert the events into a blocking API.
Two approaches I can see:
Simple is to just check if ctx->send_mbuf has enough space. If not, sleep for a while, and check again. Rely on the mongoose task emptying the buffer.
More complex is to use the mongoose MG_EV_SEND callback (which signifies that some data has been sent), and a semaphore to signal data has been sent. The OvmsSSH::EventHandler and SendCallback could then use that to co-ordinate and avoid sleeping. This is the preferred approach.
Perhaps this is general enough to be put into a library? An object that could be used to encapsulate the semaphore (initialised to indicate data has been sent), method to indicate data has been sent, and method to wait for that indication. I have a similar problem (although the reverse - receive rather than transmit) in ovms_ota now, and perhaps a generic solution could solve both our problems?
Regards, Mark.
On 22 Jan 2018, at 12:34 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
It doesn't seem as if there is a good solution. I can see two approaches:
Use a MG_F_USER_? flag to mean 'immediate write' and extend mg_send to honour that.
Add a separate mg_flush() call (used after mg_send) to flush the fd.
That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c?
All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this?
For console_ssh, I think this is where it is:
int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; }
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
Regards, Mark.
On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
Well, in turn, I'm sorry for making an API change that was driving you crazy. It would have been smarter to add it as a new function even though that would be duplicating more code.
As the code currently stands, telnet and SSH will work so long as no operation does more contiguous output than the amount of available free memory can hold, otherwise an out-of-memory crash will occur. I don't know if we consider that an acceptable risk. Maybe with v3.1 hardware it will be.
Your suggestion to put a new funtion into a separate module is a fine idea, but that function needs to access some functions in mongoose.c that are scoped static. That means we can't entirely avoid modifying mongoose.
-- Steve
On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
> Oops. Sorry. That change broke MQTT. I couldn't understand what was going on (as mg_send was sending immediately). MQTT works like this: > > void mg_mqtt_publish(struct mg_connection *nc, const char *topic, > uint16_t message_id, int flags, const void *data, > size_t len) { > size_t old_len = nc->send_mbuf.len; > > uint16_t topic_len = htons((uint16_t) strlen(topic)); > uint16_t message_id_net = htons(message_id); > > mg_send(nc, &topic_len, 2); > mg_send(nc, topic, strlen(topic)); > if (MG_MQTT_GET_QOS(flags) > 0) { > mg_send(nc, &message_id_net, 2); > } > mg_send(nc, data, len); > > mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, > nc->send_mbuf.len - old_len); > } > > It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life. > > It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-( > > I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well. > > How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten? > > Regards, Mark. > >> On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote: >> >> Mark, >> >> The update of Mongoose to v6.10 removed the change I had made so that >> the mg_send() call would transmit on the network immediately if the >> socket was ready. I needed to make that change because we would >> otherwise run out of RAM with SSH because mg_send() would just buffer >> everything until the next poll. >> >> -- Steve >> >> On Fri, 19 Jan 2018, Mark Webb-Johnson wrote: >> >>> I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE. >>> >>> It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things. >>> >>> I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I'm still trying to find out why. >>> >>> Regards, Mark. >>> >>>> On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote: >>>> >>>> >>>> This is the issue Michael pointed out. The 'server response is incomplete' problem with select(). Apologies for this; I am not sure why I didn't notice it before. >>>> >>>> Gory details are here: >>>> >>>> https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> <https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>> >>>> >>>> I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a 'breaking change' (at the bottom of the release notes for 3.0b1): >>>> >>>> https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>> >>>> >>>> LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET. >>>> >>>> It sure broke us. >>>> >>>> I've made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already. >>>> >>>> It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won't need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority. >>>> >>>> Regards, Mark. >>>> >>>>> On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com> <mailto:gregd2350@gmail.com <mailto:gregd2350@gmail.com>>> wrote: >>>>> >>>>> But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem. >>>> >>> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> >> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev > _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
WolfSSH does not implement scp, my code does. I have a loop that reads a buffer from the file and then sends it through WolfSSH which makes up outgoing packets and calls my SendCallback() one or more times, each of which issues a mg_send() call. But I break out of that loop if MG_EV_SEND has not been received. With my modified mongoose, if mg_send() was able to transmit immediately then that would cause a recursive call back to my event loop with MG_EV_SEND so this loop would spin along merrily. As the code currently stands without my mods, the loop will always break after sending one buffer. As I reviewed the code to answer your question, I see that I enter that send loop at every MG_EV_POLL. Perhaps I should add a check before entering the loop to see that the previous buffer has been sent. This may make the scp slow, though. -- Steve On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
How does wolfssh deal with the scp issue? Can it send chunk by chunk, based on MG_EV_SEND? I'm not so worried about normal command output (especially with SPI RAM), but scp terrifies me.
Regards, Mark.
On 22 Jan 2018, at 1:25 PM, Stephen Casner <casner@acm.org> wrote:
My original implementation of both telnet and ssh was with separate tasks. That was straightforward. However, you wanted them merged under mongoose to avoid the cost of the separate stack spaces. That required reworking the control flow to fit into the mongoose event-driven structure. Both telnet and wolfssh are capable of working with non-blocking I/O, so this was possible though it took me a while to work out what I needed to do.
The severity of the memory limit depends upon how low we try to go with additional components configured in. Even typing the '?' command does a reasonable chunk of output now, and in SSH that gets amplified a little by the encryption overhead.
The send_mbuf does not get full because mongoose just keeps doing realloc.
-- Steve
On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Ok, understood. I thought it was running under console task.
I guess the issue is primarily SCP? A general command output should be reasonably small, but a file could by MB in size? What happens if the send_mbuf is full?
Regards, Mark.
On 22 Jan 2018, at 12:53 PM, Stephen Casner <casner@acm.org> wrote:
Umm, Mark, we're not running a separate task. If we were, this would not be a problem. With one task, sleeping would do no good because that would be causing Mongoose to sleep. The only way we can save the state of the "user" code that is doing the output would be to recurse into Mongoose so it can actually send, but that is likely to lead to a stack overflow due to repeated recursion. After pondering many ideas, that's why I made the change to mg_send.
It is still not difficult to use one of several options make a solution that allows SSH to send immediately while MQTT does not, but it's just that some mods to mongoose.c will be required.
-- Steve
On Mon, 22 Jan 2018, Mark Webb-Johnson wrote:
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
To be hopefully clear(er):
For event driven systems sending a big file, the usual approach is to send a block, wait for SENT callback, then send the next block. Repeat until no more file left. That approach minimises the buffer usage.
We are trying to shoe-horn SSH into this event driven system, but the wolfssh system is expecting a normal blocking API.
But, we are running in a separate task, so with a semaphore / poll we can convert the events into a blocking API.
Two approaches I can see:
Simple is to just check if ctx->send_mbuf has enough space. If not, sleep for a while, and check again. Rely on the mongoose task emptying the buffer.
More complex is to use the mongoose MG_EV_SEND callback (which signifies that some data has been sent), and a semaphore to signal data has been sent. The OvmsSSH::EventHandler and SendCallback could then use that to co-ordinate and avoid sleeping. This is the preferred approach.
Perhaps this is general enough to be put into a library? An object that could be used to encapsulate the semaphore (initialised to indicate data has been sent), method to indicate data has been sent, and method to wait for that indication. I have a similar problem (although the reverse - receive rather than transmit) in ovms_ota now, and perhaps a generic solution could solve both our problems?
Regards, Mark.
On 22 Jan 2018, at 12:34 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
It doesn't seem as if there is a good solution. I can see two approaches:
Use a MG_F_USER_? flag to mean 'immediate write' and extend mg_send to honour that.
Add a separate mg_flush() call (used after mg_send) to flush the fd.
That static function is going to be a pain to workaround. Perhaps a #include for our C code in mongoose.c?
All of this is going to be fighting the event-driven mechanism of Mongoose. Is there another way of doing this?
For console_ssh, I think this is where it is:
int SendCallback(WOLFSSH* ssh, void* data, uint32_t size, void* ctx) { mg_send((mg_connection*)ctx, (char*)data, size); return size; }
Perhaps check add a loop to check if ctx->send_mbuf.len and ctx->send_mbuf.size to see how much space is free. If not enough, then sleep the task (10ms or 100ms?) and try again? Or use MG_EV_SEND to set a semaphore, and pickup on that in the SendCallback? Rely on the fact that mongoose is running in a separate task and will empty the buffer when it can.
Regards, Mark.
> On 22 Jan 2018, at 3:17 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote: > > Mark, > > Well, in turn, I'm sorry for making an API change that was driving you > crazy. It would have been smarter to add it as a new function even > though that would be duplicating more code. > > As the code currently stands, telnet and SSH will work so long as no > operation does more contiguous output than the amount of available > free memory can hold, otherwise an out-of-memory crash will occur. I > don't know if we consider that an acceptable risk. Maybe with v3.1 > hardware it will be. > > Your suggestion to put a new funtion into a separate module is a fine > idea, but that function needs to access some functions in mongoose.c > that are scoped static. That means we can't entirely avoid modifying > mongoose. > > -- Steve > > On Fri, 19 Jan 2018, Mark Webb-Johnson wrote: > >> Oops. Sorry. That change broke MQTT. I couldn't understand what was going on (as mg_send was sending immediately). MQTT works like this: >> >> void mg_mqtt_publish(struct mg_connection *nc, const char *topic, >> uint16_t message_id, int flags, const void *data, >> size_t len) { >> size_t old_len = nc->send_mbuf.len; >> >> uint16_t topic_len = htons((uint16_t) strlen(topic)); >> uint16_t message_id_net = htons(message_id); >> >> mg_send(nc, &topic_len, 2); >> mg_send(nc, topic, strlen(topic)); >> if (MG_MQTT_GET_QOS(flags) > 0) { >> mg_send(nc, &message_id_net, 2); >> } >> mg_send(nc, data, len); >> >> mg_mqtt_prepend_header(nc, MG_MQTT_CMD_PUBLISH, flags, >> nc->send_mbuf.len - old_len); >> } >> >> It uses mg_send a bunch of times, then goes back and modifies the send_mbuf by inserting a header, then finishes so that the actual transmission can occur. Seems a really dumb way to do it, but such is life. >> >> It was driving me crazy last night, so in the end I just updated mongoose this morning and hey! everything worked. Now I know why :-( >> >> I see that mg_send_dns_query() does the same (it calls mg_dns_insert_header, which then calls mbuf_insert). Making mg_send transmit immediately would break that as well. >> >> How about introducing a new mg_send_now() that calls mg_send() then sends the data immediately? Perhaps it could be a separate .h/.c file mongoose_extensions to avoid the change getting overwritten? >> >> Regards, Mark. >> >>> On 19 Jan 2018, at 2:36 PM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote: >>> >>> Mark, >>> >>> The update of Mongoose to v6.10 removed the change I had made so that >>> the mg_send() call would transmit on the network immediately if the >>> socket was ready. I needed to make that change because we would >>> otherwise run out of RAM with SSH because mg_send() would just buffer >>> everything until the next poll. >>> >>> -- Steve >>> >>> On Fri, 19 Jan 2018, Mark Webb-Johnson wrote: >>> >>>> I re-worked the ovms_server_* framework, and v2 implementation, to use MONGOOSE. >>>> >>>> It seems to be _basically_ working. It can connect/disconnect/etc. Some slight memory saving, but standardising the networking throughout on mongoose should simplify things. >>>> >>>> I am seeing problems with transmitting the FEATURES and PARAMETERS sometimes - particularly in low memory situations. I'm still trying to find out why. >>>> >>>> Regards, Mark. >>>> >>>>> On 17 Jan 2018, at 8:33 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote: >>>>> >>>>> >>>>> This is the issue Michael pointed out. The 'server response is incomplete' problem with select(). Apologies for this; I am not sure why I didn't notice it before. >>>>> >>>>> Gory details are here: >>>>> >>>>> https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510> <https://github.com/espressif/esp-idf/issues/1510 <https://github.com/espressif/esp-idf/issues/1510>> >>>>> >>>>> I think Espressif implemented requirement this in a bizarre way, likely to break compatibility, but such is life. They did point it out as a 'breaking change' (at the bottom of the release notes for 3.0b1): >>>>> >>>>> https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1> <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1 <https://github.com/espressif/esp-idf/releases/tag/v3.0-rc1>> >>>>> >>>>> LWIP socket file descriptors now take higher numeric values (via the LWIP LWIP_SOCKET_OFFSET macro). BSD sockets code should mostly work as expected (and, new in V3.0, some standard POSIX functions can now be used with sockets). However any code which assumes a socket file descriptor is always a low numbered integer may need modifying to account for LWIP_SOCKET_OFFSET. >>>>> >>>>> It sure broke us. >>>>> >>>>> I've made a one-line workaround fix (to ovms_buffer.cpp), and ovms server v2 connections are working again for me. That is committed and pushed already. >>>>> >>>>> It is kind of messy to have all these different networking implementations in our code base; I intend to move ovms_server_* to mongoose networking over the next few days. That will mean we won't need a separate task/stack for server connections, and should save us 7KB internal RAM for each connection. Also ovms_ota. But that will have to wait, as I need to get the hardware complete first (some issues with 1.8v vs 3.3v logic on VDD_SDIO of the wrover module and some of our GPIOs), and that is top priority. >>>>> >>>>> Regards, Mark. >>>>> >>>>>> On 17 Jan 2018, at 7:05 AM, Greg D. <gregd2350@gmail.com <mailto:gregd2350@gmail.com> <mailto:gregd2350@gmail.com <mailto:gregd2350@gmail.com>>> wrote: >>>>>> >>>>>> But, I'm not getting much love out of the v2 server. The connection doesn't appear to be working - "server response is incomplete". Same error whether on wifi or modem. >>>>> >>>> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> >>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev >> > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Maybe we could get mongoose to accept a pull request to add a flag telling mg_send() to always send immediately? -- Steve
I suspect that they will say ‘use MG_EV_SEND to send the next chunk’. Although I seem to remember seeing a fork of mongoose that offered a ‘send immediately’ style flag. It is certainly a common optimisation in such event driven systems. The connection flag approach seems the best. We could make a openvehicles/mongoose clone of their repository, and make our change there. That at least would be more maintainable. Regards, Mark.
On 22 Jan 2018, at 1:27 PM, Stephen Casner <casner@acm.org> wrote:
Maybe we could get mongoose to accept a pull request to add a flag telling mg_send() to always send immediately?
-- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
More issues: a) changing log levels only works without specifying a tag b) getting frequent logs like these: I (8324105) ovms-server-v2: Server response is incomplete (0 bytes) E (8324105) ovms-server-v2: Status: Error: Server response is incomplete …resulting in a server reconnect. V (8415335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8415335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... V (8415335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8415345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8415355) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53 V (8415355) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,, V (8415355) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9 | ,,,,,,N*53.... V (8415355) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8415355) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8415355) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53 V (8416335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8416335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... V (8416335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8416345) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53 V (8416345) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,, V (8416345) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9 | ,,,,,,N*53.... V (8416345) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8416345) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53 *I (8416515) ovms-server-v2: Server response is incomplete (0 bytes)* V (8416515) gsm-ppp: tx: 7e 21 45 00 00 28 00 6c 00 00 ff 06 e5 3c 0a aa | ~!E..(.l.....<.. V (8416515) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d6 3f f5 | ....K..@....M.?. V (8416515) gsm-ppp: tx: f4 17 50 11 16 2a 38 8b 00 00 b9 4c 7e | ..P..*8....L~ V (8416515) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6c 00 00 ff 06 | ...[~!E..(.l.... V (8416515) simcom: tx: e5 3c 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .<......K..@.... V (8416515) simcom: tx: 4d d6 3f f5 f4 17 50 11 16 2a 38 8b 00 00 b9 4c | M.?...P..*8....L V (8416515) simcom: tx: 7e 45 f9 | ~E. *E (8416525) ovms-server-v2: Status: Error: Server response is incomplete* V (8417315) simcom: rx: f9 09 ff 5b 7e 21 45 00 00 28 b7 32 40 00 2d 06 | ...[~!E..(.2@.-. V (8417315) simcom: rx: c0 76 bc 8a 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 | .v..K........@?. V (8417315) simcom: rx: f4 17 00 00 4d d7 50 11 39 08 15 ac 00 00 f2 89 | ....M.P.9....... V (8417325) simcom: rx: 7e 45 f9 | ~E. V (8417325) gsm-mux: ProcessFrame(CHAN=2, ADDR=09, CTRL=ff, FCS=45, LEN=51) V (8417325) gsm-mux: ChanProcessFrame(CHAN=2, ADDR=09, CTRL=ff, LEN=48, IFP=3) V (8417325) gsm-ppp: rx: 7e 21 45 00 00 28 b7 32 40 00 2d 06 c0 76 bc 8a | ~!E..(.2@.-..v.. V (8417325) gsm-ppp: rx: 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 f4 17 00 00 | K........@?..... V (8417335) gsm-ppp: rx: 4d d7 50 11 39 08 15 ac 00 00 f2 89 7e | M.P.9.......~ V (8417335) gsm-ppp: tx: 7e 21 45 00 00 28 00 6d 00 00 ff 06 e5 3b 0a aa | ~!E..(.m.....;.. V (8417335) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d7 3f f5 | ....K..@....M.?. V (8417335) gsm-ppp: tx: f4 18 50 10 16 29 38 8b 00 00 a9 af 7e | ..P..)8.....~ V (8417335) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6d 00 00 ff 06 | ...[~!E..(.m.... V (8417335) simcom: tx: e5 3b 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .;......K..@.... V (8417335) simcom: tx: 4d d7 3f f5 f4 18 50 10 16 29 38 8b 00 00 a9 af | M.?...P..)8..... V (8417335) simcom: tx: 7e 45 f9 | ~E. V (8417345) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8417345) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: > > @Steve can you update our master clone to latest from Espressif and > make sure your stuff is still ok? I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
> I guess steps for developers will be: > > 1) pull the openvehicles IDF and switch to master branch. Sub module update. > > 2) download and install xtensa build chain to match. > > 3) update OVMS master, make clean, check menu config, then build and play. Sounds right.
> I can get my part done by Sunday 14th night (HKT), so suggest to do > the Merge of the OVMS firmware master branch then. I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
On Mon, 15 Jan 2018, Michael Balzer wrote:
More issues:
a) changing log levels only works without specifying a tag
There was a bug/limitation in release/v2.1 that changing the log level for a tag did not have any effect if some log messages had already been emitted for that tag. This was because the code created a cache of tags and did not update the cache when the function to set the level was called. I added code to update the cache. When I went to merge my changes from v2.1 to v3.0 that change hit a conflict. It looked like there was already code present that should update the cache, although I did not study it carefully. OK, I looked at it more closely. There is code with a comment "search in the cache and update it if exist", but that code is doing a fast search where it is just comparing the address of the tag string to what it has cached, not the value of the string. This does not work because the cached tag string comes from the source file where the logs are generated, but the tag in the level setting command comes from argv in ovms_command.cpp. A workaround is that the cache is cleared when you set a level without specifying a tag, then you can set the level for the desired tag after that. I can change the code in the OS again and try issuing a PR to see if they will pick it up. -- Steve
I’m looking into the ‘server response is incomplete’ message now. Seems something broken in the socket layer / memory corruption. Regards, Mark.
On 16 Jan 2018, at 5:41 AM, Michael Balzer <dexter@expeedo.de> wrote:
More issues:
a) changing log levels only works without specifying a tag
b) getting frequent logs like these:
I (8324105) ovms-server-v2: Server response is incomplete (0 bytes) E (8324105) ovms-server-v2: Status: Error: Server response is incomplete
…resulting in a server reconnect.
V (8415335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8415335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... V (8415335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8415345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8415355) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53 V (8415355) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,, V (8415355) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9 | ,,,,,,N*53.... V (8415355) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8415355) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8415355) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53 V (8416335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8416335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... V (8416335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8416345) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53 V (8416345) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,, V (8416345) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9 | ,,,,,,N*53.... V (8416345) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8416345) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53 I (8416515) ovms-server-v2: Server response is incomplete (0 bytes) V (8416515) gsm-ppp: tx: 7e 21 45 00 00 28 00 6c 00 00 ff 06 e5 3c 0a aa | ~!E..(.l.....<.. V (8416515) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d6 3f f5 | ....K..@....M.?. V (8416515) gsm-ppp: tx: f4 17 50 11 16 2a 38 8b 00 00 b9 4c 7e | ..P..*8....L~ V (8416515) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6c 00 00 ff 06 | ...[~!E..(.l.... V (8416515) simcom: tx: e5 3c 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .<......K..@.... V (8416515) simcom: tx: 4d d6 3f f5 f4 17 50 11 16 2a 38 8b 00 00 b9 4c | M.?...P..*8....L V (8416515) simcom: tx: 7e 45 f9 | ~E. E (8416525) ovms-server-v2: Status: Error: Server response is incomplete V (8417315) simcom: rx: f9 09 ff 5b 7e 21 45 00 00 28 b7 32 40 00 2d 06 | ...[~!E..(.2@.-. V (8417315) simcom: rx: c0 76 bc 8a 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 | .v..K........@?. V (8417315) simcom: rx: f4 17 00 00 4d d7 50 11 39 08 15 ac 00 00 f2 89 | ....M.P.9....... V (8417325) simcom: rx: 7e 45 f9 | ~E. V (8417325) gsm-mux: ProcessFrame(CHAN=2, ADDR=09, CTRL=ff, FCS=45, LEN=51) V (8417325) gsm-mux: ChanProcessFrame(CHAN=2, ADDR=09, CTRL=ff, LEN=48, IFP=3) V (8417325) gsm-ppp: rx: 7e 21 45 00 00 28 b7 32 40 00 2d 06 c0 76 bc 8a | ~!E..(.2@.-..v.. V (8417325) gsm-ppp: rx: 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 f4 17 00 00 | K........@?..... V (8417335) gsm-ppp: rx: 4d d7 50 11 39 08 15 ac 00 00 f2 89 7e | M.P.9.......~ V (8417335) gsm-ppp: tx: 7e 21 45 00 00 28 00 6d 00 00 ff 06 e5 3b 0a aa | ~!E..(.m.....;.. V (8417335) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d7 3f f5 | ....K..@....M.?. V (8417335) gsm-ppp: tx: f4 18 50 10 16 29 38 8b 00 00 a9 af 7e | ..P..)8.....~ V (8417335) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6d 00 00 ff 06 | ...[~!E..(.m.... V (8417335) simcom: tx: e5 3b 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .;......K..@.... V (8417335) simcom: tx: 4d d7 3f f5 f4 18 50 10 16 29 38 8b 00 00 a9 af | M.?...P..)8..... V (8417335) simcom: tx: 7e 45 f9 | ~E. V (8417345) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8417345) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53.....
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
> On Fri, 12 Jan 2018, Stephen Casner wrote: > >> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >> >> @Steve can you update our master clone to latest from Espressif and >> make sure your stuff is still ok? > I have just now done this. I had already done a rebase a few days ago > before committing the improved version of the OS changes so I could > issue a pull request. There were a few commits since then, which I > have now merged. My code still runs correctly. > >> I guess steps for developers will be: >> >> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >> >> 2) download and install xtensa build chain to match. >> >> 3) update OVMS master, make clean, check menu config, then build and play. > Sounds right. > >> I can get my part done by Sunday 14th night (HKT), so suggest to do >> the Merge of the OVMS firmware master branch then. > I'll be sleeping then, so you can go ahead and do the merge. > > -- Steve _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
This is bizarre: Modifying the standard hello_world template App, to add: #include <sys/socket.h> int sock = socket(AF_INET, SOCK_STREAM, 0); printf("Allocated socket #%d\n",sock); if (sock >= 0) close(sock); I get this output when compiling against ESP IDF v3.0 / v3.1: Hello world! This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 1, 16MB external flash Allocated socket #4096 When compiling against ESP-IDF v2.1, I get: Hello world! This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 1, 16MB external flash Allocated socket #0 This messes up the select() call, using: FD_ZERO(&fds); FD_SET(sock,&fds); This commit in lwip component: commit 90bf40587ed6335f6ea3d343c3b3a43cac3a7c53 Merge: 6e7dd596 539262b5 Author: Angus Gratton <angus@espressif.com> Date: Tue Oct 17 14:17:09 2017 +0800 Merge branch 'feature/sockets_files_shared_fd_space' into 'master' lwip & vfs: POSIX I/O functions operate on sockets and files (first stage, no select() yet) See merge request !1352 It looks like Espressif have decided to change the way file descriptors work in IDF 3.x. Without telling anyone they’ve added an offset to indicate which component ‘owns’ the file descriptor. In their lwip code they seem to have redefined FD_SET, etc, to cope with that offset. But, the standard sys/select.h hasn’t changed so using that generates invalid file descriptor sets. They seem to have done this for their VFS code (so tcp/ip FDs can be mounted on VFS). But, they could simply have extended the socket structure to indicate which VFS component owned it. Why screw with such a fundamental thing as a file descriptor? The offsets seem to depend on vfs component instantiation order; which means that they are non-deterministic. I get 4096 in my test app, and 8192 in ovms v3 code. I’ll try to find a workaround solution. It seems to work for mongoose, so perhaps try to do it the way they do... Regards, Mark.
On 16 Jan 2018, at 9:19 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
I’m looking into the ‘server response is incomplete’ message now. Seems something broken in the socket layer / memory corruption.
Regards, Mark.
On 16 Jan 2018, at 5:41 AM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
More issues:
a) changing log levels only works without specifying a tag
b) getting frequent logs like these:
I (8324105) ovms-server-v2: Server response is incomplete (0 bytes) E (8324105) ovms-server-v2: Status: Error: Server response is incomplete
…resulting in a server reconnect.
V (8415335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8415335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... V (8415335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8415345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8415355) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53 V (8415355) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,, V (8415355) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9 | ,,,,,,N*53.... V (8415355) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8415355) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8415355) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53 V (8416335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8416335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53..... V (8416335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8416345) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53 V (8416345) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,, V (8416345) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9 | ,,,,,,N*53.... V (8416345) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31) V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3) V (8416345) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53 I (8416515) ovms-server-v2: Server response is incomplete (0 bytes) V (8416515) gsm-ppp: tx: 7e 21 45 00 00 28 00 6c 00 00 ff 06 e5 3c 0a aa | ~!E..(.l.....<.. V (8416515) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d6 3f f5 | ....K..@....M.?. V (8416515) gsm-ppp: tx: f4 17 50 11 16 2a 38 8b 00 00 b9 4c 7e | ..P..*8....L~ V (8416515) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6c 00 00 ff 06 | ...[~!E..(.l.... V (8416515) simcom: tx: e5 3c 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .<......K..@.... V (8416515) simcom: tx: 4d d6 3f f5 f4 17 50 11 16 2a 38 8b 00 00 b9 4c | M.?...P..*8....L V (8416515) simcom: tx: 7e 45 f9 | ~E. E (8416525) ovms-server-v2: Status: Error: Server response is incomplete V (8417315) simcom: rx: f9 09 ff 5b 7e 21 45 00 00 28 b7 32 40 00 2d 06 | ...[~!E..(.2@.-. V (8417315) simcom: rx: c0 76 bc 8a 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 | .v..K........@?. V (8417315) simcom: rx: f4 17 00 00 4d d7 50 11 39 08 15 ac 00 00 f2 89 | ....M.P.9....... V (8417325) simcom: rx: 7e 45 f9 | ~E. V (8417325) gsm-mux: ProcessFrame(CHAN=2, ADDR=09, CTRL=ff, FCS=45, LEN=51) V (8417325) gsm-mux: ChanProcessFrame(CHAN=2, ADDR=09, CTRL=ff, LEN=48, IFP=3) V (8417325) gsm-ppp: rx: 7e 21 45 00 00 28 b7 32 40 00 2d 06 c0 76 bc 8a | ~!E..(.2@.-..v.. V (8417325) gsm-ppp: rx: 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 f4 17 00 00 | K........@?..... V (8417335) gsm-ppp: rx: 4d d7 50 11 39 08 15 ac 00 00 f2 89 7e | M.P.9.......~ V (8417335) gsm-ppp: tx: 7e 21 45 00 00 28 00 6d 00 00 ff 06 e5 3b 0a aa | ~!E..(.m.....;.. V (8417335) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d7 3f f5 | ....K..@....M.?. V (8417335) gsm-ppp: tx: f4 18 50 10 16 29 38 8b 00 00 a9 af 7e | ..P..)8.....~ V (8417335) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6d 00 00 ff 06 | ...[~!E..(.m.... V (8417335) simcom: tx: e5 3b 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .;......K..@.... V (8417335) simcom: tx: 4d d7 3f f5 f4 18 50 10 16 29 38 8b 00 00 a9 af | M.?...P..)8..... V (8417335) simcom: tx: 7e 45 f9 | ~E. V (8417345) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,, V (8417345) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53.....
Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.
And…
OVMS > sd status SD CARD is inserted Name: SL32G Type: SDHC/SDXC Speed: default speed Size: 30436MB CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9 SCR: sd_spec=2, bus_width=5
→ me too (the minion) :)
BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.
After loading just the Twizy module & the v2 server, I'm now again down to…
OVMS > module memory Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0
…so it crashes as soon as it needs to actually do anything, like sending a notification.
This is without any of the new CAN logging stuff yet, that's stashed away for the transition.
I'll now check the config again for any more options to save RAM…
Regards, Michael
Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to work, the configuration needs to be changed from the sdk.default.hw30 that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled. I have attached a diff for the sdkconfig. That diff also includes a section for enabling telnet, which some of you may be using.
Please also note that in this revised implementation of "module memory" the block address is now the address that is returned from malloc, not the address of the header that precedes it, and the length of the block is now exclusive of the debugging overhead.
-- Steve
On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
> On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> <mailto:casner@acm.org> wrote: > > Mark, > > It looks like you were not able to do the sdkconfig.default and merge > as you planned? > > -- Steve > >> On Fri, 12 Jan 2018, Stephen Casner wrote: >> >>> On Sat, 13 Jan 2018, Mark Webb-Johnson wrote: >>> >>> @Steve can you update our master clone to latest from Espressif and >>> make sure your stuff is still ok? >> I have just now done this. I had already done a rebase a few days ago >> before committing the improved version of the OS changes so I could >> issue a pull request. There were a few commits since then, which I >> have now merged. My code still runs correctly. >> >>> I guess steps for developers will be: >>> >>> 1) pull the openvehicles IDF and switch to master branch. Sub module update. >>> >>> 2) download and install xtensa build chain to match. >>> >>> 3) update OVMS master, make clean, check menu config, then build and play. >> Sounds right. >> >>> I can get my part done by Sunday 14th night (HKT), so suggest to do >>> the Merge of the OVMS firmware master branch then. >> I'll be sleeping then, so you can go ahead and do the merge. >> >> -- Steve > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> > http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev> _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Sorry, no idea. So much has changed in IDF 2.x -> 3.x, and the matching compiler toolchain. Regards, Mark.
On 16 Jan 2018, at 2:10 AM, Greg D. <gregd2350@gmail.com> wrote:
Hi Mark,
Before I do the upgrade... Do you know if the new ESP code base changes the operation of the interrupt service, or provides any new capabilities in that regard? I'm still working on the CAN 2/3 bus hang, and don't want to shift things, timing wise, making the issue "go away" without actually being solved. On the other hand, perhaps they've accidentally fixed whatever is we're having trouble with, or perhaps have the ability to do things differently with the chip.
Thanks,
Greg
Mark Webb-Johnson wrote:
All committed and pushed now.
Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).
Check https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolch... <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain> to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.
I hope this goes smoothly for you.
Regards, Mark.
commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD) Author: Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> Date: Mon Jan 15 07:56:45 2018 +0800
Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware
commit 003592d553c881f735fb228b08836a13c306cec3 Merge: e58c5b6 b28db5e Author: Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> Date: Mon Jan 15 07:49:13 2018 +0800
Merge branch 'for-master'. This requires ESP-IDF v3.0 support:
OVMS developers should now:
1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.
2] Update XTENSA tools to match version required by Espressif for MASTER branch.
3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has been provided. That can be merged/copied to sdkconfig as appropriate.
4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will been provided in the next update.
5] It is recommended that developers perform at least one full 'make flash' with this version, to update the bootloader to latest. Note that the sdkconfig.default.* files are now set to require bootloader at least v2.1 (or later), and bootloader v2.0 is no longer supported.
Enjoy.
On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote:
Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.
Regards, Mark
On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org <mailto:casner@acm.org>> wrote:
Mark,
It looks like you were not able to do the sdkconfig.default and merge as you planned?
-- Steve
On Fri, 12 Jan 2018, Stephen Casner wrote:
On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:
@Steve can you update our master clone to latest from Espressif and make sure your stuff is still ok?
I have just now done this. I had already done a rebase a few days ago before committing the improved version of the OS changes so I could issue a pull request. There were a few commits since then, which I have now merged. My code still runs correctly.
I guess steps for developers will be:
1) pull the openvehicles IDF and switch to master branch. Sub module update.
2) download and install xtensa build chain to match.
3) update OVMS master, make clean, check menu config, then build and play.
Sounds right.
I can get my part done by Sunday 14th night (HKT), so suggest to do the Merge of the OVMS firmware master branch then.
I'll be sleeping then, so you can go ahead and do the merge.
-- Steve
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
participants (5)
-
Geir Øyvind Vælidalo -
Greg D. -
Mark Webb-Johnson -
Michael Balzer -
Stephen Casner