Everyone, with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each. So we've now got 178,976 bytes = ~4% left. Options beyond the 4 MB limit: a) split features, e.g. vehicle support, into two or more builds b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA c) switch to an ESP32 WROOM module with 32 MB flash (possible?) We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level). Unsurprisingly the webserver is on top, even with all assets precompressed already. *_Top 10 components:_* Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294 *_Vehicles:_* Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098 Regards, Michael Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Hi Michael, It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that. For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes. Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it). All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough. As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB. Just my 2 cents Carsten Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>: FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level). Unsurprisingly the webserver is on top, even with all assets precompressed already. Top 10 components: Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294 Vehicles: Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098 Regards, Michael Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: Everyone, with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each. So we've now got 178,976 bytes = ~4% left. Options beyond the 4 MB limit: a) split features, e.g. vehicle support, into two or more builds b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA c) switch to an ESP32 WROOM module with 32 MB flash (possible?) We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness. Regards, Michael _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com<mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926 _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Michael, Carsten, I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB. We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported. I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves). Looking at the partition table: # OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be: Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB. That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process. But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support). Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
This is the growth over a selection of main releases: Date Size Version 2018-04-08 2100352 3.1.003/ovms3.bin 2018-04-17 2111536 3.1.004/ovms3.bin 2018-05-01 2172176 3.1.005/ovms3.bin 2018-05-20 2247760 3.1.006/ovms3.bin 2018-06-18 2311072 3.1.007/ovms3.bin 2018-08-18 2386640 3.1.009/ovms3.bin 2018-11-04 2503744 3.1.011/ovms3.bin 2019-05-12 2933840 3.2.002/ovms3.bin 2020-08-04 2943440 3.2.014/ovms3.bin 2020-09-02 2957216 3.2.015/ovms3.bin 2021-03-05 3189808 3.2.016/ovms3.bin 2021-09-29 3309536 3.2.017/ovms3.bin 2021-11-22 3318000 3.2.018/ovms3.bin 2021-11-22 3367696 3.3.001/ovms3.bin 2022-03-07 3396208 3.3.002/ovms3.bin 2022-09-03 3406816 3.3.003/ovms3.bin 2024-03-23 3675344 3.3.004/ovms3.bin 2025-07-18 3895696 3.3.005/ovms3.bin So extending the partition size to 6 MB will buy us quite some time. Regarding Javascript/DBC vehicles, Michael Geddes' work on this is quite advanced, and really needs some tester now trying to implement an actual vehicle, to see what's missing. Regards, Michael Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
1. Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. 2. Provide a migration tool for partition table: 1. Copy running code to factory (from ota_0 or ota_1, whichever is current). 2. Reboot 3. Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size 4. Change boot to ota_0. 5. Reboot 3. Liase with factory so new modules use new partition format (and ship with firmware that supports it). 4. Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
One of the things missing is the bms averaging stuff. I started it but a house move and stuff got in the way... but yeah if someone wants to try I'll get onto that bit! //. On Tue, 9 Dec 2025, 01:04 Michael Balzer via OvmsDev, < ovmsdev@lists.openvehicles.com> wrote:
This is the growth over a selection of main releases:
Date Size Version 2018-04-08 2100352 3.1.003/ovms3.bin 2018-04-17 2111536 3.1.004/ovms3.bin 2018-05-01 2172176 3.1.005/ovms3.bin 2018-05-20 2247760 3.1.006/ovms3.bin 2018-06-18 2311072 3.1.007/ovms3.bin 2018-08-18 2386640 3.1.009/ovms3.bin 2018-11-04 2503744 3.1.011/ovms3.bin 2019-05-12 2933840 3.2.002/ovms3.bin 2020-08-04 2943440 3.2.014/ovms3.bin 2020-09-02 2957216 3.2.015/ovms3.bin 2021-03-05 3189808 3.2.016/ovms3.bin 2021-09-29 3309536 3.2.017/ovms3.bin 2021-11-22 3318000 3.2.018/ovms3.bin 2021-11-22 3367696 3.3.001/ovms3.bin 2022-03-07 3396208 3.3.002/ovms3.bin 2022-09-03 3406816 3.3.003/ovms3.bin 2024-03-23 3675344 3.3.004/ovms3.bin 2025-07-18 3895696 3.3.005/ovms3.bin
So extending the partition size to 6 MB will buy us quite some time.
Regarding Javascript/DBC vehicles, Michael Geddes' work on this is quite advanced, and really needs some tester now trying to implement an actual vehicle, to see what's missing.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
1. Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. 2. Provide a migration tool for partition table: 1. Copy running code to factory (from ota_0 or ota_1, whichever is current). 2. Reboot 3. Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size 4. Change boot to ota_0. 5. Reboot 3. Liase with factory so new modules use new partition format (and ship with firmware that supports it). 4. Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
*Top 10 components:*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*Vehicles:*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing listOvmsDev@lists.openvehicles.comhttp://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke <https://www.google.com/maps/search/Am+Rahmen+5+*+D-58313+Herdecke?entry=gmail&source=g> Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing listOvmsDev@lists.openvehicles.comhttp://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke <https://www.google.com/maps/search/Am+Rahmen+5+*+D-58313+Herdecke?entry=gmail&source=g> Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue. If we're about to go that way, here is an implementation of the process: * Explanation: https://johnmu.com/2024-esp32-partition-update/ * Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us. The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash. Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→ https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000. The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector. Regards, Michael Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
1. Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. 2. Provide a migration tool for partition table: 1. Copy running code to factory (from ota_0 or ota_1, whichever is current). 2. Reboot 3. Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size 4. Change boot to ota_0. 5. Reboot 3. Liase with factory so new modules use new partition format (and ship with firmware that supports it). 4. Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Michael, The links are very helpful. I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds). real 0m24.642s user 5m50.376s sys 0m38.276s We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes. Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
I’ve made some progress with this: OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13 OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass ## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot. One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed. The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks. The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with. I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week. Regards, Mark. P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Awesome :-) We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help. The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-) Regards, Michael Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real0m24.642s user5m50.376s sys0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process:
* Explanation:https://johnmu.com/2024-esp32-partition-update/ * Source:https://github.com/softplus/Esp32Repartition
It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
1. Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. 2. Provide a migration tool for partition table: 1. Copy running code to factory (from ota_0 or ota_1, whichever is current). 2. Reboot 3. Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size 4. Change boot to ota_0. 5. Reboot 3. Liase with factory so new modules use new partition format (and ship with firmware that supports it). 4. Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev<ovmsdev@lists.openvehicles.com>wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev<ovmsdev@lists.openvehicles.com>:
FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: > Everyone, > > with the latest vehicle additions, the firmware size has now > grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. > > Our flash partitioning scheme is currently designed to provide > three firmware partitions (factory, ota_0 & ota_1) of 4MB = > 4,194,304 bytes each. > > So we've now got 178,976 bytes = ~4% left. > > Options beyond the 4 MB limit: > > a) split features, e.g. vehicle support, into two or more builds > > b) repartition into two firmware partitions of 6 MB each, > reusing the factory partition for OTA > > c) switch to an ESP32 WROOM module with 32 MB flash (possible?) > > We've got some time left, new vehicles normally don't need that > much space, I just wanted to raise awareness. > > Regards, > Michael > > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OK, I have just committed this. The two new commands are: ota partitions list ota partitions upgrade [-noconfirm] Rollback is via: wget http://api.openvehicles.com/firmware/ota/v3.1/main/partitions.bin esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin General approach to manual upgrade: Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) Copy running ota firmware to factory Set to boot from flash and reboot Use ‘ota partitions upgrade’ to upgrade Reboot Items still to be addressed: Documentation (including rollback procedure). Modify partitions.csv to use this new format (when ready for production). Web interface (in particular concept of factory vs ota), if necessary (haven’t checked this). OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The ‘ota flash’ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot). For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed. Regards, Mark. P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
On Feb 23, 2026, at 12:27 AM, Michael Balzer <dexter@expeedo.de> wrote:
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
> Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>: > > FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level). > > Unsurprisingly the webserver is on top, even with all assets precompressed already. > > > Top 10 components: > > Per-archive contributions to ELF file: > Archive File DRAM .data & .bss IRAM Flash code & rodata Total > libovms_webserver.a 0 255 0 134342 399846 534443 > libstdc++.a 149 5640 0 141045 72513 219347 > libmain.a 15 2104 0 139216 40086 181421 > libduktape.a 0 0 0 141641 20367 162008 > libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 > libc-psram-workaround.a 1854 66 18391 118283 10943 149537 > liblwip.a 17 3873 0 118366 16722 138978 > libnet80211.a 938 9042 10475 92339 21900 134694 > libmbedtls.a 100 560 76 107079 26785 134600 > libvehicle_vweup.a 8 8 0 60846 43432 104294 > > > Vehicles: > > Per-archive contributions to ELF file: > Archive File DRAM .data & .bss IRAM Flash code & rodata Total > libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 > libvehicle_vweup.a 8 8 0 60846 43432 104294 > libvehicle_mgev.a 156 26 0 47756 33998 81936 > libvehicle_smarteq.a 82 15 0 59267 19527 78891 > libvehicle_smarted.a 0 9 0 48481 28801 77291 > libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 > libvehicle.a 0 68 0 57735 12062 69865 > libvehicle_bmwi3.a 0 2 0 33370 14357 47729 > libvehicle_minise.a 9432 2 0 35360 2210 47004 > libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 > libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 > libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 > libvehicle_kianiroev.a 108 7 0 24070 4006 28191 > libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 > libvehicle_boltev.a 0 5 0 15999 7864 23868 > libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 > libvehicle_maxus_edelive 156 3 0 10713 7477 18349 > libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 > libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 > libvehicle_voltampera.a 0 5 0 13221 1932 15158 > libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 > libvehicle_teslaroadster 0 6 0 10367 2238 12611 > libvehicle_thinkcity.a 0 3 0 6114 3449 9566 > libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 > libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 > libvehicle_teslamodels.a 0 2 0 5361 948 6311 > libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 > libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 > libvehicle_byd_atto3.a 0 2 0 3503 947 4452 > libvehicle_energica.a 0 1 0 3319 880 4200 > libvehicle_demo.a 0 2 0 3205 795 4002 > libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 > libvehicle_fiat500.a 0 2 0 2838 732 3572 > libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 > libvehicle_mercedesb250e 0 2 0 2113 955 3070 > libvehicle_zeva.a 0 2 0 2209 688 2899 > libvehicle_dbc.a 0 2 0 1278 1395 2675 > libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 > libvehicle_maple60s.a 0 2 0 1416 698 2116 > libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 > libvehicle_obdii.a 0 2 0 957 1007 1966 > libvehicle_teslamodel3.a 0 2 0 458 713 1173 > libvehicle_none.a 0 2 0 418 684 1104 > libvehicle_track.a 0 2 0 416 680 1098 > > > Regards, > Michael > > > Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: >> Everyone, >> >> with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. >> >> Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each. >> >> So we've now got 178,976 bytes = ~4% left. >> >> Options beyond the 4 MB limit: >> >> a) split features, e.g. vehicle support, into two or more builds >> >> b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA >> >> c) switch to an ESP32 WROOM module with 32 MB flash (possible?) >> >> We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness. >> >> Regards, >> Michael >> >> >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > -- > Michael Balzer * Am Rahmen 5 * D-58313 Herdecke > Fon 02330 9104094 * Handy 0176 20698926 > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Tested, works like a charm -- test log below. Only thing that was a bit odd: "ota" still showed the old layout directly after "ota partitions upgrade" (before the reboot). Probably cached, "ota partitions list" showed the new layout. I then tried rebooting into the old build to see if doing an OTA update from there would show any issues -- nope, works perfectly, installs to the other OTA partition and boots from there. So even after the upgrade, people can still run older firmware versions and perform normal OTA updates, the partitions are identified and located correctly by their label. Only thing odd with that was: "ota boot factory" failed correctly (did nothing), but did so without any error output. Not really a situation that will occur with normal users, so not really an issue. AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure. I can take care of the web UI modifications if you like. Regarding the product version 3.5 to distinguish 6MB capable modules, I think that's a neat & simple solution.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
I see what you mean… I wasn't aware there is so much space left unused now. Like this, if partition alignment constraints allow? Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7.46875 MB (0x778000 bytes) ota_1 app ota_1 0x00788000 7.46875 MB(0x778000 bytes) store data fat 0x00f00000 1 MB OK, that's substantially more. Another option would be to allocate say 7 MB to each OTA partition (or even keep them at 6 MB) and add the remaining free capacity to store. Performing a config restore currently already fails due to lack of space when not done from a freshly cleared config, especially with some scripts & plugins installed. With custom JS vehicles being installed to /store, that will get a bit more tight yet, and maybe JS vehicles want to use /store for some extended data storage as well. So more capacity for /store would make sense, heading in that direction. Regards, Michael OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98 OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass OVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98 OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass OVMS# module reset Resetting system... OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/ota_0/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-12 (ota1, ota2, no factory) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-711-g4feca695d OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass OVMS# ota boot ? Usage: ota boot ota_0|ota_1 ota_0 Boot from ota_0 image ota_1 Boot from ota_1 image OVMS# ota copy ? Usage: ota copy ota_0|ota_1 ota_0 OTA copy ota_0 <to> ota_1 OTA copy ota_1 <to> OVMS# ota erase ? Usage: ota erase ota_0|ota_1 ota_0 Erase ota_0 image ota_1 Erase ota_1 image Am 24.02.26 um 13:40 schrieb Mark Webb-Johnson:
OK, I have just committed this. The two new commands are:
* ota partitions list * ota partitions upgrade [-noconfirm]
Rollback is via:
* wget http://api.openvehicles.com/firmware/ota/v3.1/main/partitions.bin * esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
1. Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) 2. Copy running ota firmware to factory 3. Set to boot from flash and reboot 4. Use ‘ota partitions upgrade’ to upgrade 5. Reboot
Items still to be addressed:
* Documentation (including rollback procedure). * Modify partitions.csv to use this new format (when ready for production). * Web interface (in particular concept of factory vs ota), if necessary (haven’t checked this). * OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The ‘ota flash’ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). * Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
Regards, Mark.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
On Feb 23, 2026, at 12:27 AM, Michael Balzer <dexter@expeedo.de> wrote:
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real0m24.642s user5m50.376s sys0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process:
* Explanation:https://johnmu.com/2024-esp32-partition-update/ * Source:https://github.com/softplus/Esp32Repartition
It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
1. Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. 2. Provide a migration tool for partition table: 1. Copy running code to factory (from ota_0 or ota_1, whichever is current). 2. Reboot 3. Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size 4. Change boot to ota_0. 5. Reboot 3. Liase with factory so new modules use new partition format (and ship with firmware that supports it). 4. Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
> On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via > OvmsDev<ovmsdev@lists.openvehicles.com>wrote: > > Hi Michael, > > It was to be expected that we would eventually run out of flash > storage. That’s why I would immediately question whether we > should detach ourselves from components that aren’t really being > used. We could even start a survey or something like that. > > For example, the RE tools — the idea behind them is great, but > without documentation they’re not easy to use, and every time I > tried to work with them, it just resulted in crashes. > > Then there’s the question of who actually uses Telnet, SSH, the > Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the > Twizy integration is the only one that uses it). > > All great ideas, but in the end, how many users really make use > of them? > The fewer active components we have to maintain, the ‘easier’ it > would also be to port the code to a more current ESP-IDF version > — and that would bring significant benefits such as improved > networking features, including firewall capabilities and a much > more stable switching between LTE and Wi-Fi. I’ve looked at > llanges attempts and it’s extremely tough. > > As an example, in my own custom firmware builds, I don’t enable > the components (and of course not all vehicles, so not 100 > percent representative) mentioned above. This requires small > modifications in the code because, for example, the ESP logger > is missing but referenced in another file, etc. But my firmware > file is only about 2.8 MB. > > Just my 2 cents > Carsten > >> Am 07.12.2025 um 08:57 schrieb Michael Balzer via >> OvmsDev<ovmsdev@lists.openvehicles.com>: >> >> FYI: use `make size-components` to create a report on all >> component sizes (`make size-files` for source file level). >> >> Unsurprisingly the webserver is on top, even with all assets >> precompressed already. >> >> >> *_Top 10 components:_* >> >> Per-archive contributions to ELF file: >> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >> libovms_webserver.a 0 255 0 134342 399846 534443 >> libstdc++.a 149 5640 0 141045 72513 219347 >> libmain.a 15 2104 0 139216 40086 181421 >> libduktape.a 0 0 0 141641 20367 162008 >> libvehicle_renaulttwizy. 0 29 0 86517 >> 75357 161903 >> libc-psram-workaround.a 1854 66 18391 118283 >> 10943 149537 >> liblwip.a 17 3873 0 118366 16722 138978 >> libnet80211.a 938 9042 10475 92339 21900 134694 >> libmbedtls.a 100 560 76 107079 26785 134600 >> libvehicle_vweup.a 8 8 0 60846 43432 104294 >> >> >> *_Vehicles:_* >> >> Per-archive contributions to ELF file: >> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >> libvehicle_renaulttwizy. 0 29 0 86517 >> 75357 161903 >> libvehicle_vweup.a 8 8 0 60846 43432 104294 >> libvehicle_mgev.a 156 26 0 47756 33998 81936 >> libvehicle_smarteq.a 82 15 0 59267 19527 78891 >> libvehicle_smarted.a 0 9 0 48481 28801 77291 >> libvehicle_renaultzoe_ph 4 10 0 44622 >> 29564 74200 >> libvehicle.a 0 68 0 57735 12062 69865 >> libvehicle_bmwi3.a 0 2 0 33370 14357 47729 >> libvehicle_minise.a 9432 2 0 35360 2210 47004 >> libvehicle_hyundai_ioniq 176 7 0 32921 >> 13425 46529 >> libvehicle_nissanleaf.a 0 3 0 35491 >> 8941 44435 >> libvehicle_kiasoulev.a 240 9 0 32508 5473 >> 38230 >> libvehicle_kianiroev.a 108 7 0 24070 4006 >> 28191 >> libvehicle_mitsubishi.a 0 5 0 21645 >> 3893 25543 >> libvehicle_boltev.a 0 5 0 15999 7864 23868 >> libvehicle_niu_gtevo.a 4 12 0 18248 3733 >> 21997 >> libvehicle_maxus_edelive 156 3 0 10713 >> 7477 18349 >> libvehicle_renaultzoe.a 0 6 0 14828 >> 2859 17693 >> libvehicle_maxus_euniq56 156 3 0 8363 >> 6840 15362 >> libvehicle_voltampera.a 0 5 0 13221 >> 1932 15158 >> libvehicle_hyundai_ioniq 0 3 0 11081 >> 3191 14275 >> libvehicle_teslaroadster 0 6 0 10367 >> 2238 12611 >> libvehicle_thinkcity.a 0 3 0 6114 3449 >> 9566 >> libvehicle_jaguaripace.a 0 8 0 5445 >> 3941 9394 >> libvehicle_fiatedoblo.a 0 2 0 4262 >> 2126 6390 >> libvehicle_teslamodels.a 0 2 0 5361 >> 948 6311 >> libvehicle_toyotarav4ev. 0 2 0 5023 >> 1255 6280 >> libvehicle_maxus_t90.a 0 3 0 2567 2204 >> 4774 >> libvehicle_byd_atto3.a 0 2 0 3503 947 >> 4452 >> libvehicle_energica.a 0 1 0 3319 880 4200 >> libvehicle_demo.a 0 2 0 3205 795 4002 >> libvehicle_maxus_euniq6. 0 2 0 2470 >> 1182 3654 >> libvehicle_fiat500.a 0 2 0 2838 732 3572 >> libvehicle_zombie_vcu.a 0 4 0 1882 >> 1259 3145 >> libvehicle_mercedesb250e 0 2 0 2113 >> 955 3070 >> libvehicle_zeva.a 0 2 0 2209 688 2899 >> libvehicle_dbc.a 0 2 0 1278 1395 2675 >> libvehicle_cadillac_c2_c 0 7 0 1293 >> 1105 2405 >> libvehicle_maple60s.a 0 2 0 1416 698 2116 >> libvehicle_chevrolet_c6_ 0 2 0 1053 >> 1049 2104 >> libvehicle_obdii.a 0 2 0 957 1007 1966 >> libvehicle_teslamodel3.a 0 2 0 458 >> 713 1173 >> libvehicle_none.a 0 2 0 418 684 1104 >> libvehicle_track.a 0 2 0 416 680 1098 >> >> >> Regards, >> Michael >> >> >> Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: >>> Everyone, >>> >>> with the latest vehicle additions, the firmware size has now >>> grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. >>> >>> Our flash partitioning scheme is currently designed to provide >>> three firmware partitions (factory, ota_0 & ota_1) of 4MB = >>> 4,194,304 bytes each. >>> >>> So we've now got 178,976 bytes = ~4% left. >>> >>> Options beyond the 4 MB limit: >>> >>> a) split features, e.g. vehicle support, into two or more builds >>> >>> b) repartition into two firmware partitions of 6 MB each, >>> reusing the factory partition for OTA >>> >>> c) switch to an ESP32 WROOM module with 32 MB flash (possible?) >>> >>> We've got some time left, new vehicles normally don't need >>> that much space, I just wanted to raise awareness. >>> >>> Regards, >>> Michael >>> >>> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.openvehicles.com >>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >> >> -- >> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke >> Fon 02330 9104094 * Handy 0176 20698926 >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Michael,
Only thing that was a bit odd: "ota" still showed the old layout directly after "ota partitions upgrade" (before the reboot). Probably cached, "ota partitions list" showed the new layout.
This is a simple fix. I will change tonight.
Only thing odd with that was: "ota boot factory" failed correctly (did nothing), but did so without any error output. Not really a situation that will occur with normal users, so not really an issue.
I guess because it can no longer find the partition type ‘factory’. Both these edge cases would not occur if a reboot was done after the partition upgrade.
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working? const esp_partition_t *p = esp_ota_get_running_partition(); if ((p != NULL)&&(strcmp(p->label,"factory")!=0)) { writer->puts("Error: Cannot upgrade partition table if running partition is not factory"); return; } p = esp_ota_get_boot_partition(); if ((p != NULL)&&(strcmp(p->label,"factory")!=0)) { writer->puts("Error: Cannot upgrade partition table if boot partition is not factory"); return; }
I can take care of the web UI modifications if you like.
That would be helpful. I’m not really familiar with that code.
OK, that's substantially more.
Here are the three partition table options: Original (f12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Current (12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Suggested (12, 7MB code, more store): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616 It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked. Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade I suspect #3 would be the most complex step. Or the store move and resize could be done first as a separate step. Thoughts? Much more complex and risky... Regards, Mark
On 25 Feb 2026, at 6:00 AM, Michael Balzer <dexter@expeedo.de> wrote:
Tested, works like a charm -- test log below.
Only thing that was a bit odd: "ota" still showed the old layout directly after "ota partitions upgrade" (before the reboot). Probably cached, "ota partitions list" showed the new layout.
I then tried rebooting into the old build to see if doing an OTA update from there would show any issues -- nope, works perfectly, installs to the other OTA partition and boots from there. So even after the upgrade, people can still run older firmware versions and perform normal OTA updates, the partitions are identified and located correctly by their label.
Only thing odd with that was: "ota boot factory" failed correctly (did nothing), but did so without any error output. Not really a situation that will occur with normal users, so not really an issue.
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I can take care of the web UI modifications if you like.
Regarding the product version 3.5 to distinguish 6MB capable modules, I think that's a neat & simple solution.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
I see what you mean… I wasn't aware there is so much space left unused now. Like this, if partition alignment constraints allow?
Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7.46875 MB (0x778000 bytes) ota_1 app ota_1 0x00788000 7.46875 MB (0x778000 bytes) store data fat 0x00f00000 1 MB
OK, that's substantially more.
Another option would be to allocate say 7 MB to each OTA partition (or even keep them at 6 MB) and add the remaining free capacity to store.
Performing a config restore currently already fails due to lack of space when not done from a freshly cleared config, especially with some scripts & plugins installed.
With custom JS vehicles being installed to /store, that will get a bit more tight yet, and maybe JS vehicles want to use /store for some extended data storage as well. So more capacity for /store would make sense, heading in that direction.
Regards, Michael
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
OVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
OVMS# module reset Resetting system...
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/ota_0/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-12 (ota1, ota2, no factory) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-711-g4feca695d
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
OVMS# ota boot ? Usage: ota boot ota_0|ota_1 ota_0 Boot from ota_0 image ota_1 Boot from ota_1 image
OVMS# ota copy ? Usage: ota copy ota_0|ota_1 ota_0 OTA copy ota_0 <to> ota_1 OTA copy ota_1 <to>
OVMS# ota erase ? Usage: ota erase ota_0|ota_1 ota_0 Erase ota_0 image ota_1 Erase ota_1 image
Am 24.02.26 um 13:40 schrieb Mark Webb-Johnson:
OK, I have just committed this. The two new commands are:
ota partitions list ota partitions upgrade [-noconfirm]
Rollback is via:
wget http://api.openvehicles.com/firmware/ota/v3.1/main/partitions.bin esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) Copy running ota firmware to factory Set to boot from flash and reboot Use ‘ota partitions upgrade’ to upgrade Reboot
Items still to be addressed:
Documentation (including rollback procedure). Modify partitions.csv to use this new format (when ready for production). Web interface (in particular concept of factory vs ota), if necessary (haven’t checked this). OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The ‘ota flash’ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
Regards, Mark.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
On Feb 23, 2026, at 12:27 AM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of `spi_flash_erase_range()` & `spi_flash_write()` (→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even need `getPartitionTableAddr()`, as our table is fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev: > Michael, Carsten, > > I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB. > > We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported. > > I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves). > > Looking at the partition table: > > # OVMS 16MB flash ESP32 Partition Table > # Name, Type, SubType, Offset, Size > nvs, data, nvs, 0x9000, 0x4000 > otadata, data, ota, 0xd000, 0x2000 > phy_init, data, phy, 0xf000, 0x1000 > factory, app, factory, 0x10000, 4M > ota_0, app, ota_0, , 4M > ota_1, app, ota_1, , 4M > store, data, fat, , 1M > > Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be: > > Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. > Provide a migration tool for partition table: > Copy running code to factory (from ota_0 or ota_1, whichever is current). > Reboot > Change partition table (most likely replacing the entire table with a binary blob of the new format) > factory 4MB -> ota_0 same offset, 6MB size > ota_0 -> ota_1 6MB offset, 6MB size > Change boot to ota_0. > Reboot > Liase with factory so new modules use new partition format (and ship with firmware that supports it). > Wait a reasonable time for users to update before releasing any firmware > 4MB. > > That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process. > > But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support). > > Regards, Mark. > >> On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote: >> >> Hi Michael, >> >> It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that. >> >> For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes. >> >> Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it). >> >> All great ideas, but in the end, how many users really make use of them? >> The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough. >> >> As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB. >> >> Just my 2 cents >> Carsten >> >>> Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>: >>> >>> FYI: use `make size-components` to create a report on all component sizes (`make size-files` for source file level). >>> >>> Unsurprisingly the webserver is on top, even with all assets precompressed already. >>> >>> >>> Top 10 components: >>> >>> Per-archive contributions to ELF file: >>> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >>> libovms_webserver.a 0 255 0 134342 399846 534443 >>> libstdc++.a 149 5640 0 141045 72513 219347 >>> libmain.a 15 2104 0 139216 40086 181421 >>> libduktape.a 0 0 0 141641 20367 162008 >>> libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 >>> libc-psram-workaround.a 1854 66 18391 118283 10943 149537 >>> liblwip.a 17 3873 0 118366 16722 138978 >>> libnet80211.a 938 9042 10475 92339 21900 134694 >>> libmbedtls.a 100 560 76 107079 26785 134600 >>> libvehicle_vweup.a 8 8 0 60846 43432 104294 >>> >>> >>> Vehicles: >>> >>> Per-archive contributions to ELF file: >>> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >>> libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 >>> libvehicle_vweup.a 8 8 0 60846 43432 104294 >>> libvehicle_mgev.a 156 26 0 47756 33998 81936 >>> libvehicle_smarteq.a 82 15 0 59267 19527 78891 >>> libvehicle_smarted.a 0 9 0 48481 28801 77291 >>> libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 >>> libvehicle.a 0 68 0 57735 12062 69865 >>> libvehicle_bmwi3.a 0 2 0 33370 14357 47729 >>> libvehicle_minise.a 9432 2 0 35360 2210 47004 >>> libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 >>> libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 >>> libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 >>> libvehicle_kianiroev.a 108 7 0 24070 4006 28191 >>> libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 >>> libvehicle_boltev.a 0 5 0 15999 7864 23868 >>> libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 >>> libvehicle_maxus_edelive 156 3 0 10713 7477 18349 >>> libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 >>> libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 >>> libvehicle_voltampera.a 0 5 0 13221 1932 15158 >>> libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 >>> libvehicle_teslaroadster 0 6 0 10367 2238 12611 >>> libvehicle_thinkcity.a 0 3 0 6114 3449 9566 >>> libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 >>> libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 >>> libvehicle_teslamodels.a 0 2 0 5361 948 6311 >>> libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 >>> libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 >>> libvehicle_byd_atto3.a 0 2 0 3503 947 4452 >>> libvehicle_energica.a 0 1 0 3319 880 4200 >>> libvehicle_demo.a 0 2 0 3205 795 4002 >>> libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 >>> libvehicle_fiat500.a 0 2 0 2838 732 3572 >>> libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 >>> libvehicle_mercedesb250e 0 2 0 2113 955 3070 >>> libvehicle_zeva.a 0 2 0 2209 688 2899 >>> libvehicle_dbc.a 0 2 0 1278 1395 2675 >>> libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 >>> libvehicle_maple60s.a 0 2 0 1416 698 2116 >>> libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 >>> libvehicle_obdii.a 0 2 0 957 1007 1966 >>> libvehicle_teslamodel3.a 0 2 0 458 713 1173 >>> libvehicle_none.a 0 2 0 418 684 1104 >>> libvehicle_track.a 0 2 0 416 680 1098 >>> >>> >>> Regards, >>> Michael >>> >>> >>> Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: >>>> Everyone, >>>> >>>> with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. >>>> >>>> Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each. >>>> >>>> So we've now got 178,976 bytes = ~4% left. >>>> >>>> Options beyond the 4 MB limit: >>>> >>>> a) split features, e.g. vehicle support, into two or more builds >>>> >>>> b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA >>>> >>>> c) switch to an ESP32 WROOM module with 32 MB flash (possible?) >>>> >>>> We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness. >>>> >>>> Regards, >>>> Michael >>>> >>>> >>>> >>>> _______________________________________________ >>>> OvmsDev mailing list >>>> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >>> >>> -- >>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke >>> Fon 02330 9104094 * Handy 0176 20698926 >>> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
@Craig: nice find! Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
1. Create a 2nd fat store partition (0xE10000-0xFFFFFF) 2. Format and mount that as FAT 3. Copy files from old to new 4. Unmount and drop the old store 5. Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way. I second the store resize should be the first upgrade step: Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step. I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Craig, Michael, I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed. I also experimented with store. I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible). If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up. So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level). Thoughts? Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Mark, I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip). The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout: backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } }; We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary. But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage. Regards, Michael Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
* I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
* If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
1. Create a 2nd fat store partition (0xE10000-0xFFFFFF) 2. Format and mount that as FAT 3. Copy files from old to new 4. Unmount and drop the old store 5. Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Thinking about the web UI: I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots). The UI will provide info & links on each step, what to do if something fails. So the process will now become this? 1. Copy running ota image to "factory", reboot into "factory" 2. Upgrade store partition, reboot 3. Upgrade OTA partitions, reboot If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot? Regards, Michael Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
* I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
* If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
1. Create a 2nd fat store partition (0xE10000-0xFFFFFF) 2. Format and mount that as FAT 3. Copy files from old to new 4. Unmount and drop the old store 5. Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Michael, I think you are correct. The safest way of doing this is to split as you suggest: For #1, I think it is done now and should be ok. For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot. Then #3 will need just a small tweak on the partition sizes to complete it. Trying to do it all in one command may be too complex. I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed). I think I can provide a rollback if store partition is not as expected following reboot. I suggest: ota partition upgrade store ota partition downgrade store ota partition upgrade factory I’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy. In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools. Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this? Copy running ota image to "factory", reboot into "factory" Upgrade store partition, reboot Upgrade OTA partitions, reboot If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Mark, Am 26.02.26 um 01:47 schrieb Mark Webb-Johnson:
My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
That could be avoided: `OvmsConfig::mount()` could try mounting "store-v35" (or whatever the new name is) first, then fall back to "store". Same scheme can be applied to `module_perform_factoryreset()`. That way "store" doesn't need to be relabeled at all, the system automatically uses the new partition if available. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Am 26.02.26 um 21:51 schrieb Michael Balzer via OvmsDev:
Mark,
Am 26.02.26 um 01:47 schrieb Mark Webb-Johnson:
My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
That could be avoided: `OvmsConfig::mount()` could try mounting "store-v35" (or whatever the new name is) first, then fall back to "store".
Same scheme can be applied to `module_perform_factoryreset()`.
That way "store" doesn't need to be relabeled at all, the system automatically uses the new partition if available.
Btw, we also don't need a store rollback command then, as "store" remains intact until step #3 does the final update of the partition table. Step #3 would then also rename "store-v35" to "store", to recover compatibility with older firmware builds.
Regards, Michael
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
I'm working on support for a new car. I've been capturing two kinds of can logs, one where the car doesn't move (importantly the odometer doesn't change) and the other where I drive somewhere (odometer and tpms values change). I've noticed it can be difficult to obtain a can log for the latter case. For example today I tried to create a can log for the trip to pick up lunch. Before starting the car (or even waking it up) I started a can log and verified it it was active: OVMS# can log start vfs crtd /sd/ct5-13.crtd CAN logging to VFS active: Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd OVMS# can log status #1: open Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd Size:149 Messages:1 Dropped:0 Filtered:0 Rate:0.0% /sd/ct5-13.crtd: running Size:149 Messages:1 Discarded:0 Dropped:0 Filtered:0 Rate:0.0% When I arrive at my destination, I turned the car off and then used the ios ovms app to check the log: can log status CAN logging inactive So my can log was no longer active. When I got back home I ssh'ed into the module and found that the log was present but with a length of zero and a bogus timestamp: OVMS# vfs ls /sd [DIR] 05-Jul-2019 19:48 backup/ 0 30-Nov-1979 00:00 ct5-10.crtd 75.8M 30-Mar-2026 11:51 ct5-11.crtd 35.5M 30-Mar-2026 21:17 ct5-12.crtd 0 30-Nov-1979 00:00 ct5-13.crtd [DIR] 05-Jul-2019 19:48 logs/ [DIR] 29-Mar-2026 09:58 scripts/ In cases where I have successfully captured a can log while driving I used the app to stop logging *before* turning the car off. But every time I've tried where I've turned the car off before stopping the can log has failed. My expectation is that changing ms_v_env_on should not impact can logging. This feels like a bug but I allow that there might be something happening here that I do not understand. It also appears that nothing new is written to the normal log file while can logging is active; is that expected? Craig P.S. I configured opendkim to not sign messages to ovmsdev@lists.openvehicles.com
I did a quick test with CAN logging & regular logging enabled while charging, and found no issues. Both files get written correctly. I didn't test driving, but the CAN logging system does not use vehicle metrics or events in any special way. The file remaining at 0 bytes and unset time hints at a crash during your drive (before stopping the logger). Did you rule that out? If so, do you have some plugins or scripts bound to events, that may interfere with logging? Don't expect the file meta data to change while the logger is running, other than the system logging, the CAN logging deliberately does no `fsync()` calls to keep running at maximum speed. Regards, Michael Am 02.04.26 um 22:25 schrieb Craig Leres via OvmsDev:
I'm working on support for a new car. I've been capturing two kinds of can logs, one where the car doesn't move (importantly the odometer doesn't change) and the other where I drive somewhere (odometer and tpms values change).
I've noticed it can be difficult to obtain a can log for the latter case. For example today I tried to create a can log for the trip to pick up lunch. Before starting the car (or even waking it up) I started a can log and verified it it was active:
OVMS# can log start vfs crtd /sd/ct5-13.crtd CAN logging to VFS active: Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd OVMS# can log status #1: open Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd Size:149 Messages:1 Dropped:0 Filtered:0 Rate:0.0% /sd/ct5-13.crtd: running Size:149 Messages:1 Discarded:0 Dropped:0 Filtered:0 Rate:0.0%
When I arrive at my destination, I turned the car off and then used the ios ovms app to check the log:
can log status CAN logging inactive
So my can log was no longer active.
When I got back home I ssh'ed into the module and found that the log was present but with a length of zero and a bogus timestamp:
OVMS# vfs ls /sd [DIR] 05-Jul-2019 19:48 backup/ 0 30-Nov-1979 00:00 ct5-10.crtd 75.8M 30-Mar-2026 11:51 ct5-11.crtd 35.5M 30-Mar-2026 21:17 ct5-12.crtd 0 30-Nov-1979 00:00 ct5-13.crtd [DIR] 05-Jul-2019 19:48 logs/ [DIR] 29-Mar-2026 09:58 scripts/
In cases where I have successfully captured a can log while driving I used the app to stop logging *before* turning the car off. But every time I've tried where I've turned the car off before stopping the can log has failed.
My expectation is that changing ms_v_env_on should not impact can logging. This feels like a bug but I allow that there might be something happening here that I do not understand.
It also appears that nothing new is written to the normal log file while can logging is active; is that expected?
Craig
P.S. I configured opendkim to not sign messages to ovmsdev@lists.openvehicles.com
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/3/26 02:43, Michael Balzer via OvmsDev wrote:
I did a quick test with CAN logging & regular logging enabled while charging, and found no issues. Both files get written correctly.
I didn't test driving, but the CAN logging system does not use vehicle metrics or events in any special way.
The file remaining at 0 bytes and unset time hints at a crash during your drive (before stopping the logger). Did you rule that out?
I don't know how to check module uptime but I know it logs the vin as soon as it sees it after booting and the most recent vin log was a couple of days before the logging attempt.
If so, do you have some plugins or scripts bound to events, that may interfere with logging?
No and /sd/scripts and /store/scripts are empty.
Don't expect the file meta data to change while the logger is running, other than the system logging, the CAN logging deliberately does no `fsync()` calls to keep running at maximum speed.
Right, I knew about that. The longest can capture I have is 29 minutes long and averages over 900 frames per second. Could my problem be that the can data rate is just too high? Looking at the log more closely I see it was having trouble connecting to the v2 server between 1:23am and 10:07am: 2026-04-02 01:23:31.847 PDT W (141523227) ovms-server-v2: Connection failed 2026-04-02 01:23:31.847 PDT E (141523227) ovms-server-v2: Status: Error: Connection failed 2026-04-02 01:23:31.857 PDT E (141523237) ovms-server-v2: Status: Connection lost, reconnecting [...] 2026-04-02 10:07:01.827 PDT W (172933207) ovms-server-v2: Connection failed 2026-04-02 10:07:01.827 PDT E (172933207) ovms-server-v2: Status: Error: Connection failed 2026-04-02 10:07:01.837 PDT E (172933217) ovms-server-v2: Status: Connection lost, reconnecting Perhaps unrelated. Craig
Am 03.04.26 um 19:06 schrieb Craig Leres:
On 4/3/26 02:43, Michael Balzer via OvmsDev wrote:
I did a quick test with CAN logging & regular logging enabled while charging, and found no issues. Both files get written correctly.
I didn't test driving, but the CAN logging system does not use vehicle metrics or events in any special way.
The file remaining at 0 bytes and unset time hints at a crash during your drive (before stopping the logger). Did you rule that out?
I don't know how to check module uptime but I know it logs the vin as soon as it sees it after booting and the most recent vin log was a couple of days before the logging attempt.
The command "boot" shows uptime and last crash details, if any: OVMS# boot Last boot was 3644073 second(s) ago Time at boot: 2026-02-20 13:57:09 CET This is reset #0 since last power cycle Detected boot reason: PowerOn (1/14) Reset reason: Unknown/unset (0) Crash counters: 0 total, 0 early
If so, do you have some plugins or scripts bound to events, that may interfere with logging?
No and /sd/scripts and /store/scripts are empty.
Don't expect the file meta data to change while the logger is running, other than the system logging, the CAN logging deliberately does no `fsync()` calls to keep running at maximum speed.
Right, I knew about that.
The longest can capture I have is 29 minutes long and averages over 900 frames per second. Could my problem be that the can data rate is just too high?
Try filtering then, 900 frames per second is rather high. Assuming most of the frames are 8 bytes in length, that rate amounts to ~45 kB per second = 82 MB in 30 minutes. My largest CRTD recording archived is 52 MB, with 1083482 frames in 51 minutes = averaging at ~ 350 frames per second (recorded January 2022 from my Seat Mii's Comfort CAN). I don't remember the module state besides the CAN recording, but I assume I had regular file logging paused to save SD bandwidth. Filtering will tell if the data rate or file size is causing the issue. Regards, Michael
Looking at the log more closely I see it was having trouble connecting to the v2 server between 1:23am and 10:07am:
2026-04-02 01:23:31.847 PDT W (141523227) ovms-server-v2: Connection failed 2026-04-02 01:23:31.847 PDT E (141523227) ovms-server-v2: Status: Error: Connection failed 2026-04-02 01:23:31.857 PDT E (141523237) ovms-server-v2: Status: Connection lost, reconnecting [...] 2026-04-02 10:07:01.827 PDT W (172933207) ovms-server-v2: Connection failed 2026-04-02 10:07:01.827 PDT E (172933207) ovms-server-v2: Status: Error: Connection failed 2026-04-02 10:07:01.837 PDT E (172933217) ovms-server-v2: Status: Connection lost, reconnecting
Perhaps unrelated.
Craig
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/3/26 10:28, Michael Balzer wrote:
The command "boot" shows uptime and last crash details, if any:
OVMS# boot Last boot was 3644073 second(s) ago Time at boot: 2026-02-20 13:57:09 CET This is reset #0 since last power cycle Detected boot reason: PowerOn (1/14) Reset reason: Unknown/unset (0) Crash counters: 0 total, 0 early
I don't believe I'm crashing: OVMS# boot Last boot was 88094 second(s) ago Time at boot: 2026-04-03 10:14:11 PDT This is reset #6 since last power cycle Detected boot reason: SoftReset (12/12) Reset reason: esp_restart (3) Crash counters: 0 total, 0 early I think my only restarts are when I reboot to test a new local version.
The longest can capture I have is 29 minutes long and averages over 900 frames per second. Could my problem be that the can data rate is just too high?
Try filtering then, 900 frames per second is rather high.
Assuming most of the frames are 8 bytes in length, that rate amounts to ~45 kB per second = 82 MB in 30 minutes.
The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero... I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch? Craig OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting OVMS# can log start vfs crtd /sd/ct5-14.crtd '034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380' CAN logging to VFS active: Type:vfs Format:crtd(discard) Filter:034-3f Vehicle:CT5 Path:/sd/ct5-14.crtd
Am 04.04.26 um 20:02 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote:
Try filtering then, 900 frames per second is rather high. The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
The issue isn't the microRL constraint. `can_log_vfs_start()` & `can::AddLogger()` don't support multiple filters in a string, so you're limited by the maximum argument count as defined by the command registration, in this case 9 (see `OvmsCanLogVFSInit::OvmsCanLogVFSInit()`). You can raise that limit without adding RAM requirements, it's just an argc validation. If you do so, please copy to all log formats. Another option could be to add a negation syntax & function for the log filters, if that helps. Or to add a special filter option to skip all-zero frames. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/5/26 01:24, Michael Balzer wrote:
Am 04.04.26 um 20:02 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote:
Try filtering then, 900 frames per second is rather high. The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
The issue isn't the microRL constraint.
`can_log_vfs_start()` & `can::AddLogger()` don't support multiple filters in a string, so you're limited by the maximum argument count as defined by the command registration, in this case 9 (see `OvmsCanLogVFSInit::OvmsCanLogVFSInit()`).
(As I later discovered!)
You can raise that limit without adding RAM requirements, it's just an argc validation.
If you do so, please copy to all log formats.
Understood.
Another option could be to add a negation syntax & function for the log filters, if that helps. Or to add a special filter option to skip all- zero frames.
I don't think that helps, there are some pids that sometimes have all zero payload bytes (e.g. 0x063 which I use to determine "is running"): 1772821542.210018 1R11 063 00 00 00 00 00 00 1772821741.875479 1R11 063 00 00 00 f5 00 00 1772821683.149862 1R11 063 00 00 16 82 00 00 1772821685.590338 1R11 063 00 00 14 00 00 00 1772821741.976706 1R11 063 00 00 00 00 00 00 Appended is what I did locally for testing, raising _COMMAND_TOKEN_NMB impacts the stack use of several routines in microrl.c, I can't judge if that's a problem or not. Two other things I see: the usage message has way more info about the syntax than the user manual: "<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f", vs or log specific CAN packets by applying a filter e.g 0x55b the Nissan LEAF SoC CAN message ovms# can log start vfs crtd /sd/can.crtd 55b PR #1367 is my initial attempt at this. Also, validation of the filter arguments needs to be stricter, currently just about any junk string is "accepted" as a filter: OVMS# can log start vfs crtd /sd/ct5-14.crtd asdjfjkadsfj CAN logging to VFS active: Type:vfs Format:crtd(discard) Filter:00a-d Vehicle:CT5 Path:/sd/ct5-14.crtd Craig ======================================================================== sea 82 % git diff master diff --git a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp index 0fe74e80..b3194ca0 100644 --- a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp +++ b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp @@ -82,7 +82,7 @@ OvmsCanLogVFSInit::OvmsCanLogVFSInit() "<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f", - 1, 9, true, vfs_file_validate); + 1, _COMMAND_TOKEN_NMB, true, vfs_file_validate); } } } diff --git a/vehicle/OVMS.V3/components/microrl/microrl_config.h b/vehicle/OVMS.V3/components/microrl/microrl_config.h index 614b9614..e8b2a9c7 100644 --- a/vehicle/OVMS.V3/components/microrl/microrl_config.h +++ b/vehicle/OVMS.V3/components/microrl/microrl_config.h @@ -21,7 +21,7 @@ typed in command line exceed this value, then prints message about it and command line not to be parced and 'execute' callback will not calls. Token is word separate by white space, for example 3 token line: "IRin> set mode test" */ -#define _COMMAND_TOKEN_NMB 16 +#define _COMMAND_TOKEN_NMB 128 /* Define you prompt string here. You can use colors escape code, for highlight you prompt,
Seems fair enough to allow more filters. Yeah, the parsing is pretty permissive (using sscanf). Maybe just echoing back what was parsed might not be a bad idea? //.ichael On Mon, 6 Apr 2026 at 05:18, Craig Leres via OvmsDev < ovmsdev@lists.openvehicles.com> wrote:
On 4/5/26 01:24, Michael Balzer wrote:
Am 04.04.26 um 20:02 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote:
Try filtering then, 900 frames per second is rather high. The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
The issue isn't the microRL constraint.
`can_log_vfs_start()` & `can::AddLogger()` don't support multiple filters in a string, so you're limited by the maximum argument count as defined by the command registration, in this case 9 (see `OvmsCanLogVFSInit::OvmsCanLogVFSInit()`).
(As I later discovered!)
You can raise that limit without adding RAM requirements, it's just an argc validation.
If you do so, please copy to all log formats.
Understood.
Another option could be to add a negation syntax & function for the log filters, if that helps. Or to add a special filter option to skip all- zero frames.
I don't think that helps, there are some pids that sometimes have all zero payload bytes (e.g. 0x063 which I use to determine "is running"):
1772821542.210018 1R11 063 00 00 00 00 00 00 1772821741.875479 1R11 063 00 00 00 f5 00 00 1772821683.149862 1R11 063 00 00 16 82 00 00 1772821685.590338 1R11 063 00 00 14 00 00 00 1772821741.976706 1R11 063 00 00 00 00 00 00
Appended is what I did locally for testing, raising _COMMAND_TOKEN_NMB impacts the stack use of several routines in microrl.c, I can't judge if that's a problem or not.
Two other things I see: the usage message has way more info about the syntax than the user manual:
"<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f",
vs
or log specific CAN packets by applying a filter e.g 0x55b the Nissan LEAF SoC CAN message
ovms# can log start vfs crtd /sd/can.crtd 55b
PR #1367 is my initial attempt at this.
Also, validation of the filter arguments needs to be stricter, currently just about any junk string is "accepted" as a filter:
OVMS# can log start vfs crtd /sd/ct5-14.crtd asdjfjkadsfj CAN logging to VFS active: Type:vfs Format:crtd(discard) Filter:00a-d Vehicle:CT5 Path:/sd/ct5-14.crtd
Craig
======================================================================== sea 82 % git diff master diff --git a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp index 0fe74e80..b3194ca0 100644 --- a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp +++ b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp @@ -82,7 +82,7 @@ OvmsCanLogVFSInit::OvmsCanLogVFSInit() "<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f", - 1, 9, true, vfs_file_validate); + 1, _COMMAND_TOKEN_NMB, true, vfs_file_validate); } } } diff --git a/vehicle/OVMS.V3/components/microrl/microrl_config.h b/vehicle/OVMS.V3/components/microrl/microrl_config.h index 614b9614..e8b2a9c7 100644 --- a/vehicle/OVMS.V3/components/microrl/microrl_config.h +++ b/vehicle/OVMS.V3/components/microrl/microrl_config.h @@ -21,7 +21,7 @@ typed in command line exceed this value, then prints message about it and command line not to be parced and 'execute' callback will not calls. Token is word separate by white space, for example 3 token line: "IRin> set mode test" */ -#define _COMMAND_TOKEN_NMB 16 +#define _COMMAND_TOKEN_NMB 128
/* Define you prompt string here. You can use colors escape code, for highlight you prompt,
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Am 05.04.26 um 23:17 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote: You can raise that limit without adding RAM requirements, it's just an argc validation. Appended is what I did locally for testing, raising _COMMAND_TOKEN_NMB impacts the stack use of several routines in microrl.c, I can't judge if that's a problem or not.
Sorry, I didn't recognize you'd also need to raise _COMMAND_TOKEN_NMB for your filter. Raising _COMMAND_TOKEN_NMB from 16 to 128 has a significant impact on the microRL stack requirements, raising stack allocation for any command execution from 64 to 512 Bytes. That's not safe, as we cannot guarantee any context running e.g. BufferedShell commands has sufficient stack, and especially as some commands have high stack requirements themselves. So there are two options to solve this: a) Rework microRL's `microrl_get_complite()` and `new_line_handler()` to use the heap instead of the stack for the token arrays (and make sure the SPIRAM heap is used if available). b) Rework `canfilter::AddFilter()` to allow passing multiple filter specs in a string, so you can use a quoted argument. …or maybe both. I'd opt for (a) to begin with, as that has a general positive impact on stack usage. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/6/26 00:13, Michael Balzer wrote:
a) Rework microRL's `microrl_get_complite()` and `new_line_handler()` to use the heap instead of the stack for the token arrays (and make sure the SPIRAM heap is used if available).
Could you please point me to existing code that uses the heap in the way you suggest? I found some esp documentation: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/sy... When external RAM is enabled, external SPI RAM can be allocated using standard malloc calls, or via heap_caps_malloc(MALLOC_CAP_SPIRAM), depending on the configuration. See Configuring External RAM for more details. I also see the call to heap_caps_malloc() in components/zip/zlib/zutil.c: voidpf ZLIB_INTERNAL zcalloc (opaque, items, size) voidpf opaque; unsigned items; unsigned size; { (void)opaque; void* ret = heap_caps_malloc(items * size, MALLOC_CAP_SPIRAM); if (ret) return ret; else return malloc(items * size); } Is using calloc/malloc/realloc good enough? Craig
I tried 3.3.005-714-gbcf42700. Unfortunately the upgrade step results in a crash. OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-714-gbcf42700-dirty/factory/main (build idf v3.3.4-854-g9063c8662 Feb 25 2026 10:36:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-714-gbcf42700-dirty OTA_O image: 3.3.005-714-gbcf42700-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass OVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... sea 4 % ./backtrace.sh 0x400891b7:0x3ffebbd0 0x40089451:0x3ffebbf0 0x401c82cc:0x3ffebc10 0x40084cfa:0x3ffebc30 0x4018397d:0x3ffebc60 0x40180c03:0x3ffebcf0 0x40180c3f:0x3ffebd20 0x40146bdd:0x3ffebd40 0x40289911:0x3ffebd60 0x400e1091:0x3ffebd80 0x40137f55:0x3ffebe40 0x400e077d:0x3ffebe70 0x400e0a10:0x3ffebea0 0x400e0a43:0x3ffebf50 0x401566d1:0x3ffebf70 0x40157ed7:0x3ffebfb0 0x401582e1:0x3ffebfe0 0x40158593:0x3ffec050 0x40154aa2:0x3ffec0b0 0x4015a2fe:0x3ffec0d0 0x401442ec:0x3ffec100 0x40144415:0x3ffec160 + xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x400891b7:0x3ffebbd0 0x40089451:0x3ffebbf0 0x401c82cc:0x3ffebc10 0x40084cfa:0x3ffebc30 0x4018397d:0x3ffebc60 0x40180c03:0x3ffebcf0 0x40180c3f:0x3ffebd20 0x40146bdd:0x3ffebd40 0x40289911:0x3ffebd60 0x400e1091:0x3ffebd80 0x40137f55:0x3ffebe40 0x400e077d:0x3ffebe70 0x400e0a10:0x3ffebea0 0x400e0a43:0x3ffebf50 0x401566d1:0x3ffebf70 0x40157ed7:0x3ffebfb0 0x401582e1:0x3ffebfe0 0x40158593:0x3ffec050 0x40154aa2:0x3ffec0b0 0x4015a2fe:0x3ffec0d0 0x401442ec:0x3ffec100 0x40144415:0x3ffec160 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/spi_flash/flash_ops.c:124 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/spi_flash/flash_ops.c:220 (discriminator 2) components/ovms_ota/src/ovms_partitions.cpp:394 components/ovms_ota/src/ovms_ota.cpp:1088 components/ovms_ota/src/ovms_ota.cpp:1088 main/ovms_shell.cpp:67 main/ovms_shell.cpp:80 (discriminator 2) components/console_ssh/src/console_ssh.cpp:527 main/ovms_console.cpp:184 components/console_ssh/src/console_ssh.cpp:420 components/console_ssh/src/console_ssh.cpp:420 components/console_ssh/src/console_ssh.cpp:420 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/src/mongoose_client.cpp:52 main/ovms_netmanager.cpp:1000 main/ovms_netmanager.cpp:980
On 2/24/26 04:40, Mark Webb-Johnson via OvmsDev wrote:
OK, I have just committed this. The two new commands are:
* ota partitions list * ota partitions upgrade [-noconfirm]
Rollback is via:
* wget http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin <http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin> * esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
1. Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) 2. Copy running ota firmware to factory 3. Set to boot from flash and reboot 4. Use âota partitions upgradeâ to upgrade 5. Reboot
Items still to be addressed:
* Documentation (including rollback procedure). * Modify partitions.csv to use this new format (when ready for production). * Web interface (in particular concept of factory vs ota), if necessary (havenât checked this). * OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The âota flashâ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). * Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
I took a run at this with my dev module: OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695-dirty/ota_1/main (build idf v3.3.4-854-g9063c8662 Feb 24 2026 15:08:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: ota_1 Boot partition: ota_1 Factory image: 3.3.005-20-g60be058e OTA_O image: 3.3.005-700-g0a804e40-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota copy ota_1 factory OTA copy ota_1 (08454144) -> factory (00010000) size 4194304 Error: ESP32 error #258 starting OTA operation OVMS# I couldn't find 258 so I internet searched it and learned that it is ESP_ERR_NOT_SUPPORTED (from components/esp32/include/esp_err.h, might be worth displaying in hex given that's how it's defined). I checked and my esp-idf is up to date. Craig sea 12 % pwd /home/sea/u0/leres/esp/esp-idf sea 13 % git log | head commit 9063c8662ca5d67b5490c1503bd4377b380feed3 Author: Michael Balzer <balzer@expeedo.de> Date: Tue Jan 27 17:34:34 2026 +0100 Fix mutex priority inheritance bug in spi_flash component: backport of esp-idf commit aea2fe08162e1ba86ad0189178c1f1cf29238bbd, see https://github.com/espressif/esp-idf/issues/5116 and https://github.com/espressif/esp-idf/issues/7580 commit 75bdad1e05dbbfe2ef79e8398749ff6aaa7deb92 sea 17 % pwd /home/sea/u0/leres/src/OVMS.V3 sea 18 % git log | head commit 4feca695d922f26b5f3864fe43c6855b8f223675 Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Tue Feb 24 20:21:30 2026 +0800 Support for ota 0+1 partition table format, and 6MB firmware commit 1241912b48628e89947e1600f86365b4b174c6c8 Merge: 92f93554 141bad50 Author: Michael Balzer <dexter@dexters-web.de> Date: Sun Feb 22 15:57:13 2026 +0100
Craig, Thanks for discovering this. It is nasty. Error 258 is 0x102 and that is ESP_ERR_INVALID_ARG (not ESP_ERR_NOT_SUPPORTED). I find it esp_ota_begin (components/app_update/esp_ota_ops.c): if (!is_ota_partition(partition)) { return ESP_ERR_INVALID_ARG; } and that is: static bool is_ota_partition(const esp_partition_t *p) { return (p != NULL && p->type == ESP_PARTITION_TYPE_APP && p->subtype >= ESP_PARTITION_SUBTYPE_APP_OTA_0 && p->subtype < ESP_PARTITION_SUBTYPE_APP_OTA_MAX); } It seems we can’t target the factory partition with those ota operations. I guess the problem is the whole ota rollback thing (where if it can’t boot into an ota partition, it falls back to the previous one). It is not too onerous to workaround, though. We can simply change to use esp_partition_erase_range and esp_partition_write when targeting the factory. I will try to implement it that way tonight. Regards, Mark.
On 25 Feb 2026, at 7:34 AM, Craig Leres <leres@xse.com> wrote:
On 2/24/26 04:40, Mark Webb-Johnson via OvmsDev wrote:
OK, I have just committed this. The two new commands are: * ota partitions list * ota partitions upgrade [-noconfirm] Rollback is via: * wget http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin <http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin> * esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin General approach to manual upgrade: 1. Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) 2. Copy running ota firmware to factory 3. Set to boot from flash and reboot 4. Use âota partitions upgradeâ to upgrade 5. Reboot Items still to be addressed: * Documentation (including rollback procedure). * Modify partitions.csv to use this new format (when ready for production). * Web interface (in particular concept of factory vs ota), if necessary (havenât checked this). * OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The âota flashâ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). * Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot). For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
I took a run at this with my dev module:
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695-dirty/ota_1/main (build idf v3.3.4-854-g9063c8662 Feb 24 2026 15:08:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: ota_1 Boot partition: ota_1 Factory image: 3.3.005-20-g60be058e OTA_O image: 3.3.005-700-g0a804e40-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota copy ota_1 factory OTA copy ota_1 (08454144) -> factory (00010000) size 4194304 Error: ESP32 error #258 starting OTA operation OVMS#
I couldn't find 258 so I internet searched it and learned that it is ESP_ERR_NOT_SUPPORTED (from components/esp32/include/esp_err.h, might be worth displaying in hex given that's how it's defined).
I checked and my esp-idf is up to date.
Craig
sea 12 % pwd /home/sea/u0/leres/esp/esp-idf sea 13 % git log | head commit 9063c8662ca5d67b5490c1503bd4377b380feed3 Author: Michael Balzer <balzer@expeedo.de> Date: Tue Jan 27 17:34:34 2026 +0100
Fix mutex priority inheritance bug in spi_flash component: backport of esp-idf commit aea2fe08162e1ba86ad0189178c1f1cf29238bbd, see https://github.com/espressif/esp-idf/issues/5116 and https://github.com/espressif/esp-idf/issues/7580
commit 75bdad1e05dbbfe2ef79e8398749ff6aaa7deb92 sea 17 % pwd /home/sea/u0/leres/src/OVMS.V3 sea 18 % git log | head commit 4feca695d922f26b5f3864fe43c6855b8f223675 Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Tue Feb 24 20:21:30 2026 +0800
Support for ota 0+1 partition table format, and 6MB firmware
commit 1241912b48628e89947e1600f86365b4b174c6c8 Merge: 92f93554 141bad50 Author: Michael Balzer <dexter@dexters-web.de> Date: Sun Feb 22 15:57:13 2026 +0100
participants (5)
-
Carsten Schmiemann -
Craig Leres -
Mark Webb-Johnson -
Michael Balzer -
Michael Geddes