Firmware size approaching 4 MB limit
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
FYI: use make size-components to create a report on all component
sizes (make size-files for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use make size-components to create a report on all component sizes (make size-files for source file level).
Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com<mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
OVMS 16MB flash ESP32 Partition Table
Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
This is the growth over a selection of main releases:
Date Size Version 2018-04-08 2100352 3.1.003/ovms3.bin 2018-04-17 2111536 3.1.004/ovms3.bin 2018-05-01 2172176 3.1.005/ovms3.bin 2018-05-20 2247760 3.1.006/ovms3.bin 2018-06-18 2311072 3.1.007/ovms3.bin 2018-08-18 2386640 3.1.009/ovms3.bin 2018-11-04 2503744 3.1.011/ovms3.bin 2019-05-12 2933840 3.2.002/ovms3.bin 2020-08-04 2943440 3.2.014/ovms3.bin 2020-09-02 2957216 3.2.015/ovms3.bin 2021-03-05 3189808 3.2.016/ovms3.bin 2021-09-29 3309536 3.2.017/ovms3.bin 2021-11-22 3318000 3.2.018/ovms3.bin 2021-11-22 3367696 3.3.001/ovms3.bin 2022-03-07 3396208 3.3.002/ovms3.bin 2022-09-03 3406816 3.3.003/ovms3.bin 2024-03-23 3675344 3.3.004/ovms3.bin 2025-07-18 3895696 3.3.005/ovms3.bin
So extending the partition size to 6 MB will buy us quite some time.
Regarding Javascript/DBC vehicles, Michael Geddes' work on this is quite advanced, and really needs some tester now trying to implement an actual vehicle, to see what's missing.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1MAssuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
- Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users.
- Provide a migration tool for partition table:
- Copy running code to factory (from ota_0 or ota_1, whichever is current).
- Reboot
- Change partition table (most likely replacing the entire table with a binary blob of the new format)
- factory 4MB -> ota_0 same offset, 6MB size
- ota_0 -> ota_1 6MB offset, 6MB size
- Change boot to ota_0.
- Reboot
- Liase with factory so new modules use new partition format (and ship with firmware that supports it).
- Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
One of the things missing is the bms averaging stuff. I started it but a house move and stuff got in the way... but yeah if someone wants to try I'll get onto that bit!
//.
On Tue, 9 Dec 2025, 01:04 Michael Balzer via OvmsDev, < ovmsdev@lists.openvehicles.com> wrote:
This is the growth over a selection of main releases:
Date Size Version 2018-04-08 2100352 3.1.003/ovms3.bin 2018-04-17 2111536 3.1.004/ovms3.bin 2018-05-01 2172176 3.1.005/ovms3.bin 2018-05-20 2247760 3.1.006/ovms3.bin 2018-06-18 2311072 3.1.007/ovms3.bin 2018-08-18 2386640 3.1.009/ovms3.bin 2018-11-04 2503744 3.1.011/ovms3.bin 2019-05-12 2933840 3.2.002/ovms3.bin 2020-08-04 2943440 3.2.014/ovms3.bin 2020-09-02 2957216 3.2.015/ovms3.bin 2021-03-05 3189808 3.2.016/ovms3.bin 2021-09-29 3309536 3.2.017/ovms3.bin 2021-11-22 3318000 3.2.018/ovms3.bin 2021-11-22 3367696 3.3.001/ovms3.bin 2022-03-07 3396208 3.3.002/ovms3.bin 2022-09-03 3406816 3.3.003/ovms3.bin 2024-03-23 3675344 3.3.004/ovms3.bin 2025-07-18 3895696 3.3.005/ovms3.bin
So extending the partition size to 6 MB will buy us quite some time.
Regarding Javascript/DBC vehicles, Michael Geddes' work on this is quite advanced, and really needs some tester now trying to implement an actual vehicle, to see what's missing.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
OVMS 16MB flash ESP32 Partition Table
Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
- Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users.
- Provide a migration tool for partition table:
- Copy running code to factory (from ota_0 or ota_1, whichever is current).
- Reboot
- Change partition table (most likely replacing the entire table with a binary blob of the new format)
- factory 4MB -> ota_0 same offset, 6MB size
- ota_0 -> ota_1 6MB offset, 6MB size
- Change boot to ota_0.
- Reboot
- Liase with factory so new modules use new partition format (and ship with firmware that supports it).
- Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
*Top 10 components:*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*Vehicles:*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing listOvmsDev@lists.openvehicles.comhttp://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke <https://www.google.com/maps/search/Am+Rahmen+5+*+D-58313+Herdecke?entry=gmail&source=g> Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing listOvmsDev@lists.openvehicles.comhttp://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke <https://www.google.com/maps/search/Am+Rahmen+5+*+D-58313+Herdecke?entry=gmail&source=g> Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process:
- Explanation: https://johnmu.com/2024-esp32-partition-update/
- Source: https://github.com/softplus/Esp32Repartition
It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a
matter of spi_flash_erase_range() & spi_flash_write() (→
https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297).
We probably don't even need getPartitionTableAddr(), as our table is
fixed at 0x8000.
The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1MAssuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
- Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users.
- Provide a migration tool for partition table:
- Copy running code to factory (from ota_0 or ota_1, whichever is current).
- Reboot
- Change partition table (most likely replacing the entire table with a binary blob of the new format)
- factory 4MB -> ota_0 same offset, 6MB size
- ota_0 -> ota_1 6MB offset, 6MB size
- Change boot to ota_0.
- Reboot
- Liase with factory so new modules use new partition format (and ship with firmware that supports it).
- Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of
spi_flash_erase_range()&spi_flash_write()(→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even needgetPartitionTableAddr(), as our table is fixed at 0x8000.The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
OVMS 16MB flash ESP32 Partition Table
Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot
The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of
spi_flash_erase_range()&spi_flash_write()(→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even needgetPartitionTableAddr(), as our table is fixed at 0x8000.The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
OVMS 16MB flash ESP32 Partition Table
Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
Top 10 components:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
Vehicles:
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev:
Everyone,
with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881.
Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each.
So we've now got 178,976 bytes = ~4% left.
Options beyond the 4 MB limit:
a) split features, e.g. vehicle support, into two or more builds
b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA
c) switch to an ESP32 WROOM module with 32 MB flash (possible?)
We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13 OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass ## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d passThis is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real0m24.642s user5m50.376s sys0m38.276sWe’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process:
- Explanation:https://johnmu.com/2024-esp32-partition-update/
- Source:https://github.com/softplus/Esp32Repartition
It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of
spi_flash_erase_range()&spi_flash_write()(→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even needgetPartitionTableAddr(), as our table is fixed at 0x8000.The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1MAssuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
- Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users.
- Provide a migration tool for partition table:
- Copy running code to factory (from ota_0 or ota_1, whichever is current).
- Reboot
- Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size
- Change boot to ota_0.
- Reboot
- Liase with factory so new modules use new partition format (and ship with firmware that supports it).
- Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev<ovmsdev@lists.openvehicles.com>wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev<ovmsdev@lists.openvehicles.com>:
FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level).Unsurprisingly the webserver is on top, even with all assets precompressed already.
*_Top 10 components:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libovms_webserver.a 0 255 0 134342 399846 534443 libstdc++.a 149 5640 0 141045 72513 219347 libmain.a 15 2104 0 139216 40086 181421 libduktape.a 0 0 0 141641 20367 162008 libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libc-psram-workaround.a 1854 66 18391 118283 10943 149537 liblwip.a 17 3873 0 118366 16722 138978 libnet80211.a 938 9042 10475 92339 21900 134694 libmbedtls.a 100 560 76 107079 26785 134600 libvehicle_vweup.a 8 8 0 60846 43432 104294
*_Vehicles:_*
Per-archive contributions to ELF file: Archive File DRAM .data & .bss IRAM Flash code & rodata Total libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 libvehicle_vweup.a 8 8 0 60846 43432 104294 libvehicle_mgev.a 156 26 0 47756 33998 81936 libvehicle_smarteq.a 82 15 0 59267 19527 78891 libvehicle_smarted.a 0 9 0 48481 28801 77291 libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 libvehicle.a 0 68 0 57735 12062 69865 libvehicle_bmwi3.a 0 2 0 33370 14357 47729 libvehicle_minise.a 9432 2 0 35360 2210 47004 libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 libvehicle_kianiroev.a 108 7 0 24070 4006 28191 libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 libvehicle_boltev.a 0 5 0 15999 7864 23868 libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 libvehicle_maxus_edelive 156 3 0 10713 7477 18349 libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 libvehicle_voltampera.a 0 5 0 13221 1932 15158 libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 libvehicle_teslaroadster 0 6 0 10367 2238 12611 libvehicle_thinkcity.a 0 3 0 6114 3449 9566 libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 libvehicle_teslamodels.a 0 2 0 5361 948 6311 libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 libvehicle_byd_atto3.a 0 2 0 3503 947 4452 libvehicle_energica.a 0 1 0 3319 880 4200 libvehicle_demo.a 0 2 0 3205 795 4002 libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 libvehicle_fiat500.a 0 2 0 2838 732 3572 libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 libvehicle_mercedesb250e 0 2 0 2113 955 3070 libvehicle_zeva.a 0 2 0 2209 688 2899 libvehicle_dbc.a 0 2 0 1278 1395 2675 libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 libvehicle_maple60s.a 0 2 0 1416 698 2116 libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 libvehicle_obdii.a 0 2 0 957 1007 1966 libvehicle_teslamodel3.a 0 2 0 458 713 1173 libvehicle_none.a 0 2 0 418 684 1104 libvehicle_track.a 0 2 0 416 680 1098
Regards, Michael
Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: > Everyone, > > with the latest vehicle additions, the firmware size has now > grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. > > Our flash partitioning scheme is currently designed to provide > three firmware partitions (factory, ota_0 & ota_1) of 4MB = > 4,194,304 bytes each. > > So we've now got 178,976 bytes = ~4% left. > > Options beyond the 4 MB limit: > > a) split features, e.g. vehicle support, into two or more builds > > b) repartition into two firmware partitions of 6 MB each, > reusing the factory partition for OTA > > c) switch to an ESP32 WROOM module with 32 MB flash (possible?) > > We've got some time left, new vehicles normally don't need that > much space, I just wanted to raise awareness. > > Regards, > Michael > > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OK, I have just committed this. The two new commands are:
ota partitions list ota partitions upgrade [-noconfirm]
Rollback is via:
wget http://api.openvehicles.com/firmware/ota/v3.1/main/partitions.bin esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) Copy running ota firmware to factory Set to boot from flash and reboot Use ‘ota partitions upgrade’ to upgrade Reboot
Items still to be addressed:
Documentation (including rollback procedure). Modify partitions.csv to use this new format (when ready for production). Web interface (in particular concept of factory vs ota), if necessary (haven’t checked this). OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The ‘ota flash’ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
Regards, Mark.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
On Feb 23, 2026, at 12:27 AM, Michael Balzer <dexter@expeedo.de> wrote:
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot
The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of
spi_flash_erase_range()&spi_flash_write()(→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even needgetPartitionTableAddr(), as our table is fixed at 0x8000.The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
OVMS 16MB flash ESP32 Partition Table
Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1M
Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. Provide a migration tool for partition table: Copy running code to factory (from ota_0 or ota_1, whichever is current). Reboot Change partition table (most likely replacing the entire table with a binary blob of the new format) factory 4MB -> ota_0 same offset, 6MB size ota_0 -> ota_1 6MB offset, 6MB size Change boot to ota_0. Reboot Liase with factory so new modules use new partition format (and ship with firmware that supports it). Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Hi Michael,
It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that.
For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes.
Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it).
All great ideas, but in the end, how many users really make use of them? The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough.
As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB.
Just my 2 cents Carsten
> Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>: > > FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level). > > Unsurprisingly the webserver is on top, even with all assets precompressed already. > > > Top 10 components: > > Per-archive contributions to ELF file: > Archive File DRAM .data & .bss IRAM Flash code & rodata Total > libovms_webserver.a 0 255 0 134342 399846 534443 > libstdc++.a 149 5640 0 141045 72513 219347 > libmain.a 15 2104 0 139216 40086 181421 > libduktape.a 0 0 0 141641 20367 162008 > libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 > libc-psram-workaround.a 1854 66 18391 118283 10943 149537 > liblwip.a 17 3873 0 118366 16722 138978 > libnet80211.a 938 9042 10475 92339 21900 134694 > libmbedtls.a 100 560 76 107079 26785 134600 > libvehicle_vweup.a 8 8 0 60846 43432 104294 > > > Vehicles: > > Per-archive contributions to ELF file: > Archive File DRAM .data & .bss IRAM Flash code & rodata Total > libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 > libvehicle_vweup.a 8 8 0 60846 43432 104294 > libvehicle_mgev.a 156 26 0 47756 33998 81936 > libvehicle_smarteq.a 82 15 0 59267 19527 78891 > libvehicle_smarted.a 0 9 0 48481 28801 77291 > libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 > libvehicle.a 0 68 0 57735 12062 69865 > libvehicle_bmwi3.a 0 2 0 33370 14357 47729 > libvehicle_minise.a 9432 2 0 35360 2210 47004 > libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 > libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 > libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 > libvehicle_kianiroev.a 108 7 0 24070 4006 28191 > libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 > libvehicle_boltev.a 0 5 0 15999 7864 23868 > libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 > libvehicle_maxus_edelive 156 3 0 10713 7477 18349 > libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 > libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 > libvehicle_voltampera.a 0 5 0 13221 1932 15158 > libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 > libvehicle_teslaroadster 0 6 0 10367 2238 12611 > libvehicle_thinkcity.a 0 3 0 6114 3449 9566 > libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 > libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 > libvehicle_teslamodels.a 0 2 0 5361 948 6311 > libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 > libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 > libvehicle_byd_atto3.a 0 2 0 3503 947 4452 > libvehicle_energica.a 0 1 0 3319 880 4200 > libvehicle_demo.a 0 2 0 3205 795 4002 > libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 > libvehicle_fiat500.a 0 2 0 2838 732 3572 > libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 > libvehicle_mercedesb250e 0 2 0 2113 955 3070 > libvehicle_zeva.a 0 2 0 2209 688 2899 > libvehicle_dbc.a 0 2 0 1278 1395 2675 > libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 > libvehicle_maple60s.a 0 2 0 1416 698 2116 > libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 > libvehicle_obdii.a 0 2 0 957 1007 1966 > libvehicle_teslamodel3.a 0 2 0 458 713 1173 > libvehicle_none.a 0 2 0 418 684 1104 > libvehicle_track.a 0 2 0 416 680 1098 > > > Regards, > Michael > > > Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: >> Everyone, >> >> with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. >> >> Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each. >> >> So we've now got 178,976 bytes = ~4% left. >> >> Options beyond the 4 MB limit: >> >> a) split features, e.g. vehicle support, into two or more builds >> >> b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA >> >> c) switch to an ESP32 WROOM module with 32 MB flash (possible?) >> >> We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness. >> >> Regards, >> Michael >> >> >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > -- > Michael Balzer * Am Rahmen 5 * D-58313 Herdecke > Fon 02330 9104094 * Handy 0176 20698926 > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Tested, works like a charm -- test log below.
Only thing that was a bit odd: "ota" still showed the old layout directly after "ota partitions upgrade" (before the reboot). Probably cached, "ota partitions list" showed the new layout.
I then tried rebooting into the old build to see if doing an OTA update from there would show any issues -- nope, works perfectly, installs to the other OTA partition and boots from there. So even after the upgrade, people can still run older firmware versions and perform normal OTA updates, the partitions are identified and located correctly by their label.
Only thing odd with that was: "ota boot factory" failed correctly (did nothing), but did so without any error output. Not really a situation that will occur with normal users, so not really an issue.
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I can take care of the web UI modifications if you like.
Regarding the product version 3.5 to distinguish 6MB capable modules, I think that's a neat & simple solution.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
I see what you mean… I wasn't aware there is so much space left unused now. Like this, if partition alignment constraints allow?
Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7.46875 MB (0x778000 bytes) ota_1 app ota_1 0x00788000 7.46875 MB(0x778000 bytes) store data fat 0x00f00000 1 MB
OK, that's substantially more.
Another option would be to allocate say 7 MB to each OTA partition (or even keep them at 6 MB) and add the remaining free capacity to store.
Performing a config restore currently already fails due to lack of space when not done from a freshly cleared config, especially with some scripts & plugins installed.
With custom JS vehicles being installed to /store, that will get a bit more tight yet, and maybe JS vehicles want to use /store for some extended data storage as well. So more capacity for /store would make sense, heading in that direction.
Regards, Michael
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
OVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
OVMS# module reset Resetting system...
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/ota_0/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-12 (ota1, ota2, no factory) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-711-g4feca695d
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
OVMS# ota boot ? Usage: ota boot ota_0|ota_1 ota_0 Boot from ota_0 image ota_1 Boot from ota_1 image
OVMS# ota copy ? Usage: ota copy ota_0|ota_1 ota_0 OTA copy ota_0 <to> ota_1 OTA copy ota_1 <to>
OVMS# ota erase ? Usage: ota erase ota_0|ota_1 ota_0 Erase ota_0 image ota_1 Erase ota_1 image
Am 24.02.26 um 13:40 schrieb Mark Webb-Johnson:
OK, I have just committed this. The two new commands are:
- ota partitions list
- ota partitions upgrade [-noconfirm]
Rollback is via:
- wget http://api.openvehicles.com/firmware/ota/v3.1/main/partitions.bin
- esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
- Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later)
- Copy running ota firmware to factory
- Set to boot from flash and reboot
- Use ‘ota partitions upgrade’ to upgrade
- Reboot
Items still to be addressed:
- Documentation (including rollback procedure).
- Modify partitions.csv to use this new format (when ready for production).
- Web interface (in particular concept of factory vs ota), if necessary (haven’t checked this).
- OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The ‘ota flash’ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months).
- Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
Regards, Mark.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
On Feb 23, 2026, at 12:27 AM, Michael Balzer <dexter@expeedo.de> wrote:
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13 OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass ## Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot ## The upgrade process checks this and will refuse to upgrade unless that is the case OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d passThis is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real0m24.642s user5m50.376s sys0m38.276sWe’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process:
- Explanation:https://johnmu.com/2024-esp32-partition-update/
- Source:https://github.com/softplus/Esp32Repartition
It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of
spi_flash_erase_range()&spi_flash_write()(→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even needgetPartitionTableAddr(), as our table is fixed at 0x8000.The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev:
Michael, Carsten,
I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB.
We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported.
I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves).
Looking at the partition table:
# OVMS 16MB flash ESP32 Partition Table # Name, Type, SubType, Offset, Size nvs, data, nvs, 0x9000, 0x4000 otadata, data, ota, 0xd000, 0x2000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 4M ota_0, app, ota_0, , 4M ota_1, app, ota_1, , 4M store, data, fat, , 1MAssuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be:
- Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users.
- Provide a migration tool for partition table: 1. Copy running code to factory (from ota_0 or ota_1, whichever is current). 2. Reboot 3. Change partition table (most likely replacing the entire table with a binary blob of the new format) 1. factory 4MB -> ota_0 same offset, 6MB size 2. ota_0 -> ota_1 6MB offset, 6MB size 4. Change boot to ota_0. 5. Reboot
- Liase with factory so new modules use new partition format (and ship with firmware that supports it).
- Wait a reasonable time for users to update before releasing any firmware > 4MB.
That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process.
But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support).
Regards, Mark.
> On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via > OvmsDev<ovmsdev@lists.openvehicles.com>wrote: > > Hi Michael, > > It was to be expected that we would eventually run out of flash > storage. That’s why I would immediately question whether we > should detach ourselves from components that aren’t really being > used. We could even start a survey or something like that. > > For example, the RE tools — the idea behind them is great, but > without documentation they’re not easy to use, and every time I > tried to work with them, it just resulted in crashes. > > Then there’s the question of who actually uses Telnet, SSH, the > Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the > Twizy integration is the only one that uses it). > > All great ideas, but in the end, how many users really make use > of them? > The fewer active components we have to maintain, the ‘easier’ it > would also be to port the code to a more current ESP-IDF version > — and that would bring significant benefits such as improved > networking features, including firewall capabilities and a much > more stable switching between LTE and Wi-Fi. I’ve looked at > llanges attempts and it’s extremely tough. > > As an example, in my own custom firmware builds, I don’t enable > the components (and of course not all vehicles, so not 100 > percent representative) mentioned above. This requires small > modifications in the code because, for example, the ESP logger > is missing but referenced in another file, etc. But my firmware > file is only about 2.8 MB. > > Just my 2 cents > Carsten > >> Am 07.12.2025 um 08:57 schrieb Michael Balzer via >> OvmsDev<ovmsdev@lists.openvehicles.com>: >> >> FYI: use
make size-componentsto create a report on all >> component sizes (make size-filesfor source file level). >> >> Unsurprisingly the webserver is on top, even with all assets >> precompressed already. >> >> >> *_Top 10 components:_* >> >> Per-archive contributions to ELF file: >> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >> libovms_webserver.a 0 255 0 134342 399846 534443 >> libstdc++.a 149 5640 0 141045 72513 219347 >> libmain.a 15 2104 0 139216 40086 181421 >> libduktape.a 0 0 0 141641 20367 162008 >> libvehicle_renaulttwizy. 0 29 0 86517 >> 75357 161903 >> libc-psram-workaround.a 1854 66 18391 118283 >> 10943 149537 >> liblwip.a 17 3873 0 118366 16722 138978 >> libnet80211.a 938 9042 10475 92339 21900 134694 >> libmbedtls.a 100 560 76 107079 26785 134600 >> libvehicle_vweup.a 8 8 0 60846 43432 104294 >> >> >> *_Vehicles:_* >> >> Per-archive contributions to ELF file: >> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >> libvehicle_renaulttwizy. 0 29 0 86517 >> 75357 161903 >> libvehicle_vweup.a 8 8 0 60846 43432 104294 >> libvehicle_mgev.a 156 26 0 47756 33998 81936 >> libvehicle_smarteq.a 82 15 0 59267 19527 78891 >> libvehicle_smarted.a 0 9 0 48481 28801 77291 >> libvehicle_renaultzoe_ph 4 10 0 44622 >> 29564 74200 >> libvehicle.a 0 68 0 57735 12062 69865 >> libvehicle_bmwi3.a 0 2 0 33370 14357 47729 >> libvehicle_minise.a 9432 2 0 35360 2210 47004 >> libvehicle_hyundai_ioniq 176 7 0 32921 >> 13425 46529 >> libvehicle_nissanleaf.a 0 3 0 35491 >> 8941 44435 >> libvehicle_kiasoulev.a 240 9 0 32508 5473 >> 38230 >> libvehicle_kianiroev.a 108 7 0 24070 4006 >> 28191 >> libvehicle_mitsubishi.a 0 5 0 21645 >> 3893 25543 >> libvehicle_boltev.a 0 5 0 15999 7864 23868 >> libvehicle_niu_gtevo.a 4 12 0 18248 3733 >> 21997 >> libvehicle_maxus_edelive 156 3 0 10713 >> 7477 18349 >> libvehicle_renaultzoe.a 0 6 0 14828 >> 2859 17693 >> libvehicle_maxus_euniq56 156 3 0 8363 >> 6840 15362 >> libvehicle_voltampera.a 0 5 0 13221 >> 1932 15158 >> libvehicle_hyundai_ioniq 0 3 0 11081 >> 3191 14275 >> libvehicle_teslaroadster 0 6 0 10367 >> 2238 12611 >> libvehicle_thinkcity.a 0 3 0 6114 3449 >> 9566 >> libvehicle_jaguaripace.a 0 8 0 5445 >> 3941 9394 >> libvehicle_fiatedoblo.a 0 2 0 4262 >> 2126 6390 >> libvehicle_teslamodels.a 0 2 0 5361 >> 948 6311 >> libvehicle_toyotarav4ev. 0 2 0 5023 >> 1255 6280 >> libvehicle_maxus_t90.a 0 3 0 2567 2204 >> 4774 >> libvehicle_byd_atto3.a 0 2 0 3503 947 >> 4452 >> libvehicle_energica.a 0 1 0 3319 880 4200 >> libvehicle_demo.a 0 2 0 3205 795 4002 >> libvehicle_maxus_euniq6. 0 2 0 2470 >> 1182 3654 >> libvehicle_fiat500.a 0 2 0 2838 732 3572 >> libvehicle_zombie_vcu.a 0 4 0 1882 >> 1259 3145 >> libvehicle_mercedesb250e 0 2 0 2113 >> 955 3070 >> libvehicle_zeva.a 0 2 0 2209 688 2899 >> libvehicle_dbc.a 0 2 0 1278 1395 2675 >> libvehicle_cadillac_c2_c 0 7 0 1293 >> 1105 2405 >> libvehicle_maple60s.a 0 2 0 1416 698 2116 >> libvehicle_chevrolet_c6_ 0 2 0 1053 >> 1049 2104 >> libvehicle_obdii.a 0 2 0 957 1007 1966 >> libvehicle_teslamodel3.a 0 2 0 458 >> 713 1173 >> libvehicle_none.a 0 2 0 418 684 1104 >> libvehicle_track.a 0 2 0 416 680 1098 >> >> >> Regards, >> Michael >> >> >> Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: >>> Everyone, >>> >>> with the latest vehicle additions, the firmware size has now >>> grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. >>> >>> Our flash partitioning scheme is currently designed to provide >>> three firmware partitions (factory, ota_0 & ota_1) of 4MB = >>> 4,194,304 bytes each. >>> >>> So we've now got 178,976 bytes = ~4% left. >>> >>> Options beyond the 4 MB limit: >>> >>> a) split features, e.g. vehicle support, into two or more builds >>> >>> b) repartition into two firmware partitions of 6 MB each, >>> reusing the factory partition for OTA >>> >>> c) switch to an ESP32 WROOM module with 32 MB flash (possible?) >>> >>> We've got some time left, new vehicles normally don't need >>> that much space, I just wanted to raise awareness. >>> >>> Regards, >>> Michael >>> >>> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.openvehicles.com >>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >> >> -- >> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke >> Fon 02330 9104094 * Handy 0176 20698926 >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Michael,
Only thing that was a bit odd: "ota" still showed the old layout directly after "ota partitions upgrade" (before the reboot). Probably cached, "ota partitions list" showed the new layout.
This is a simple fix. I will change tonight.
Only thing odd with that was: "ota boot factory" failed correctly (did nothing), but did so without any error output. Not really a situation that will occur with normal users, so not really an issue.
I guess because it can no longer find the partition type ‘factory’.
Both these edge cases would not occur if a reboot was done after the partition upgrade.
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
const esp_partition_t *p = esp_ota_get_running_partition(); if ((p != NULL)&&(strcmp(p->label,"factory")!=0)) { writer->puts("Error: Cannot upgrade partition table if running partition is not factory"); return; }
p = esp_ota_get_boot_partition(); if ((p != NULL)&&(strcmp(p->label,"factory")!=0)) { writer->puts("Error: Cannot upgrade partition table if boot partition is not factory"); return; }
I can take care of the web UI modifications if you like.
That would be helpful. I’m not really familiar with that code.
OK, that's substantially more.
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
Regards, Mark
On 25 Feb 2026, at 6:00 AM, Michael Balzer <dexter@expeedo.de> wrote:
Tested, works like a charm -- test log below.
Only thing that was a bit odd: "ota" still showed the old layout directly after "ota partitions upgrade" (before the reboot). Probably cached, "ota partitions list" showed the new layout.
I then tried rebooting into the old build to see if doing an OTA update from there would show any issues -- nope, works perfectly, installs to the other OTA partition and boots from there. So even after the upgrade, people can still run older firmware versions and perform normal OTA updates, the partitions are identified and located correctly by their label.
Only thing odd with that was: "ota boot factory" failed correctly (did nothing), but did so without any error output. Not really a situation that will occur with normal users, so not really an issue.
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I can take care of the web UI modifications if you like.
Regarding the product version 3.5 to distinguish 6MB capable modules, I think that's a neat & simple solution.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
I see what you mean… I wasn't aware there is so much space left unused now. Like this, if partition alignment constraints allow?
Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7.46875 MB (0x778000 bytes) ota_1 app ota_1 0x00788000 7.46875 MB (0x778000 bytes) store data fat 0x00f00000 1 MBOK, that's substantially more.
Another option would be to allocate say 7 MB to each OTA partition (or even keep them at 6 MB) and add the remaining free capacity to store.
Performing a config restore currently already fails due to lack of space when not done from a freshly cleared config, especially with some scripts & plugins installed.
With custom JS vehicles being installed to /store, that will get a bit more tight yet, and maybe JS vehicles want to use /store for some extended data storage as well. So more capacity for /store would make sense, heading in that direction.
Regards, Michael
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 passOVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/factory/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-711-g4feca695d OTA_O image: 3.3.005-711-g4feca695d OTA_1 image: 3.3.005-704-g6a1fed98
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d passOVMS# module reset Resetting system...
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695d/ota_0/edge (build idf v3.3.4-854-g9063c8662c Feb 24 2026 21:17:07) Partition type: v3-12 (ota1, ota2, no factory) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-711-g4feca695d
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d passOVMS# ota boot ? Usage: ota boot ota_0|ota_1 ota_0 Boot from ota_0 image ota_1 Boot from ota_1 image
OVMS# ota copy ?
Usage: ota copy ota_0|ota_1 ota_0 OTA copy ota_0 <to> ota_1 OTA copy ota_1 <to>OVMS# ota erase ? Usage: ota erase ota_0|ota_1 ota_0 Erase ota_0 image ota_1 Erase ota_1 image
Am 24.02.26 um 13:40 schrieb Mark Webb-Johnson:
OK, I have just committed this. The two new commands are:
ota partitions list ota partitions upgrade [-noconfirm]
Rollback is via:
wget http://api.openvehicles.com/firmware/ota/v3.1/main/partitions.bin esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later) Copy running ota firmware to factory Set to boot from flash and reboot Use ‘ota partitions upgrade’ to upgrade Reboot
Items still to be addressed:
Documentation (including rollback procedure). Modify partitions.csv to use this new format (when ready for production). Web interface (in particular concept of factory vs ota), if necessary (haven’t checked this). OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The ‘ota flash’ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months). Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
Regards, Mark.
P.S. There is a possibility to do this another way to make even more flash space available, but that would mean moving the store partition right to the end of the 16MB flash, which is not so simple. Is 6MB really enough? Probably so if we can move to javascript vehicle modules, otherwise probably not (long-term).
On Feb 23, 2026, at 12:27 AM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
Awesome :-)
We need to update the user manual's firmware rescue guide (https://docs.openvehicles.com/en/latest/userguide/factory.html#flash-factory...) accordingly, so in case anything goes wrong, users can help themselves or get local help.
The 4KB RAM overhead shouldn't be an issue I think, but we could add a free memory check and recommend switching to the "NONE" vehicle while performing the operation, if memory is too tight.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
…and 3328 bytes of RAM in total… so much for the "overhead" ;-)
Regards, Michael
Am 22.02.26 um 16:00 schrieb Mark Webb-Johnson:
I’ve made some progress with this:
OVMS# ota status nocheck Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/1 Firmware: 3.3.005-704-g6a1fed98-dirty/factory/edge (build idf v3.3.4-854-g9063c8662-dirty Feb 22 2026 22:28:00) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-704-g6a1fed98-dirty OTA_O image: 3.3.005-662-g1f318f04 OTA_1 image: 3.3.005-643-gdbec4a13
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass
Before upgrade, user needs to copy current OTA to FACTORY, set boot partition to FACTORY, then reboot
The upgrade process checks this and will refuse to upgrade unless that is the case
OVMS# ota partitions upgrade 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)... Writing new partition table (4096 bytes at 0x00008000)... Partition table upgraded successfully - reboot required
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 6 MB ota_1 app ota_1 0x00610000 6 MB store data fat 0x00c10000 1 MB Digest: a0d94b1efa7f6d8852b44150db218e8d pass
This is mostly implemented in a new ovms_partitions.{h,cpp} in ovms_ota component, with only minor extensions to ovms_ota itself. I have removed the ‘factory’ commands from ovms_ota if the new partition layout is detected at boot.
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
The partition table must also be held in internal (not external SPI) RAM - which is about a 4KB overhead just for these checks.
The usual app-flash still works (and now targets the first OTA at 0x0010000, rather than factory), so this approach seems feasible and workable now. Once we are happy with this, and have a production firmware supporting this layout, I can also provide a new partitions.bin to the factory for new units to ship with.
I have a few minor enhancements to make: (a) add a ‘yes/no’ (like factory reset), (b) an option to load partition table in internal/external ram (internal only at the moment), and (c) some minor tidy-ups. I suggest then to just check it in with this basic manual functionality that can be experimented with. Absent any comments / suggestions, I should be able to commit this early in the coming week.
Regards, Mark.
P.S. OVMS v2 had to fit in 96KB of flash ;-)
On Jan 21, 2026, at 11:12 AM, Mark Webb-Johnson <mark@webb-johnson.net> <mailto:mark@webb-johnson.net> wrote:
Michael,
The links are very helpful.
I have some time, and inclination to handle this (now that my new home build host is up and running and a make of OVMS from clean, with 32 cores and a fast SSD, is under 25 seconds).
real 0m24.642s user 5m50.376s sys 0m38.276s
We’ve already got the ‘ota copy’ command, so I will start with trying to work on manipulation of the partition table and improving the ‘ota status’ command to show partition table format and sizes.
Regards, Mark.
On 11 Jan 2026, at 5:34 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part On the migration tool for changing the partition table from a running application, we're (of course) not the first having that issue.
If we're about to go that way, here is an implementation of the process: Explanation: https://johnmu.com/2024-esp32-partition-update/ Source: https://github.com/softplus/Esp32Repartition It's written for the Arduino IDE with the PlatformIO lib to create a UI, but the core function is straight forward and should be adaptable for us.
The implementation allows for direct manipulations of the partition table, i.e. doesn't need a prepared table blob to flash.
Using a prepared blob should be even more simple, basically just a matter of
spi_flash_erase_range()&spi_flash_write()(→https://github.com/softplus/Esp32Repartition/blob/main/src/part_mgr.cpp#L297). We probably don't even needgetPartitionTableAddr(), as our table is fixed at 0x8000.The code needed is small. When using a blob, the table in theory uses a full flash sector of 4096 bytes, but probably doesn't fill the whole sector.
Regards, Michael
Am 08.12.25 um 07:01 schrieb Mark Webb-Johnson via OvmsDev: > Michael, Carsten, > > I think that if targeting things to cull, we would need to be balance size vs importance. For example, the RE tools mentioned is just 10KB total size. By comparison TPMS is 9KB, and web server is 538KB. > > We learned with OVMS v2 that the biggest culprits in the long-run are always the vehicle modules. The core system stays pretty stable, but the space required for *all* vehicle modules grows with the number of vehicles supported. > > I don’t think we can simply switch to 32MB flash, as that would abandon the existing users. We would also need to source a standard certified 32MB module (which I don’t think Espressif offer themselves). > > Looking at the partition table: > > # OVMS 16MB flash ESP32 Partition Table > # Name, Type, SubType, Offset, Size > nvs, data, nvs, 0x9000, 0x4000 > otadata, data, ota, 0xd000, 0x2000 > phy_init, data, phy, 0xf000, 0x1000 > factory, app, factory, 0x10000, 4M > ota_0, app, ota_0, , 4M > ota_1, app, ota_1, , 4M > store, data, fat, , 1M > > Assuming we could (and that is a big assumption given our older SDK) change that at runtime, then a possible migration path could be: > > Have our code support both old and new partition table formats, and refuse to update to old format if firmware > 4MB. Get that code out into the hands of users. > Provide a migration tool for partition table: > Copy running code to factory (from ota_0 or ota_1, whichever is current). > Reboot > Change partition table (most likely replacing the entire table with a binary blob of the new format) > factory 4MB -> ota_0 same offset, 6MB size > ota_0 -> ota_1 6MB offset, 6MB size > Change boot to ota_0. > Reboot > Liase with factory so new modules use new partition format (and ship with firmware that supports it). > Wait a reasonable time for users to update before releasing any firmware > 4MB. > > That would work more similarly to other more modern ESP frameworks which don’t bother with ‘factory’. It would allow us another 50% expansion. But it does run the risk of bricking (requiring espflash to recover) during the process. > > But longer-term, the solution to me seems to be to allow the vehicle module code to overlay - so only the single vehicle you choose is loaded. And (absent any dynamic linking of modular code in freertos), the only straightforward way of doing that I know of is migrating vehicle support to Javascript (which comes along with a host of other advantages - most notably not having to be a C++ embedded developer to add/refine vehicle support). > > Regards, Mark. > >> On 7 Dec 2025, at 4:39 PM, Carsten Schmiemann via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote: >> >> Hi Michael, >> >> It was to be expected that we would eventually run out of flash storage. That’s why I would immediately question whether we should detach ourselves from components that aren’t really being used. We could even start a survey or something like that. >> >> For example, the RE tools — the idea behind them is great, but without documentation they’re not easy to use, and every time I tried to work with them, it just resulted in crashes. >> >> Then there’s the question of who actually uses Telnet, SSH, the Duktape framework, the DBC parser, OBD2ECU, or CANopen (yes, the Twizy integration is the only one that uses it). >> >> All great ideas, but in the end, how many users really make use of them? >> The fewer active components we have to maintain, the ‘easier’ it would also be to port the code to a more current ESP-IDF version — and that would bring significant benefits such as improved networking features, including firewall capabilities and a much more stable switching between LTE and Wi-Fi. I’ve looked at llanges attempts and it’s extremely tough. >> >> As an example, in my own custom firmware builds, I don’t enable the components (and of course not all vehicles, so not 100 percent representative) mentioned above. This requires small modifications in the code because, for example, the ESP logger is missing but referenced in another file, etc. But my firmware file is only about 2.8 MB. >> >> Just my 2 cents >> Carsten >> >>> Am 07.12.2025 um 08:57 schrieb Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com>: >>> >>> FYI: use
make size-componentsto create a report on all component sizes (make size-filesfor source file level). >>> >>> Unsurprisingly the webserver is on top, even with all assets precompressed already. >>> >>> >>> Top 10 components: >>> >>> Per-archive contributions to ELF file: >>> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >>> libovms_webserver.a 0 255 0 134342 399846 534443 >>> libstdc++.a 149 5640 0 141045 72513 219347 >>> libmain.a 15 2104 0 139216 40086 181421 >>> libduktape.a 0 0 0 141641 20367 162008 >>> libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 >>> libc-psram-workaround.a 1854 66 18391 118283 10943 149537 >>> liblwip.a 17 3873 0 118366 16722 138978 >>> libnet80211.a 938 9042 10475 92339 21900 134694 >>> libmbedtls.a 100 560 76 107079 26785 134600 >>> libvehicle_vweup.a 8 8 0 60846 43432 104294 >>> >>> >>> Vehicles: >>> >>> Per-archive contributions to ELF file: >>> Archive File DRAM .data & .bss IRAM Flash code & rodata Total >>> libvehicle_renaulttwizy. 0 29 0 86517 75357 161903 >>> libvehicle_vweup.a 8 8 0 60846 43432 104294 >>> libvehicle_mgev.a 156 26 0 47756 33998 81936 >>> libvehicle_smarteq.a 82 15 0 59267 19527 78891 >>> libvehicle_smarted.a 0 9 0 48481 28801 77291 >>> libvehicle_renaultzoe_ph 4 10 0 44622 29564 74200 >>> libvehicle.a 0 68 0 57735 12062 69865 >>> libvehicle_bmwi3.a 0 2 0 33370 14357 47729 >>> libvehicle_minise.a 9432 2 0 35360 2210 47004 >>> libvehicle_hyundai_ioniq 176 7 0 32921 13425 46529 >>> libvehicle_nissanleaf.a 0 3 0 35491 8941 44435 >>> libvehicle_kiasoulev.a 240 9 0 32508 5473 38230 >>> libvehicle_kianiroev.a 108 7 0 24070 4006 28191 >>> libvehicle_mitsubishi.a 0 5 0 21645 3893 25543 >>> libvehicle_boltev.a 0 5 0 15999 7864 23868 >>> libvehicle_niu_gtevo.a 4 12 0 18248 3733 21997 >>> libvehicle_maxus_edelive 156 3 0 10713 7477 18349 >>> libvehicle_renaultzoe.a 0 6 0 14828 2859 17693 >>> libvehicle_maxus_euniq56 156 3 0 8363 6840 15362 >>> libvehicle_voltampera.a 0 5 0 13221 1932 15158 >>> libvehicle_hyundai_ioniq 0 3 0 11081 3191 14275 >>> libvehicle_teslaroadster 0 6 0 10367 2238 12611 >>> libvehicle_thinkcity.a 0 3 0 6114 3449 9566 >>> libvehicle_jaguaripace.a 0 8 0 5445 3941 9394 >>> libvehicle_fiatedoblo.a 0 2 0 4262 2126 6390 >>> libvehicle_teslamodels.a 0 2 0 5361 948 6311 >>> libvehicle_toyotarav4ev. 0 2 0 5023 1255 6280 >>> libvehicle_maxus_t90.a 0 3 0 2567 2204 4774 >>> libvehicle_byd_atto3.a 0 2 0 3503 947 4452 >>> libvehicle_energica.a 0 1 0 3319 880 4200 >>> libvehicle_demo.a 0 2 0 3205 795 4002 >>> libvehicle_maxus_euniq6. 0 2 0 2470 1182 3654 >>> libvehicle_fiat500.a 0 2 0 2838 732 3572 >>> libvehicle_zombie_vcu.a 0 4 0 1882 1259 3145 >>> libvehicle_mercedesb250e 0 2 0 2113 955 3070 >>> libvehicle_zeva.a 0 2 0 2209 688 2899 >>> libvehicle_dbc.a 0 2 0 1278 1395 2675 >>> libvehicle_cadillac_c2_c 0 7 0 1293 1105 2405 >>> libvehicle_maple60s.a 0 2 0 1416 698 2116 >>> libvehicle_chevrolet_c6_ 0 2 0 1053 1049 2104 >>> libvehicle_obdii.a 0 2 0 957 1007 1966 >>> libvehicle_teslamodel3.a 0 2 0 458 713 1173 >>> libvehicle_none.a 0 2 0 418 684 1104 >>> libvehicle_track.a 0 2 0 416 680 1098 >>> >>> >>> Regards, >>> Michael >>> >>> >>> Am 06.12.25 um 10:33 schrieb Michael Balzer via OvmsDev: >>>> Everyone, >>>> >>>> with the latest vehicle additions, the firmware size has now grown to 4,015,328 bytes in build 3.3.005-485-gc4664881. >>>> >>>> Our flash partitioning scheme is currently designed to provide three firmware partitions (factory, ota_0 & ota_1) of 4MB = 4,194,304 bytes each. >>>> >>>> So we've now got 178,976 bytes = ~4% left. >>>> >>>> Options beyond the 4 MB limit: >>>> >>>> a) split features, e.g. vehicle support, into two or more builds >>>> >>>> b) repartition into two firmware partitions of 6 MB each, reusing the factory partition for OTA >>>> >>>> c) switch to an ESP32 WROOM module with 32 MB flash (possible?) >>>> >>>> We've got some time left, new vehicles normally don't need that much space, I just wanted to raise awareness. >>>> >>>> Regards, >>>> Michael >>>> >>>> >>>> >>>> _______________________________________________ >>>> OvmsDev mailing list >>>> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >>> >>> -- >>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke >>> Fon 02330 9104094 * Handy 0176 20698926 >>> >>> _______________________________________________ >>> OvmsDev mailing list >>> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev >> >> _______________________________________________ >> OvmsDev mailing list >> OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev > > > > _______________________________________________ > OvmsDev mailing list > OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Current (12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Suggested (12, 7MB code, more store): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
- Create a 2nd fat store partition (0xE10000-0xFFFFFF)
- Format and mount that as FAT
- Copy files from old to new
- Unmount and drop the old store
- Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Current (12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Suggested (12, 7MB code, more store): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
- Create a 2nd fat store partition (0xE10000-0xFFFFFF)
- Format and mount that as FAT
- Copy files from old to new
- Unmount and drop the old store
- Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this?
- Copy running ota image to "factory", reboot into "factory"
- Upgrade store partition, reboot
- Upgrade OTA partitions, reboot
If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Current (12): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused* 0xD10000 0x2F0000 1000000 13697024 3080192 Suggested (12, 7MB code, more store): Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
- Create a 2nd fat store partition (0xE10000-0xFFFFFF)
- Format and mount that as FAT
- Copy files from old to new
- Unmount and drop the old store
- Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store ota partition downgrade store ota partition upgrade factory
I’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this? Copy running ota image to "factory", reboot into "factory" Upgrade store partition, reboot Upgrade OTA partitions, reboot If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Mark,
Am 26.02.26 um 01:47 schrieb Mark Webb-Johnson:
My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
That could be avoided: OvmsConfig::mount() could try mounting
"store-v35" (or whatever the new name is) first, then fall back to "store".
Same scheme can be applied to module_perform_factoryreset().
That way "store" doesn't need to be relabeled at all, the system automatically uses the new partition if available.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Am 26.02.26 um 21:51 schrieb Michael Balzer via OvmsDev:
Mark,
Am 26.02.26 um 01:47 schrieb Mark Webb-Johnson:
My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
That could be avoided:
OvmsConfig::mount()could try mounting "store-v35" (or whatever the new name is) first, then fall back to "store".Same scheme can be applied to
module_perform_factoryreset().That way "store" doesn't need to be relabeled at all, the system automatically uses the new partition if available.
Btw, we also don't need a store rollback command then, as "store" remains intact until step #3 does the final update of the partition table.
Step #3 would then also rename "store-v35" to "store", to recover compatibility with older firmware builds.
Regards, Michael
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
I'm working on support for a new car. I've been capturing two kinds of can logs, one where the car doesn't move (importantly the odometer doesn't change) and the other where I drive somewhere (odometer and tpms values change).
I've noticed it can be difficult to obtain a can log for the latter case. For example today I tried to create a can log for the trip to pick up lunch. Before starting the car (or even waking it up) I started a can log and verified it it was active:
OVMS# can log start vfs crtd /sd/ct5-13.crtd
CAN logging to VFS active: Type:vfs Format:crtd(discard)
Vehicle:CT5 Path:/sd/ct5-13.crtd OVMS# can log status #1: open Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd Size:149 Messages:1 Dropped:0 Filtered:0 Rate:0.0% /sd/ct5-13.crtd: running Size:149 Messages:1 Discarded:0 Dropped:0 Filtered:0 Rate:0.0%
When I arrive at my destination, I turned the car off and then used the ios ovms app to check the log:
can log status
CAN logging inactive
So my can log was no longer active.
When I got back home I ssh'ed into the module and found that the log was present but with a length of zero and a bogus timestamp:
OVMS# vfs ls /sd
[DIR] 05-Jul-2019 19:48 backup/
0 30-Nov-1979 00:00 ct5-10.crtd
75.8M 30-Mar-2026 11:51 ct5-11.crtd
35.5M 30-Mar-2026 21:17 ct5-12.crtd
0 30-Nov-1979 00:00 ct5-13.crtd
[DIR] 05-Jul-2019 19:48 logs/
[DIR] 29-Mar-2026 09:58 scripts/
In cases where I have successfully captured a can log while driving I used the app to stop logging *before* turning the car off. But every time I've tried where I've turned the car off before stopping the can log has failed.
My expectation is that changing ms_v_env_on should not impact can logging. This feels like a bug but I allow that there might be something happening here that I do not understand.
It also appears that nothing new is written to the normal log file while can logging is active; is that expected?
Craig
P.S. I configured opendkim to not sign messages to ovmsdev@lists.openvehicles.com
I did a quick test with CAN logging & regular logging enabled while charging, and found no issues. Both files get written correctly.
I didn't test driving, but the CAN logging system does not use vehicle metrics or events in any special way.
The file remaining at 0 bytes and unset time hints at a crash during your drive (before stopping the logger). Did you rule that out?
If so, do you have some plugins or scripts bound to events, that may interfere with logging?
Don't expect the file meta data to change while the logger is running,
other than the system logging, the CAN logging deliberately does no
fsync() calls to keep running at maximum speed.
Regards, Michael
Am 02.04.26 um 22:25 schrieb Craig Leres via OvmsDev:
I'm working on support for a new car. I've been capturing two kinds of can logs, one where the car doesn't move (importantly the odometer doesn't change) and the other where I drive somewhere (odometer and tpms values change).
I've noticed it can be difficult to obtain a can log for the latter case. For example today I tried to create a can log for the trip to pick up lunch. Before starting the car (or even waking it up) I started a can log and verified it it was active:
OVMS# can log start vfs crtd /sd/ct5-13.crtd CAN logging to VFS active: Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd OVMS# can log status #1: open Type:vfs Format:crtd(discard) Vehicle:CT5 Path:/sd/ct5-13.crtd Size:149 Messages:1 Dropped:0 Filtered:0 Rate:0.0% /sd/ct5-13.crtd: running Size:149 Messages:1 Discarded:0 Dropped:0 Filtered:0 Rate:0.0%
When I arrive at my destination, I turned the car off and then used the ios ovms app to check the log:
can log status CAN logging inactive
So my can log was no longer active.
When I got back home I ssh'ed into the module and found that the log was present but with a length of zero and a bogus timestamp:
OVMS# vfs ls /sd [DIR] 05-Jul-2019 19:48 backup/ 0 30-Nov-1979 00:00 ct5-10.crtd 75.8M 30-Mar-2026 11:51 ct5-11.crtd 35.5M 30-Mar-2026 21:17 ct5-12.crtd 0 30-Nov-1979 00:00 ct5-13.crtd [DIR] 05-Jul-2019 19:48 logs/ [DIR] 29-Mar-2026 09:58 scripts/
In cases where I have successfully captured a can log while driving I used the app to stop logging *before* turning the car off. But every time I've tried where I've turned the car off before stopping the can log has failed.
My expectation is that changing ms_v_env_on should not impact can logging. This feels like a bug but I allow that there might be something happening here that I do not understand.
It also appears that nothing new is written to the normal log file while can logging is active; is that expected?
Craig
P.S. I configured opendkim to not sign messages to ovmsdev@lists.openvehicles.com
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/3/26 02:43, Michael Balzer via OvmsDev wrote:
I did a quick test with CAN logging & regular logging enabled while charging, and found no issues. Both files get written correctly.
I didn't test driving, but the CAN logging system does not use vehicle metrics or events in any special way.
The file remaining at 0 bytes and unset time hints at a crash during your drive (before stopping the logger). Did you rule that out?
I don't know how to check module uptime but I know it logs the vin as soon as it sees it after booting and the most recent vin log was a couple of days before the logging attempt.
If so, do you have some plugins or scripts bound to events, that may interfere with logging?
No and /sd/scripts and /store/scripts are empty.
Don't expect the file meta data to change while the logger is running, other than the system logging, the CAN logging deliberately does no
fsync()calls to keep running at maximum speed.
Right, I knew about that.
The longest can capture I have is 29 minutes long and averages over 900 frames per second. Could my problem be that the can data rate is just too high?
Looking at the log more closely I see it was having trouble connecting to the v2 server between 1:23am and 10:07am:
2026-04-02 01:23:31.847 PDT W (141523227) ovms-server-v2:
Connection failed 2026-04-02 01:23:31.847 PDT E (141523227) ovms-server-v2: Status: Error: Connection failed 2026-04-02 01:23:31.857 PDT E (141523237) ovms-server-v2: Status: Connection lost, reconnecting [...] 2026-04-02 10:07:01.827 PDT W (172933207) ovms-server-v2: Connection failed 2026-04-02 10:07:01.827 PDT E (172933207) ovms-server-v2: Status: Error: Connection failed 2026-04-02 10:07:01.837 PDT E (172933217) ovms-server-v2: Status: Connection lost, reconnecting
Perhaps unrelated.
Craig
Am 03.04.26 um 19:06 schrieb Craig Leres:
On 4/3/26 02:43, Michael Balzer via OvmsDev wrote:
I did a quick test with CAN logging & regular logging enabled while charging, and found no issues. Both files get written correctly.
I didn't test driving, but the CAN logging system does not use vehicle metrics or events in any special way.
The file remaining at 0 bytes and unset time hints at a crash during your drive (before stopping the logger). Did you rule that out?
I don't know how to check module uptime but I know it logs the vin as soon as it sees it after booting and the most recent vin log was a couple of days before the logging attempt.
The command "boot" shows uptime and last crash details, if any:
OVMS# boot Last boot was 3644073 second(s) ago Time at boot: 2026-02-20 13:57:09 CET This is reset #0 since last power cycle Detected boot reason: PowerOn (1/14) Reset reason: Unknown/unset (0) Crash counters: 0 total, 0 early
If so, do you have some plugins or scripts bound to events, that may interfere with logging?
No and /sd/scripts and /store/scripts are empty.
Don't expect the file meta data to change while the logger is running, other than the system logging, the CAN logging deliberately does no
fsync()calls to keep running at maximum speed.Right, I knew about that.
The longest can capture I have is 29 minutes long and averages over 900 frames per second. Could my problem be that the can data rate is just too high?
Try filtering then, 900 frames per second is rather high.
Assuming most of the frames are 8 bytes in length, that rate amounts to ~45 kB per second = 82 MB in 30 minutes.
My largest CRTD recording archived is 52 MB, with 1083482 frames in 51 minutes = averaging at ~ 350 frames per second (recorded January 2022 from my Seat Mii's Comfort CAN). I don't remember the module state besides the CAN recording, but I assume I had regular file logging paused to save SD bandwidth.
Filtering will tell if the data rate or file size is causing the issue.
Regards, Michael
Looking at the log more closely I see it was having trouble connecting to the v2 server between 1:23am and 10:07am:
2026-04-02 01:23:31.847 PDT W (141523227) ovms-server-v2: Connection failed 2026-04-02 01:23:31.847 PDT E (141523227) ovms-server-v2: Status: Error: Connection failed 2026-04-02 01:23:31.857 PDT E (141523237) ovms-server-v2: Status: Connection lost, reconnecting [...] 2026-04-02 10:07:01.827 PDT W (172933207) ovms-server-v2: Connection failed 2026-04-02 10:07:01.827 PDT E (172933207) ovms-server-v2: Status: Error: Connection failed 2026-04-02 10:07:01.837 PDT E (172933217) ovms-server-v2: Status: Connection lost, reconnecting
Perhaps unrelated.
Craig
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/3/26 10:28, Michael Balzer wrote:
The command "boot" shows uptime and last crash details, if any:
OVMS# boot Last boot was 3644073 second(s) ago Time at boot: 2026-02-20 13:57:09 CET This is reset #0 since last power cycle Detected boot reason: PowerOn (1/14) Reset reason: Unknown/unset (0) Crash counters: 0 total, 0 early
I don't believe I'm crashing:
OVMS# boot
Last boot was 88094 second(s) ago
Time at boot: 2026-04-03 10:14:11 PDT
This is reset #6 since last power cycle
Detected boot reason: SoftReset (12/12)
Reset reason: esp_restart (3)
Crash counters: 0 total, 0 early
I think my only restarts are when I reboot to test a new local version.
The longest can capture I have is 29 minutes long and averages over 900 frames per second. Could my problem be that the can data rate is just too high?
Try filtering then, 900 frames per second is rather high.
Assuming most of the frames are 8 bytes in length, that rate amounts to ~45 kB per second = 82 MB in 30 minutes.
The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
OVMS# can log start vfs crtd /sd/ct5-14.crtd '034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380' CAN logging to VFS active: Type:vfs Format:crtd(discard) Filter:034-3f Vehicle:CT5 Path:/sd/ct5-14.crtd
Am 04.04.26 um 20:02 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote:
Try filtering then, 900 frames per second is rather high. The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
The issue isn't the microRL constraint.
can_log_vfs_start() & can::AddLogger() don't support multiple
filters in a string, so you're limited by the maximum argument count as
defined by the command registration, in this case 9 (see
OvmsCanLogVFSInit::OvmsCanLogVFSInit()).
You can raise that limit without adding RAM requirements, it's just an argc validation. If you do so, please copy to all log formats.
Another option could be to add a negation syntax & function for the log filters, if that helps. Or to add a special filter option to skip all-zero frames.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/5/26 01:24, Michael Balzer wrote:
Am 04.04.26 um 20:02 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote:
Try filtering then, 900 frames per second is rather high. The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
The issue isn't the microRL constraint.
`can_log_vfs_start()` & `can::AddLogger()` don't support multiple filters in a string, so you're limited by the maximum argument count as defined by the command registration, in this case 9 (see `OvmsCanLogVFSInit::OvmsCanLogVFSInit()`).
(As I later discovered!)
You can raise that limit without adding RAM requirements, it's just an argc validation.
If you do so, please copy to all log formats.
Understood.
Another option could be to add a negation syntax & function for the log filters, if that helps. Or to add a special filter option to skip all- zero frames.
I don't think that helps, there are some pids that sometimes have all zero payload bytes (e.g. 0x063 which I use to determine "is running"): 1772821542.210018 1R11 063 00 00 00 00 00 00 1772821741.875479 1R11 063 00 00 00 f5 00 00 1772821683.149862 1R11 063 00 00 16 82 00 00 1772821685.590338 1R11 063 00 00 14 00 00 00 1772821741.976706 1R11 063 00 00 00 00 00 00 Appended is what I did locally for testing, raising _COMMAND_TOKEN_NMB impacts the stack use of several routines in microrl.c, I can't judge if that's a problem or not. Two other things I see: the usage message has way more info about the syntax than the user manual: "<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f", vs or log specific CAN packets by applying a filter e.g 0x55b the Nissan LEAF SoC CAN message ovms# can log start vfs crtd /sd/can.crtd 55b PR #1367 is my initial attempt at this. Also, validation of the filter arguments needs to be stricter, currently just about any junk string is "accepted" as a filter: OVMS# can log start vfs crtd /sd/ct5-14.crtd asdjfjkadsfj CAN logging to VFS active: Type:vfs Format:crtd(discard) Filter:00a-d Vehicle:CT5 Path:/sd/ct5-14.crtd Craig ======================================================================== sea 82 % git diff master diff --git a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp index 0fe74e80..b3194ca0 100644 --- a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp +++ b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp @@ -82,7 +82,7 @@ OvmsCanLogVFSInit::OvmsCanLogVFSInit() "<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f", - 1, 9, true, vfs_file_validate); + 1, _COMMAND_TOKEN_NMB, true, vfs_file_validate); } } } diff --git a/vehicle/OVMS.V3/components/microrl/microrl_config.h b/vehicle/OVMS.V3/components/microrl/microrl_config.h index 614b9614..e8b2a9c7 100644 --- a/vehicle/OVMS.V3/components/microrl/microrl_config.h +++ b/vehicle/OVMS.V3/components/microrl/microrl_config.h @@ -21,7 +21,7 @@ typed in command line exceed this value, then prints message about it and command line not to be parced and 'execute' callback will not calls. Token is word separate by white space, for example 3 token line: "IRin> set mode test" */ -#define _COMMAND_TOKEN_NMB 16 +#define _COMMAND_TOKEN_NMB 128 /* Define you prompt string here. You can use colors escape code, for highlight you prompt,
Seems fair enough to allow more filters. Yeah, the parsing is pretty permissive (using sscanf). Maybe just echoing back what was parsed might not be a bad idea? //.ichael On Mon, 6 Apr 2026 at 05:18, Craig Leres via OvmsDev < ovmsdev@lists.openvehicles.com> wrote:
On 4/5/26 01:24, Michael Balzer wrote:
Am 04.04.26 um 20:02 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote:
Try filtering then, 900 frames per second is rather high. The actual size of that capture is 75.8 MB. It's clear the programmers did not care about saving bandwidth, many pids have payloads that are always all zero...
I wrote a python script to generate a filter string that skips pids with always-zero payloads. This has the potential to save 42% of the captured frames but I'm running into parsing limits. Quoting doesn't help/work. I guess I can raise _QUOTED_TOKEN_NMB from 10 to 100 to make this work locally for testing; would it be bad to raise it in the master branch?
Craig
OVMS# can log start vfs crtd /sd/ct5-14.crtd 034 03f 04b-04c 063 073-0cc 147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380 ERROR:too many tokens or invalid quoting
The issue isn't the microRL constraint.
`can_log_vfs_start()` & `can::AddLogger()` don't support multiple filters in a string, so you're limited by the maximum argument count as defined by the command registration, in this case 9 (see `OvmsCanLogVFSInit::OvmsCanLogVFSInit()`).
(As I later discovered!)
You can raise that limit without adding RAM requirements, it's just an argc validation.
If you do so, please copy to all log formats.
Understood.
Another option could be to add a negation syntax & function for the log filters, if that helps. Or to add a special filter option to skip all- zero frames.
I don't think that helps, there are some pids that sometimes have all zero payload bytes (e.g. 0x063 which I use to determine "is running"):
1772821542.210018 1R11 063 00 00 00 00 00 00 1772821741.875479 1R11 063 00 00 00 f5 00 00 1772821683.149862 1R11 063 00 00 16 82 00 00 1772821685.590338 1R11 063 00 00 14 00 00 00 1772821741.976706 1R11 063 00 00 00 00 00 00
Appended is what I did locally for testing, raising _COMMAND_TOKEN_NMB impacts the stack use of several routines in microrl.c, I can't judge if that's a problem or not.
Two other things I see: the usage message has way more info about the syntax than the user manual:
"<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f",
vs
or log specific CAN packets by applying a filter e.g 0x55b the Nissan LEAF SoC CAN message
ovms# can log start vfs crtd /sd/can.crtd 55b
PR #1367 is my initial attempt at this.
Also, validation of the filter arguments needs to be stricter, currently just about any junk string is "accepted" as a filter:
OVMS# can log start vfs crtd /sd/ct5-14.crtd asdjfjkadsfj CAN logging to VFS active: Type:vfs Format:crtd(discard) Filter:00a-d Vehicle:CT5 Path:/sd/ct5-14.crtd
Craig
======================================================================== sea 82 % git diff master diff --git a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp index 0fe74e80..b3194ca0 100644 --- a/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp +++ b/vehicle/OVMS.V3/components/can/src/canlog_vfs.cpp @@ -82,7 +82,7 @@ OvmsCanLogVFSInit::OvmsCanLogVFSInit() "<path> [filter1] ... [filterN]\n" "Filter: <bus> | <id>[-<id>] | <bus>:<id>[-<id>]\n" "Example: 2:2a0-37f", - 1, 9, true, vfs_file_validate); + 1, _COMMAND_TOKEN_NMB, true, vfs_file_validate); } } } diff --git a/vehicle/OVMS.V3/components/microrl/microrl_config.h b/vehicle/OVMS.V3/components/microrl/microrl_config.h index 614b9614..e8b2a9c7 100644 --- a/vehicle/OVMS.V3/components/microrl/microrl_config.h +++ b/vehicle/OVMS.V3/components/microrl/microrl_config.h @@ -21,7 +21,7 @@ typed in command line exceed this value, then prints message about it and command line not to be parced and 'execute' callback will not calls. Token is word separate by white space, for example 3 token line: "IRin> set mode test" */ -#define _COMMAND_TOKEN_NMB 16 +#define _COMMAND_TOKEN_NMB 128
/* Define you prompt string here. You can use colors escape code, for highlight you prompt,
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Am 05.04.26 um 23:17 schrieb Craig Leres:
On 4/3/26 10:28, Michael Balzer wrote: You can raise that limit without adding RAM requirements, it's just an argc validation. Appended is what I did locally for testing, raising _COMMAND_TOKEN_NMB impacts the stack use of several routines in microrl.c, I can't judge if that's a problem or not.
Sorry, I didn't recognize you'd also need to raise _COMMAND_TOKEN_NMB for your filter.
Raising _COMMAND_TOKEN_NMB from 16 to 128 has a significant impact on the microRL stack requirements, raising stack allocation for any command execution from 64 to 512 Bytes. That's not safe, as we cannot guarantee any context running e.g. BufferedShell commands has sufficient stack, and especially as some commands have high stack requirements themselves.
So there are two options to solve this:
a) Rework microRL's microrl_get_complite() and new_line_handler() to
use the heap instead of the stack for the token arrays (and make sure
the SPIRAM heap is used if available).
b) Rework canfilter::AddFilter() to allow passing multiple filter
specs in a string, so you can use a quoted argument.
…or maybe both. I'd opt for (a) to begin with, as that has a general positive impact on stack usage.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/6/26 00:13, Michael Balzer wrote:
a) Rework microRL's
microrl_get_complite()andnew_line_handler()to use the heap instead of the stack for the token arrays (and make sure the SPIRAM heap is used if available).
Could you please point me to existing code that uses the heap in the way you suggest?
I found some esp documentation:
https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/sy...
When external RAM is enabled, external SPI RAM can be allocated
using standard malloc calls, or via
heap_caps_malloc(MALLOC_CAP_SPIRAM), depending on the configuration.
See Configuring External RAM for more details.
I also see the call to heap_caps_malloc() in components/zip/zlib/zutil.c:
voidpf ZLIB_INTERNAL zcalloc (opaque, items, size)
voidpf opaque;
unsigned items;
unsigned size;
{
(void)opaque;
void* ret = heap_caps_malloc(items * size, MALLOC_CAP_SPIRAM);
if (ret)
return ret;
else
return malloc(items * size);
}
Is using calloc/malloc/realloc good enough?
Craig
Apparently I'm still having issues with capture.
I'm trying to make short trace (~5 mins). I start with:
can log start vfs crtd /sd/ct5-02.crtd 034 03f 04b-04c 063 073-0cc
147-1be 225-297 2a9-2c9 2ce-2d1 2d4-2d7 2da 2dc 2e0 2e2-382 3a2-3a8 3aa-3ba 3d0-3d5 41b 42b 442-458 462 467-469 46d 478-483 489-48d 494 499-49d 49f-4a0 4a2 4a4-4cc 4ce-4d0 4d7-4da 4dc 4df 4e3 4e6-4f8 4fa-580 582-588 58d-58f 591-594 596-597 5a4-5b8 5ba 5bd-5be 5c1 5c3 5c5-5c6 5c8-5e1 5e4-5e5 5eb 5ef 5f1-5f2 5f4 5f6-5fe 701-702 707-714 717 71d 71f-721 724-728 72e-752 754-757 759-764 767 769 771-772 774 779-784 786-78a 78e 791-7a6 7a8-7ab 7ae-7b9 7bb 7bd 7bf 7d0-7d4 7df-14daf380
and everything looks good. A couple of minutes later when I check, the module has rebooted. When I tried this earlier today I didn't think there was a stack trace. But I got one a few minutes ago:
OVMS# boot
Last boot was 423 second(s) ago
Time at boot: 2026-07-01 21:15:10 PDT
This is reset #2 since last power cycle
Detected boot reason: Crash (12/12)
Reset reason: Task watchdog (6)
Crash counters: 2 total, 0 early
Auto init level: Full
Last crash: abort() was called on core 0
Current task on core 0: IDLE0, 512 stack bytes free
Current task on core 1: OVMS CanLog, 2232 stack bytes free
Backtrace:
0x40089137 0x400893d1 0x4010a354 0x40083df2
Event: vehicle.aux.12v.dip@ovms-server-v3 36 secs
WDT tasks: IDLE1
Version: 3.3.006-67-gd4d8b577/ota_0/main (build idf
v3.3.4-854-g9063c8662 Jun 3 2026 15:19:31, product v3.3-5)
Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
Which works out to:
sea 3 % ./backtrace.sh 0x40089137 0x400893d1 0x4010a354 0x40083df2
+ xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x40089137
0x400893d1 0x4010a354 0x40083df2
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/task_wdt.c:274
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/freertos/xtensa_vectors.S:1154
So a watchdog timer?
I've also notice it's not unusual when this car+module is on the home wifi (and the car idle for hours) that I can ping it but not ssh/http do not respond and I cannot send commands via the ios app.
I just updated to 3.3.006-120-g465b5249. I guess tomorrow I'll try creating a crtd without any filter terms. If that doesn't work I'll try changing the vehicle to obdii. Other ideas?
Craig
P.S. I was too impatient when 3.3.006-120-g465b5249 was booting and got a stack trace when trying to ssh in during the first ~30 seconds of uptime. Looks like I'm hitting the abort() at the end of esp_sha_read_digest_state():
abort(); // SHA peripheral returned all zero state, probably due to
fault injection
I sure wish the we wouldn't accept a tcp connection until the crypto is all done initializing... (Is this difficult to fix?)
sea 3 % ./backtrace.sh 0x40089137 0x400893d1 0x4028b989 0x4026bef2 0x40265c9a 0x4026768f 0x40276ac3 0x4026795e 0x40267989 0x40156720 0x40157db0 0x40159501 0x40159a23 0x40155f32 0x4015b78e 0x40145470 0x40145599
- xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x40089137 0x400893d1 0x4028b989 0x4026bef2 0x40265c9a 0x4026768f 0x40276ac3 0x4026795e 0x40267989 0x40156720 0x40157db0 0x40159501 0x40159a23 0x40155f32 0x4015b78e 0x40145470 0x40145599 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/hwcrypto/sha.c:272 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/port/esp_sha256.c:349 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_cli.c:3894 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:1701 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:1701 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:1701 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:1701 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:1701 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/src/mongoose_client.cpp:52 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:1000 /home/sea/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:980
I plugged a laptop into the ovms console and went for a drive today. I pulled over not even 1/2 way around the block and the console was unresponsive. Then it rebooted. It was responsive for a short while then repeated. The stack traces are the same as yesterday (watchdog). Clearly can0 is seeing more frames than it can handle when the car is running (even when not trying capture frames).
Looks like I need to learn how to use the esp32 acceptance filter. It also does not appear any current vehicles use it (vehicle_niu_gtevo has some code that is commented out -- I'm guessing someone didn't get it working and bailed).
Since the largest PID I've ever seen is 7EE, I can use dual filter mode (two 11 bit filters) if needed. If I can eliminate the top ten uninteresting PIDs (i.e. always all zero's or all one's payloads) it will screen 27% of the frames. I also want to be careful I'm not hiding PIDs that I don't yet understand but are of interest (e.g. TPMS).
Craig
This has been a nightmare. Then a digression migrating servers. Then more of a nightmare.
Firstly, please don’t do these commands on production modules at the moment. Things may change.
Secondly, only do these commands on boxes with standard partition tables and flash sizes.
The sequence that currently works is as follows:
ota partitions store upgrade module reset (aka ‘reboot’) sd unmount ota partitions store migrate ota partitions upgrade factory module reset (aka ‘reboot’)
You can use ’ota status nocheck' and ‘ota partitions list’ to see status along the way.
This is my fourth re-write of this, and the best I can get it at the moment. It needs another set of eyes looking over it, because the process is fraught with danger along the way.
Some comments (all learned the hard way):
After making the new store2 partition, it can’t be mounted before reboot (I guess that the esp idf caches the partition table at boot). Our esp idf components/fatfs/src/ffconf.h has #defined FF_VOLUMES 2 which means we can’t mount more than two FAT filesystems. /store and /store2 for migration, but what about SD CARD? We can increase that (our esp idf), but even then a lot of our code assumes 0: is /store and 1: is /sd, and things got messy when I tried. The recursive copy /store to /store2 without using a lot of stack was tricky, but seems to be ok and as reliable as I could make it. I keep a list of directories to copy, to avoid direct subroutine call recursion. There is a ’ota partitions store downgrade’ to go back before step 5. We’ll probably want to move ‘ota partitions store mount/unmount’ out before production, as it opens us up to the user passwords being accessible - or at least add those /store2 paths to the protected paths list. The good news is I learnt a lot about the internals of ESP IDF, fatfs, partitions, et al.
At the moment this is ugly and not idiot proof. We are also down to about 60KB of free flash space (4MB limit).
Please have a play with it. Happy to accept suggestions as to how to get this usable and production ready.
OVMS# vfs df /store: Size: 1936 kB Avail: 1804 kB Used: 7% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
OVMS# ota partitions list
Partition table:
Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB
otadata data ota 0x0000d000 8 KB
phy_init data phy 0x0000f000 4 KB
ota_0 app ota_0 0x00010000 7 MB
ota_1 app ota_1 0x00710000 7 MB
store data fat 0x00e10000 1984 KB
Digest: 520877a00f2b1f7506844246b950ab23 pass
Regards, Mark.
On 26 Feb 2026, at 8:47 AM, Mark Webb-Johnson via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store ota partition downgrade store ota partition upgrade factory
I’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this? Copy running ota image to "factory", reboot into "factory" Upgrade store partition, reboot Upgrade OTA partitions, reboot If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
Michael,
> AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
Here are the three partition table options:
Original (f12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 factory app factory 0x10000 0x400000 410000 65536 4194304 ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Current (12):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 store data fat 0xC10000 0x100000 D10000 12648448 1048576 *unused*
0xD10000 0x2F0000 1000000 13697024 3080192
Suggested (12, 7MB code, more store):
Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 nvs data nvs 0x9000 0x4000 D000 36864 16384 otadata data ota 0xD000 0x2000 F000 53248 8192 phy_init data phy 0xF000 0x1000 10000 61440 4096 ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: Create a 2nd fat store partition (0xE10000-0xFFFFFF) Format and mount that as FAT Copy files from old to new Unmount and drop the old store Finish the rest of the partition upgrade
I suspect #3 would be the most complex step.
Or the store move and resize could be done first as a separate step.
Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
… and I forgot to mention you need to ensure that you are running from factory before doing this.
On 17 Apr 2026, at 4:27 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This has been a nightmare. Then a digression migrating servers. Then more of a nightmare.
Firstly, please don’t do these commands on production modules at the moment. Things may change.
Secondly, only do these commands on boxes with standard partition tables and flash sizes.
The sequence that currently works is as follows:
ota partitions store upgrade module reset (aka ‘reboot’) sd unmount ota partitions store migrate ota partitions upgrade factory module reset (aka ‘reboot’)
You can use ’ota status nocheck' and ‘ota partitions list’ to see status along the way.
This is my fourth re-write of this, and the best I can get it at the moment. It needs another set of eyes looking over it, because the process is fraught with danger along the way.
Some comments (all learned the hard way):
After making the new store2 partition, it can’t be mounted before reboot (I guess that the esp idf caches the partition table at boot). Our esp idf components/fatfs/src/ffconf.h has #defined FF_VOLUMES 2 which means we can’t mount more than two FAT filesystems. /store and /store2 for migration, but what about SD CARD? We can increase that (our esp idf), but even then a lot of our code assumes 0: is /store and 1: is /sd, and things got messy when I tried. The recursive copy /store to /store2 without using a lot of stack was tricky, but seems to be ok and as reliable as I could make it. I keep a list of directories to copy, to avoid direct subroutine call recursion. There is a ’ota partitions store downgrade’ to go back before step 5. We’ll probably want to move ‘ota partitions store mount/unmount’ out before production, as it opens us up to the user passwords being accessible - or at least add those /store2 paths to the protected paths list. The good news is I learnt a lot about the internals of ESP IDF, fatfs, partitions, et al.
At the moment this is ugly and not idiot proof. We are also down to about 60KB of free flash space (4MB limit).
Please have a play with it. Happy to accept suggestions as to how to get this usable and production ready.
OVMS# vfs df /store: Size: 1936 kB Avail: 1804 kB Used: 7% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passRegards, Mark.
On 26 Feb 2026, at 8:47 AM, Mark Webb-Johnson via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store ota partition downgrade store ota partition upgrade factory
I’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this? Copy running ota image to "factory", reboot into "factory" Upgrade store partition, reboot Upgrade OTA partitions, reboot If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter@expeedo.de> <mailto:dexter@expeedo.de> wrote:
@Craig: nice find!
Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson: > Michael, > >> AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure. > > I thought it did that check. Maybe not working?
No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
> > Here are the three partition table options: > > Original (f12): > > Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) > bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 > partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 > nvs data nvs 0x9000 0x4000 D000 36864 16384 > otadata data ota 0xD000 0x2000 F000 53248 8192 > phy_init data phy 0xF000 0x1000 10000 61440 4096 > factory app factory 0x10000 0x400000 410000 65536 4194304 > ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304 > ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304 > store data fat 0xC10000 0x100000 D10000 12648448 1048576 > *unused* > > 0xD10000 0x2F0000 1000000 13697024 3080192 > > Current (12): > > Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) > bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 > partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 > nvs data nvs 0x9000 0x4000 D000 36864 16384 > otadata data ota 0xD000 0x2000 F000 53248 8192 > phy_init data phy 0xF000 0x1000 10000 61440 4096 > ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 > ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456 > store data fat 0xC10000 0x100000 D10000 12648448 1048576 > *unused* > > 0xD10000 0x2F0000 1000000 13697024 3080192 > > Suggested (12, 7MB code, more store): > > Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec) > bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768 > partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096 > nvs data nvs 0x9000 0x4000 D000 36864 16384 > otadata data ota 0xD000 0x2000 F000 53248 8192 > phy_init data phy 0xF000 0x1000 10000 61440 4096 > ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 > ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032 > store data fat 0xE10000 0x1F0000 1000000 14745600 2031616 > > It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked. > > Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative: > Create a 2nd fat store partition (0xE10000-0xFFFFFF) > Format and mount that as FAT > Copy files from old to new > Unmount and drop the old store > Finish the rest of the partition upgrade > > I suspect #3 would be the most complex step. > > Or the store move and resize could be done first as a separate step. > > Thoughts? Much more complex and risky...
There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
I second the store resize should be the first upgrade step:
Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
… and you forgot to push ;-)
Regards, Michael
Am 17.04.26 um 10:30 schrieb Mark Webb-Johnson:
… and I forgot to mention you need to ensure that you are running from factory before doing this.
On 17 Apr 2026, at 4:27 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This has been a nightmare. Then a digression migrating servers. Then more of a nightmare.
Firstly, please don’t do these commands on production modules at the moment. Things may change.
Secondly, only do these commands on boxes with standard partition tables and flash sizes.
The sequence that currently works is as follows:
- ota partitions store upgrade
- module reset (aka ‘reboot’)
- sd unmount
- ota partitions store migrate
- ota partitions upgrade factory
- module reset (aka ‘reboot’)
You can use ’ota status nocheck' and ‘ota partitions list’ to see status along the way.
This is my fourth re-write of this, and the best I can get it at the moment. It needs another set of eyes looking over it, because the process is fraught with danger along the way.
Some comments (all learned the hard way):
- After making the new store2 partition, it can’t be mounted before reboot (I guess that the esp idf caches the partition table at boot).
- Our esp idf components/fatfs/src/ffconf.h has #defined FF_VOLUMES 2 which means we can’t mount more than two FAT filesystems. /store and /store2 for migration, but what about SD CARD? We can increase that (our esp idf), but even then a lot of our code assumes 0: is /store and 1: is /sd, and things got messy when I tried.
- The recursive copy /store to /store2 without using a lot of stack was tricky, but seems to be ok and as reliable as I could make it. I keep a list of directories to copy, to avoid direct subroutine call recursion.
- There is a ’ota partitions store downgrade’ to go back before step 5.
- We’ll probably want to move ‘ota partitions store mount/unmount’ out before production, as it opens us up to the user passwords being accessible - or at least add those /store2 paths to the protected paths list.
- The good news is I learnt a lot about the internals of ESP IDF, fatfs, partitions, et al.
At the moment this is ugly and not idiot proof. We are also down to about 60KB of free flash space (4MB limit).
Please have a play with it. Happy to accept suggestions as to how to get this usable and production ready.
OVMS# vfs df /store: Size: 1936 kB Avail: 1804 kB Used: 7% /sd : Size: 3802 MB Avail: 3787 MB Used: 1% OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passRegards, Mark.
On 26 Feb 2026, at 8:47 AM, Mark Webb-Johnson via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store ota partition downgrade store ota partition upgrade factoryI’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this?
- Copy running ota image to "factory", reboot into "factory"
- Upgrade store partition, reboot
- Upgrade OTA partitions, reboot
If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
> On Feb 25, 2026, at 3:59 PM, Michael Balzer<dexter@expeedo.de>wrote: > > @Craig: nice find! > > Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson: >> Michael, >> >>> AFAICT "ota partitions upgrade" currently does not verify the >>> system has booted from "factory" before allowing the operation >>> -- I think that should be added as a safety measure. >> >> I thought it did that check. Maybe not working? > > No, you're right, should be OK that way. But as Craig's result > shows, I still should test it ;-) > >> >> Here are the three partition table options: >> >> Original (f12): >> >> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >> Offset (Dec) Size (Dec) >> bootloader bootloader bootloader 0x0000 0x8000 8000 0 >> 32768 >> partition_table partition_table partition_table 0x8000 >> 0x1000 9000 32768 4096 >> nvs data nvs 0x9000 0x4000 D000 36864 16384 >> otadata data ota 0xD000 0x2000 F000 53248 8192 >> phy_init data phy 0xF000 0x1000 10000 61440 4096 >> factory app factory 0x10000 0x400000 410000 65536 >> 4194304 >> ota_0 app ota_0 0x410000 0x400000 810000 4259840 >> 4194304 >> ota_1 app ota_1 0x810000 0x400000 C10000 8454144 >> 4194304 >> store data fat 0xC10000 0x100000 D10000 12648448 >> 1048576 >> *unused* >> >> 0xD10000 0x2F0000 1000000 13697024 3080192 >> >> >> Current (12): >> >> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >> Offset (Dec) Size (Dec) >> bootloader bootloader bootloader 0x0000 0x8000 8000 0 >> 32768 >> partition_table partition_table partition_table 0x8000 >> 0x1000 9000 32768 4096 >> nvs data nvs 0x9000 0x4000 D000 36864 16384 >> otadata data ota 0xD000 0x2000 F000 53248 8192 >> phy_init data phy 0xF000 0x1000 10000 61440 4096 >> ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 >> ota_1 app ota_1 0x610000 0x600000 C10000 6356992 >> 6291456 >> store data fat 0xC10000 0x100000 D10000 12648448 >> 1048576 >> *unused* >> >> 0xD10000 0x2F0000 1000000 13697024 3080192 >> >> >> Suggested (12, 7MB code, more store): >> >> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >> Offset (Dec) Size (Dec) >> bootloader bootloader bootloader 0x0000 0x8000 8000 0 >> 32768 >> partition_table partition_table partition_table 0x8000 >> 0x1000 9000 32768 4096 >> nvs data nvs 0x9000 0x4000 D000 36864 16384 >> otadata data ota 0xD000 0x2000 F000 53248 8192 >> phy_init data phy 0xF000 0x1000 10000 61440 4096 >> ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 >> ota_1 app ota_1 0x710000 0x700000 E10000 7405568 >> 7340032 >> store data fat 0xE10000 0x1F0000 1000000 14745600 >> 2031616 >> >> >> It would make sense to resize store *now* if we are going to do >> it, but I am wary of requiring a factory reset. Also need to >> double check that the unused space at the end is truly unused. >> I am assuming the boot loader is in the first 32KB, but haven’t >> actually checked. >> >> Looking at the third (resize store) arrangement, it would be >> moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t >> seem to overlap. I wonder what simply copy that partition over >> (without reformatting flash) would do? Either corrupt the >> filesystem, ignore the extra space, or magically have the new >> space available to FAT? I suspect corruption. So, there is >> another (more complex) alternative: >> >> 1. Create a 2nd fat store partition (0xE10000-0xFFFFFF) >> 2. Format and mount that as FAT >> 3. Copy files from old to new >> 4. Unmount and drop the old store >> 5. Finish the rest of the partition upgrade >> >> >> I suspect #3 would be the most complex step. >> >> Or the store move and resize could be done first as a separate >> step. >> >> Thoughts? Much more complex and risky... > > There's the fatresize tool, didn't check the code, but given we > can simply create the new partition cleanly and copy the files > there, we should go that way. > > I second the store resize should be the first upgrade step: > > Unmounting /store while running may be risky, that was never > needed and up to now only occurrs during shutdown as a final step. > > I suggest we just relable the old "store" partition and do a > reboot instead. esp_vfs_fat_spiflash_mount() searches for the > partition by name, so the system should automatically mount & > use the new "store" on boot. > > Regards, > Michael > > -- > Michael Balzer * Am Rahmen 5 * D-58313 Herdecke > Fon 02330 9104094 * Handy 0176 20698926 >
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
First test result: I've performed the full upgrade&migrate procedure on my test module, starting from the old partition scheme and running from ota, so with the initial copy to factory.
It all worked without any issue.
Migrate copied 94 files in 22 directories with a total of 620 kB.
OVMS# vfs df /store: Size: 976 kB Avail: 356 kB Used: 64%
→ migrate →
OVMS# vfs df /store: Size: 1936 kB Avail: 1316 kB Used: 33%
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-826-gb7d9ad0d9/ota_0/edge (build idf v3.3.4-854-g9063c8662c Apr 17 2026 13:10:34) Partition type: v3-35 (ota1, ota2, no factory, maximized store) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-826-gb7d9ad0d9
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass
I tried booting into main/3.3.005-3 after the full migration, which also still works flawlessly.
Btw, command ota is the same as ota status nocheck.
For others testing this: when mounting store2, vfs df will display
nonsense, as it assumes mount 1 is the SD card, also file/path name
completion doesn't know about /store2, you need to enter the path
manually. Use vfs rls /store2 to review the whole tree (be aware that
will omit the ovms_config path on /store).
But I second removing the store2 mount option ASAP, the output of ota partitions store migrate should be sufficient to determine success/failure.
I'll review the code next.
And yes, we need to automate the upgrade, a normal user should only need to trigger the process, not perform the steps needed manually. We should even think about initiating the upgrade automatically once we gain some trust in it.
As part of the automation, I think we should disable the auto init on the intermediary reboots, so the system runs on a minimal setup when performing the critical steps.
Regarding remaining space, there are some code parts we can remove/disable. Not sure how much that gives us, I'll have a look.
Regards, Michael
Am 17.04.26 um 10:30 schrieb Mark Webb-Johnson:
… and I forgot to mention you need to ensure that you are running from factory before doing this.
On 17 Apr 2026, at 4:27 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This has been a nightmare. Then a digression migrating servers. Then more of a nightmare.
Firstly, please don’t do these commands on production modules at the moment. Things may change.
Secondly, only do these commands on boxes with standard partition tables and flash sizes.
The sequence that currently works is as follows:
- ota partitions store upgrade
- module reset (aka ‘reboot’)
- sd unmount
- ota partitions store migrate
- ota partitions upgrade factory
- module reset (aka ‘reboot’)
You can use ’ota status nocheck' and ‘ota partitions list’ to see status along the way.
This is my fourth re-write of this, and the best I can get it at the moment. It needs another set of eyes looking over it, because the process is fraught with danger along the way.
Some comments (all learned the hard way):
- After making the new store2 partition, it can’t be mounted before reboot (I guess that the esp idf caches the partition table at boot).
- Our esp idf components/fatfs/src/ffconf.h has #defined FF_VOLUMES 2 which means we can’t mount more than two FAT filesystems. /store and /store2 for migration, but what about SD CARD? We can increase that (our esp idf), but even then a lot of our code assumes 0: is /store and 1: is /sd, and things got messy when I tried.
- The recursive copy /store to /store2 without using a lot of stack was tricky, but seems to be ok and as reliable as I could make it. I keep a list of directories to copy, to avoid direct subroutine call recursion.
- There is a ’ota partitions store downgrade’ to go back before step 5.
- We’ll probably want to move ‘ota partitions store mount/unmount’ out before production, as it opens us up to the user passwords being accessible - or at least add those /store2 paths to the protected paths list.
- The good news is I learnt a lot about the internals of ESP IDF, fatfs, partitions, et al.
At the moment this is ugly and not idiot proof. We are also down to about 60KB of free flash space (4MB limit).
Please have a play with it. Happy to accept suggestions as to how to get this usable and production ready.
OVMS# vfs df /store: Size: 1936 kB Avail: 1804 kB Used: 7% /sd : Size: 3802 MB Avail: 3787 MB Used: 1% OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passRegards, Mark.
On 26 Feb 2026, at 8:47 AM, Mark Webb-Johnson via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store ota partition downgrade store ota partition upgrade factoryI’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this?
- Copy running ota image to "factory", reboot into "factory"
- Upgrade store partition, reboot
- Upgrade OTA partitions, reboot
If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
Craig, Michael,
I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
I also experimented with store.
I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
Thoughts?
Regards, Mark.
> On Feb 25, 2026, at 3:59 PM, Michael Balzer<dexter@expeedo.de>wrote: > > @Craig: nice find! > > Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson: >> Michael, >> >>> AFAICT "ota partitions upgrade" currently does not verify the >>> system has booted from "factory" before allowing the operation >>> -- I think that should be added as a safety measure. >> >> I thought it did that check. Maybe not working? > > No, you're right, should be OK that way. But as Craig's result > shows, I still should test it ;-) > >> >> Here are the three partition table options: >> >> Original (f12): >> >> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >> Offset (Dec) Size (Dec) >> bootloader bootloader bootloader 0x0000 0x8000 8000 0 >> 32768 >> partition_table partition_table partition_table 0x8000 >> 0x1000 9000 32768 4096 >> nvs data nvs 0x9000 0x4000 D000 36864 16384 >> otadata data ota 0xD000 0x2000 F000 53248 8192 >> phy_init data phy 0xF000 0x1000 10000 61440 4096 >> factory app factory 0x10000 0x400000 410000 65536 >> 4194304 >> ota_0 app ota_0 0x410000 0x400000 810000 4259840 >> 4194304 >> ota_1 app ota_1 0x810000 0x400000 C10000 8454144 >> 4194304 >> store data fat 0xC10000 0x100000 D10000 12648448 >> 1048576 >> *unused* >> >> 0xD10000 0x2F0000 1000000 13697024 3080192 >> >> >> Current (12): >> >> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >> Offset (Dec) Size (Dec) >> bootloader bootloader bootloader 0x0000 0x8000 8000 0 >> 32768 >> partition_table partition_table partition_table 0x8000 >> 0x1000 9000 32768 4096 >> nvs data nvs 0x9000 0x4000 D000 36864 16384 >> otadata data ota 0xD000 0x2000 F000 53248 8192 >> phy_init data phy 0xF000 0x1000 10000 61440 4096 >> ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 >> ota_1 app ota_1 0x610000 0x600000 C10000 6356992 >> 6291456 >> store data fat 0xC10000 0x100000 D10000 12648448 >> 1048576 >> *unused* >> >> 0xD10000 0x2F0000 1000000 13697024 3080192 >> >> >> Suggested (12, 7MB code, more store): >> >> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >> Offset (Dec) Size (Dec) >> bootloader bootloader bootloader 0x0000 0x8000 8000 0 >> 32768 >> partition_table partition_table partition_table 0x8000 >> 0x1000 9000 32768 4096 >> nvs data nvs 0x9000 0x4000 D000 36864 16384 >> otadata data ota 0xD000 0x2000 F000 53248 8192 >> phy_init data phy 0xF000 0x1000 10000 61440 4096 >> ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 >> ota_1 app ota_1 0x710000 0x700000 E10000 7405568 >> 7340032 >> store data fat 0xE10000 0x1F0000 1000000 14745600 >> 2031616 >> >> >> It would make sense to resize store *now* if we are going to do >> it, but I am wary of requiring a factory reset. Also need to >> double check that the unused space at the end is truly unused. >> I am assuming the boot loader is in the first 32KB, but haven’t >> actually checked. >> >> Looking at the third (resize store) arrangement, it would be >> moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t >> seem to overlap. I wonder what simply copy that partition over >> (without reformatting flash) would do? Either corrupt the >> filesystem, ignore the extra space, or magically have the new >> space available to FAT? I suspect corruption. So, there is >> another (more complex) alternative: >> >> 1. Create a 2nd fat store partition (0xE10000-0xFFFFFF) >> 2. Format and mount that as FAT >> 3. Copy files from old to new >> 4. Unmount and drop the old store >> 5. Finish the rest of the partition upgrade >> >> >> I suspect #3 would be the most complex step. >> >> Or the store move and resize could be done first as a separate >> step. >> >> Thoughts? Much more complex and risky... > > There's the fatresize tool, didn't check the code, but given we > can simply create the new partition cleanly and copy the files > there, we should go that way. > > I second the store resize should be the first upgrade step: > > Unmounting /store while running may be risky, that was never > needed and up to now only occurrs during shutdown as a final step. > > I suggest we just relable the old "store" partition and do a > reboot instead. esp_vfs_fat_spiflash_mount() searches for the > partition by name, so the system should automatically mount & > use the new "store" on boot. > > Regards, > Michael > > -- > Michael Balzer * Am Rahmen 5 * D-58313 Herdecke > Fon 02330 9104094 * Handy 0176 20698926 >
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Pushed my review results, please have a look, Mark.
- add erasing the flash region to store upgrade to ensure clean start and/or reformat in case of a corrupted filesystem - add store2 unmount to store downgrade - add file size check to file copy to detect changes and a common case of /store filesystem corruptions (metadata mismatch)
The file size check won't catch data changes, its main purpose is detecting a /store corruption, where I often had the effect of files seemingly being OK with size in the directory listing, but apparently empty on reading.
We could build & compare MD5 checksums for both files (like vfs_stat
does) to also catch changes resulting in equal sizes, but I think that's
not necessary and potentially slowing down the copy significantly.
Regards, Michael
Am 17.04.26 um 15:05 schrieb Michael Balzer via OvmsDev:
First test result: I've performed the full upgrade&migrate procedure on my test module, starting from the old partition scheme and running from ota, so with the initial copy to factory.
It all worked without any issue.
Migrate copied 94 files in 22 directories with a total of 620 kB.
OVMS# vfs df /store: Size: 976 kB Avail: 356 kB Used: 64%
→ migrate →
OVMS# vfs df /store: Size: 1936 kB Avail: 1316 kB Used: 33%
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-826-gb7d9ad0d9/ota_0/edge (build idf v3.3.4-854-g9063c8662c Apr 17 2026 13:10:34) Partition type: v3-35 (ota1, ota2, no factory, maximized store) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-826-gb7d9ad0d9
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass
I tried booting into main/3.3.005-3 after the full migration, which also still works flawlessly.
Btw, command
otais the same asota status nocheck.For others testing this: when mounting store2,
vfs dfwill display nonsense, as it assumes mount 1 is the SD card, also file/path name completion doesn't know about/store2, you need to enter the path manually. Usevfs rls /store2to review the whole tree (be aware that will omit theovms_configpath on/store).But I second removing the store2 mount option ASAP, the output of
ota partitions store migrateshould be sufficient to determine success/failure.I'll review the code next.
And yes, we need to automate the upgrade, a normal user should only need to trigger the process, not perform the steps needed manually. We should even think about initiating the upgrade automatically once we gain some trust in it.
As part of the automation, I think we should disable the auto init on the intermediary reboots, so the system runs on a minimal setup when performing the critical steps.
Regarding remaining space, there are some code parts we can remove/disable. Not sure how much that gives us, I'll have a look.
Regards, Michael
Am 17.04.26 um 10:30 schrieb Mark Webb-Johnson:
… and I forgot to mention you need to ensure that you are running from factory before doing this.
On 17 Apr 2026, at 4:27 PM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
This has been a nightmare. Then a digression migrating servers. Then more of a nightmare.
Firstly, please don’t do these commands on production modules at the moment. Things may change.
Secondly, only do these commands on boxes with standard partition tables and flash sizes.
The sequence that currently works is as follows:
- ota partitions store upgrade
- module reset (aka ‘reboot’)
- sd unmount
- ota partitions store migrate
- ota partitions upgrade factory
- module reset (aka ‘reboot’)
You can use ’ota status nocheck' and ‘ota partitions list’ to see status along the way.
This is my fourth re-write of this, and the best I can get it at the moment. It needs another set of eyes looking over it, because the process is fraught with danger along the way.
Some comments (all learned the hard way):
- After making the new store2 partition, it can’t be mounted before reboot (I guess that the esp idf caches the partition table at boot).
- Our esp idf components/fatfs/src/ffconf.h has #defined FF_VOLUMES 2 which means we can’t mount more than two FAT filesystems. /store and /store2 for migration, but what about SD CARD? We can increase that (our esp idf), but even then a lot of our code assumes 0: is /store and 1: is /sd, and things got messy when I tried.
- The recursive copy /store to /store2 without using a lot of stack was tricky, but seems to be ok and as reliable as I could make it. I keep a list of directories to copy, to avoid direct subroutine call recursion.
- There is a ’ota partitions store downgrade’ to go back before step 5.
- We’ll probably want to move ‘ota partitions store mount/unmount’ out before production, as it opens us up to the user passwords being accessible - or at least add those /store2 paths to the protected paths list.
- The good news is I learnt a lot about the internals of ESP IDF, fatfs, partitions, et al.
At the moment this is ugly and not idiot proof. We are also down to about 60KB of free flash space (4MB limit).
Please have a play with it. Happy to accept suggestions as to how to get this usable and production ready.
OVMS# vfs df /store: Size: 1936 kB Avail: 1804 kB Used: 7% /sd : Size: 3802 MB Avail: 3787 MB Used: 1% OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passRegards, Mark.
On 26 Feb 2026, at 8:47 AM, Mark Webb-Johnson via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store ota partition downgrade store ota partition upgrade factoryI’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something fails.
So the process will now become this?
- Copy running ota image to "factory", reboot into "factory"
- Upgrade store partition, reboot
- Upgrade OTA partitions, reboot
If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
Regards, Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
Mark,
I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
backup_dir[] = { { "ovms_config", false }, { "events", true }, { "scripts", true }, { "obd2ecu", true }, { "dbc", true }, { "plugin", true }, { "tls", true }, { "trustedca", true }, { "usr", true }, { NULL, false } };
We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
Regards, Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson: > Craig, Michael, > > I have fixed the (a) partition type in ‘ota’ after upgrade, and > (b) copy to factory. Both commits pushed. > > I also experimented with store. > > * I can modify the partition table, rename old store to > xstore, and create a new store at 0xe10000 same size 1MB, > copy over the store partition. Reboot and all comes up fine > (with repositioned store accessible). > > > * If I do the same, but set new store size to 0x1f0000, then > the store can’t be mounted (corrupt, presumably). I guess > the wear-leveling thing, or maybe internal fat filesystem > metadata is messing things up. > > > So it seems that either we leave store at 1MB (move it to the > end of flash and make the code partitions as big as we can), or > we resize it to the 0x1f0000 and handle the complexities of > first modifying the partition table for the new store, then > mounting two fat file systems, recursively copying over the > files, and then adjusting the partition tables again. The > recursive copy worries me (memory consumption and if deeply > nested then all those directory handles open at the same time - > one for each level). > > Thoughts? > > Regards, Mark. > >> On Feb 25, 2026, at 3:59 PM, Michael >> Balzer<dexter@expeedo.de>wrote: >> >> @Craig: nice find! >> >> Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson: >>> Michael, >>> >>>> AFAICT "ota partitions upgrade" currently does not verify the >>>> system has booted from "factory" before allowing the >>>> operation -- I think that should be added as a safety measure. >>> >>> I thought it did that check. Maybe not working? >> >> No, you're right, should be OK that way. But as Craig's result >> shows, I still should test it ;-) >> >>> >>> Here are the three partition table options: >>> >>> Original (f12): >>> >>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >>> Offset (Dec) Size (Dec) >>> bootloader bootloader bootloader 0x0000 0x8000 8000 >>> 0 32768 >>> partition_table partition_table partition_table 0x8000 >>> 0x1000 9000 32768 4096 >>> nvs data nvs 0x9000 0x4000 D000 36864 16384 >>> otadata data ota 0xD000 0x2000 F000 53248 8192 >>> phy_init data phy 0xF000 0x1000 10000 61440 4096 >>> factory app factory 0x10000 0x400000 410000 65536 >>> 4194304 >>> ota_0 app ota_0 0x410000 0x400000 810000 4259840 >>> 4194304 >>> ota_1 app ota_1 0x810000 0x400000 C10000 8454144 >>> 4194304 >>> store data fat 0xC10000 0x100000 D10000 12648448 >>> 1048576 >>> *unused* >>> >>> 0xD10000 0x2F0000 1000000 13697024 3080192 >>> >>> >>> Current (12): >>> >>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >>> Offset (Dec) Size (Dec) >>> bootloader bootloader bootloader 0x0000 0x8000 8000 >>> 0 32768 >>> partition_table partition_table partition_table 0x8000 >>> 0x1000 9000 32768 4096 >>> nvs data nvs 0x9000 0x4000 D000 36864 16384 >>> otadata data ota 0xD000 0x2000 F000 53248 8192 >>> phy_init data phy 0xF000 0x1000 10000 61440 4096 >>> ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456 >>> ota_1 app ota_1 0x610000 0x600000 C10000 6356992 >>> 6291456 >>> store data fat 0xC10000 0x100000 D10000 12648448 >>> 1048576 >>> *unused* >>> >>> 0xD10000 0x2F0000 1000000 13697024 3080192 >>> >>> >>> Suggested (12, 7MB code, more store): >>> >>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) >>> Offset (Dec) Size (Dec) >>> bootloader bootloader bootloader 0x0000 0x8000 8000 >>> 0 32768 >>> partition_table partition_table partition_table 0x8000 >>> 0x1000 9000 32768 4096 >>> nvs data nvs 0x9000 0x4000 D000 36864 16384 >>> otadata data ota 0xD000 0x2000 F000 53248 8192 >>> phy_init data phy 0xF000 0x1000 10000 61440 4096 >>> ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032 >>> ota_1 app ota_1 0x710000 0x700000 E10000 7405568 >>> 7340032 >>> store data fat 0xE10000 0x1F0000 1000000 14745600 >>> 2031616 >>> >>> >>> It would make sense to resize store *now* if we are going to >>> do it, but I am wary of requiring a factory reset. Also need >>> to double check that the unused space at the end is truly >>> unused. I am assuming the boot loader is in the first 32KB, >>> but haven’t actually checked. >>> >>> Looking at the third (resize store) arrangement, it would be >>> moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t >>> seem to overlap. I wonder what simply copy that partition over >>> (without reformatting flash) would do? Either corrupt the >>> filesystem, ignore the extra space, or magically have the new >>> space available to FAT? I suspect corruption. So, there is >>> another (more complex) alternative: >>> >>> 1. Create a 2nd fat store partition (0xE10000-0xFFFFFF) >>> 2. Format and mount that as FAT >>> 3. Copy files from old to new >>> 4. Unmount and drop the old store >>> 5. Finish the rest of the partition upgrade >>> >>> >>> I suspect #3 would be the most complex step. >>> >>> Or the store move and resize could be done first as a separate >>> step. >>> >>> Thoughts? Much more complex and risky... >> >> There's the fatresize tool, didn't check the code, but given we >> can simply create the new partition cleanly and copy the files >> there, we should go that way. >> >> I second the store resize should be the first upgrade step: >> >> Unmounting /store while running may be risky, that was never >> needed and up to now only occurrs during shutdown as a final step. >> >> I suggest we just relable the old "store" partition and do a >> reboot instead. esp_vfs_fat_spiflash_mount() searches for the >> partition by name, so the system should automatically mount & >> use the new "store" on boot. >> >> Regards, >> Michael >> >> -- >> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke >> Fon 02330 9104094 * Handy 0176 20698926 >> >
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Mark,
Am 18.04.26 um 00:17 schrieb Michael Balzer via OvmsDev:
The sequence that currently works is as follows:
- ota partitions store upgrade
- module reset (aka ‘reboot’)
- sd unmount
- ota partitions store migrate
- ota partitions upgrade factory
- module reset (aka ‘reboot’)
What's the reasoning behind not letting ota partitions store migrate
do the SD unmounting if needed?
Integrating this would simplify the process and the upgrade wizard logic, and I don't see the necessity to require the user doing this. Do I miss something?
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
I recall when I tried it I found the unmount function wasn’t atomic, so not trivial to do.
I guess we could issue the shell command (which is atomic), but perhaps that would cause problems (with recursion into the shell)?
Regards, Mark
On Apr 22, 2026, at 3:12 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Mark,
Am 18.04.26 um 00:17 schrieb Michael Balzer via OvmsDev:
The sequence that currently works is as follows: ota partitions store upgrade module reset (aka ‘reboot’) sd unmount ota partitions store migrate ota partitions upgrade factory module reset (aka ‘reboot’)
What's the reasoning behind not letting
ota partitions store migratedo the SD unmounting if needed?Integrating this would simplify the process and the upgrade wizard logic, and I don't see the necessity to require the user doing this. Do I miss something?
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <OpenPGP_signature.asc>
On 4/21/26 15:04, Mark Webb-Johnson wrote:
I recall when I tried it I found the unmount function wasn’t atomic, so not trivial to do.
I guess we could issue the shell command (which is atomic), but perhaps that would cause problems (with recursion into the shell)?
I think what is needed is a way to poll sdcard::m_unmounting or even a new sdcard entry point that starts the unmount and waits for it to complete (or fail).
A problem I see is that if there are files open on the sdcard (e.g. a log file or can log), the unmount attempt will fail. So you probably want equivalent to "log close" before trying to unmount the sd.
Craig
OVMS# sd unmount Error: SD unmount failed: ESP_FAIL OVMS# log status Log listeners : 3 File logging status: inactive Log file path : /sd/logs/ovms-dev.log Current size : 550.5 kB Cycle size : 1024 kB Cycle count : 0 Dropped messages : 0 Messages logged : 3227 Total fsync time : 6.0 s
Completing the partition table re-work.
I think our current target arrangement is pretty good:
Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass
But we have a number of things to resolve before ready for production. The ones I can think of are:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
Feedback appreciated. Is this an acceptable approach?
Regards, Mark
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme:
- "v3.3" = hardware 3.3, original partitioning
- "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning
That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Am 24.04.26 um 12:33 schrieb Michael Balzer via OvmsDev:
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Missed: I'll also do 3.2.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Am 24.04.26 um 12:33 schrieb Michael Balzer via OvmsDev:
The auto upgrade process could then do a store downgrade and _set a config variable to reflect the abort cause_ (should do so as well on other issues encountered, so we can inform the user).
Another / additional idea on this part: simply store the entire command
output as a file on /sd if available, else on /store if enough space
is left or on /store2 after a successful store migration.
Basically use BufferedShell::ExecuteCommand() or a StringWriter to
capture the command output(s), then save_file() to save the result to
a defined path.
That way we could make the full step result details available to users without a USB shell (display them in the web UI upgrade guide), and keep a way for us to get captured logs for analysis in case of issues.
The largest command output is that of store migrate, on my test module
this step's output was 9599 bytes in size. So this will generally work
on /store even with much more files than my test module has (which
already covers multiple vehicles).
Of course only unless /store is corrupted. But a corrupted /store
will normally also result in all writes being lost anyway, so no config
variable change would be kept, the auto upgrade would appear stuck in
step 1.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/…
v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list
Partition table:
Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB
otadata data ota 0x0000d000 8 KB
phy_init data phy 0x0000f000 4 KB
ota_0 app ota_0 0x00010000 7 MB
ota_1 app ota_1 0x00710000 7 MB
store data fat 0x00e10000 1984 KB
Digest: 520877a00f2b1f7506844246b950ab23 pass
OVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme: "v3.3" = hardware 3.3, original partitioning "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/… v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something
like GetPath() & SetPath()?
I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass OVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme:
- "v3.3" = hardware 3.3, original partitioning
- "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning
That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback): - Type: Page - Page: /dev/partitioning - Label: Partitioning (dev) - Menu: Config - Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/… v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass OVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme:
- "v3.3" = hardware 3.3, original partitioning
- "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning
That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Thank you, Michael. I’m trying to test, but I’m having trouble getting <partitioning.htm> into OVMS/store/plugin. Tried “curl”, but getting what appears to be a “security” error in accessing the source file. Apologies if this is a naive error on my part.
Chip Cangialose
On Apr 30, 2026, at 2:20 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback):
- Type: Page
- Page: /dev/partitioning
- Label: Partitioning (dev)
- Menu: Config
- Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/…
v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passOVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme: "v3.3" = hardware 3.3, original partitioning "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Chip,
see here: https://docs.openvehicles.com/en/latest/plugin/README.html#installing-web-pl...
Regards, Michael
Am 01.05.26 um 03:04 schrieb Charles B Cangialose:
Thank you, Michael. I’m trying to test, but I’m having trouble getting <partitioning.htm> into OVMS/store/plugin. Tried “curl”, but getting what appears to be a “security” error in accessing the source file. Apologies if this is a naive error on my part.
Chip Cangialose
On Apr 30, 2026, at 2:20 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback): - Type: Page - Page: /dev/partitioning - Label: Partitioning (dev) - Menu: Config - Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/… v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass OVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
> 1) Making it simple > 1.1) Automate the steps > The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
> 1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
> 1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme:
- "v3.3" = hardware 3.3, original partitioning
- "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning
That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
> 2) 4MB vs 7MB firmware > 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
> 2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
> 2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
> 3) Notifying users > 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
> 4) Versioning > 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
> 5) Factory > 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Thank you. Will make second attempt this afo.
On Apr 30, 2026, at 10:45 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Chip,
see here: https://docs.openvehicles.com/en/latest/plugin/README.html#installing-web-pl...
Regards, Michael
Am 01.05.26 um 03:04 schrieb Charles B Cangialose:
Thank you, Michael. I’m trying to test, but I’m having trouble getting <partitioning.htm> into OVMS/store/plugin. Tried “curl”, but getting what appears to be a “security” error in accessing the source file. Apologies if this is a naive error on my part.
Chip Cangialose
On Apr 30, 2026, at 2:20 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback):
- Type: Page
- Page: /dev/partitioning
- Label: Partitioning (dev)
- Menu: Config
- Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/…
v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passOVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson: > 1) Making it simple > 1.1) Automate the steps > 1.2) Provide a user interface > 1.3) Module informing servers of status (so we can see progress) > > 2) 4MB vs 7MB firmware > 2.1) Server support builds > 2.2) Module downloads correct build > 2.3) Firmware fitting in the partition > > 3) Notifying users > 3.1) Developers > 3.2) End users > > 4) Versioning > 4.1) Firmware versions > > 5) Factory > 5.1) Factory production support > > So let's look at each of these, and I make a proposal how to solve. > >> 1) Making it simple >> 1.1) Automate the steps >> > The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up. > > My suggestion here is to: > > a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade. > > b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary. > > c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot). > > d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure. > > I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
> >> 1.2) Provide a user interface > > Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
>> 1.3) Module informing servers of status (so we can see progress) > > In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3. > > In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version. > > It seems that we can quite simply: > > a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout. > > b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string. > > The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme: "v3.3" = hardware 3.3, original partitioning "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
>> 2) 4MB vs 7MB firmware >> 2.1) Server support builds > > We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment). > >> 2.2) Module downloads correct build > > The GetOVMSProduct() change in 1.3(a) above should resolve this. > >> 2.3) Firmware fitting in the partition > > This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
>> 3) Notifying users >> 3.1) Developers > > I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
> 3.2) End users > > I can think of two approaches: > > a) A pop-up on web interface main screen if partition table upgrade is available. > > b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
>> 4) Versioning >> 4.1) Firmware versions > > If we follow the approach in 1.3 above, no change to current versioning scheme is needed. > > There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
>> 5) Factory >> 5.1) Factory production support > > Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory. > > I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1. >
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
I was able to implement the Partitioning plugin, but the page did not give me the option to “Upgrade”, see below. I’m guessing that this may be due to deprecated hardware, or firmware (?).

On Apr 30, 2026, at 10:45 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Chip,
see here: https://docs.openvehicles.com/en/latest/plugin/README.html#installing-web-pl...
Regards, Michael
Am 01.05.26 um 03:04 schrieb Charles B Cangialose:
Thank you, Michael. I’m trying to test, but I’m having trouble getting <partitioning.htm> into OVMS/store/plugin. Tried “curl”, but getting what appears to be a “security” error in accessing the source file. Apologies if this is a naive error on my part.
Chip Cangialose
On Apr 30, 2026, at 2:20 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback):
- Type: Page
- Page: /dev/partitioning
- Label: Partitioning (dev)
- Menu: Config
- Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/…
v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passOVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson: > 1) Making it simple > 1.1) Automate the steps > 1.2) Provide a user interface > 1.3) Module informing servers of status (so we can see progress) > > 2) 4MB vs 7MB firmware > 2.1) Server support builds > 2.2) Module downloads correct build > 2.3) Firmware fitting in the partition > > 3) Notifying users > 3.1) Developers > 3.2) End users > > 4) Versioning > 4.1) Firmware versions > > 5) Factory > 5.1) Factory production support > > So let's look at each of these, and I make a proposal how to solve. > >> 1) Making it simple >> 1.1) Automate the steps >> > The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up. > > My suggestion here is to: > > a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade. > > b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary. > > c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot). > > d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure. > > I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
> >> 1.2) Provide a user interface > > Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
>> 1.3) Module informing servers of status (so we can see progress) > > In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3. > > In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version. > > It seems that we can quite simply: > > a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout. > > b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string. > > The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme: "v3.3" = hardware 3.3, original partitioning "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
>> 2) 4MB vs 7MB firmware >> 2.1) Server support builds > > We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment). > >> 2.2) Module downloads correct build > > The GetOVMSProduct() change in 1.3(a) above should resolve this. > >> 2.3) Firmware fitting in the partition > > This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
>> 3) Notifying users >> 3.1) Developers > > I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
> 3.2) End users > > I can think of two approaches: > > a) A pop-up on web interface main screen if partition table upgrade is available. > > b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
>> 4) Versioning >> 4.1) Firmware versions > > If we follow the approach in 1.3 above, no change to current versioning scheme is needed. > > There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
>> 5) Factory >> 5.1) Factory production support > > Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory. > > I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1. >
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Michael,
The UI and process worked for me, but two issues/comments:
First time I hit the upgrade button, I got a panic. Maybe something I did as I was copying factory->ota_0 and trying to do this on a module that it had been done and undone several dozen times already. The backtrace was not helpful, and module locked up (no auto reboot). A simple reboot brought it back to life, back in factory state. Second time was much smoother. I got the web screen showing me the factory partition update, and then reboot. But it never showed any more progress. The upgrade itself completed in the background after a few minutes just fine.
Regards, Mark.
P.S. I’ve updated the partitions.csv to new format, as that should be ok to do now.
On May 1, 2026, at 5:20 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback):
- Type: Page
- Page: /dev/partitioning
- Label: Partitioning (dev)
- Menu: Config
- Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/…
v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passOVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme: "v3.3" = hardware 3.3, original partitioning "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Mark,
thanks for testing.
OK, so there is a valid case for unplugging, that's when the module doesn't come back online by itself within 5 minutes. I'll try to describe that in a user friendly way.
Regarding the panic, maybe we should generally start the procedure by rebooting into upgrade mode? I.e. not perform the partition copy step from the fully booted system. As a side benefit, we would also get that step's log in the update log file. The cost is another reboot.
Regarding showing more progress in the UI, do you mean actual progress details from the intermediate reboots, or just a progress bar that roughly follows the time normally required (max), to give the user an idea about the remaining time?
The upgrade boot mode inhibits network init, so the browser cannot reconnect during the update. I assumed you had deliberately done that to minimize risk, so the UI currently is meant to only take care of the initial & final, and possibly intermediate state from failure.
We can change that and add some channel to transport details, the question is, is it worth the risk & effort. After a failure, the user can now retrieve the update log, which is normally sufficient. Also, the intermediate reboots for the partition table changes probably are too short for the browser to reconnect (takes ~20 seconds), so even with a network init, we probably still won't see progress in the UI until in store migration.
A purely time based frontend progress bar on the other hand would be simple to add. We could use the final timeout to tell the user it's OK to unplug the module now, as the process probably got stuck somewhere.
Regards, Michael
Am 01.05.26 um 13:06 schrieb Mark Webb-Johnson:
Michael,
The UI and process worked for me, but two issues/comments:
- First time I hit the upgrade button, I got a panic. Maybe something I did as I was copying factory->ota_0 and trying to do this on a module that it had been done and undone several dozen times already. The backtrace was not helpful, and module locked up (no auto reboot). A simple reboot brought it back to life, back in factory state.
- Second time was much smoother. I got the web screen showing me the factory partition update, and then reboot. But it never showed any more progress. The upgrade itself completed in the background after a few minutes just fine.
Regards, Mark.
P.S. I’ve updated the partitions.csv to new format, as that should be ok to do now.
On May 1, 2026, at 5:20 AM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Signed PGP part Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback): - Type: Page - Page: /dev/partitioning - Label: Partitioning (dev) - Menu: Config - Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/… v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass OVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
> 1) Making it simple > 1.1) Automate the steps > The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
> 1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
> 1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme:
- "v3.3" = hardware 3.3, original partitioning
- "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning
That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
> 2) 4MB vs 7MB firmware > 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
> 2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
> 2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
> 3) Notifying users > 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
> 4) Versioning > 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
> 5) Factory > 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/30/26 14:20, Michael Balzer via OvmsDev wrote:
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback): - Type: Page - Page: /dev/partitioning - Label: Partitioning (dev) - Menu: Config - Auth: Cookie I tried this today but I am clearly missing something fundamental.
I'm running 3.3.005-853-g014e4be1 on my dev module. I did not switch to the factory partition (I figured I would deal with that if the plugin didn't once I got the plugin working).
I created /store/plugin and scp'ed partitioning.htm there. I can edit it via //#/edit
When I visit /#/cfg/plugins and click on add, Page is already selected. I put /dev/partitioning into the "Unique name" input box and then there is nothing else I can do except click the off/on buttons; both "Edit" and "Save" are grayed out.
I tried a couple of of different browsers (firefox, chrome) but they all look the same.
Craig
Craig,
just follow the instructions: https://docs.openvehicles.com/en/latest/plugin/README.html#installing-web-pl...
Regards, Michael
Am 01.05.26 um 23:19 schrieb Craig Leres:
On 4/30/26 14:20, Michael Balzer via OvmsDev wrote:
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback): - Type: Page - Page: /dev/partitioning - Label: Partitioning (dev) - Menu: Config - Auth: Cookie I tried this today but I am clearly missing something fundamental.
I'm running 3.3.005-853-g014e4be1 on my dev module. I did not switch to the factory partition (I figured I would deal with that if the plugin didn't once I got the plugin working).
I created /store/plugin and scp'ed partitioning.htm there. I can edit it via //#/edit
When I visit /#/cfg/plugins and click on add, Page is already selected. I put /dev/partitioning into the "Unique name" input box and then there is nothing else I can do except click the off/on buttons; both "Edit" and "Save" are grayed out.
I tried a couple of of different browsers (firefox, chrome) but they all look the same.
Craig
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 5/1/26 14:23, Michael Balzer via OvmsDev wrote:
just follow the instructions: https://docs.openvehicles.com/en/latest/plugin/README.html#installing- web-plugins
When I visit /#/cfg/plugins and click on add, Page is already selected. I put /dev/partitioning into the "Unique name" input box and then there is nothing else I can do except click the off/on buttons; both "Edit" and "Save" are grayed out.
I put /dev/partitioning into the "Unique name" input box and then there is nothing else I can do except click the off/on buttons; both "Edit" and "Save" are grayed out.
Ah. So I think html 5 was blocking the form because I had put illegal chars into the input box ('/').
Craig
On 5/1/26 14:19, Craig Leres via OvmsDev wrote:
I created /store/plugin and scp'ed partitioning.htm there. I can edit it via //#/edit
I could not get the plugin to use the html file I had scp'ed to /store/plugin so I used cut-n-paste to put it in the plugin editor plugin content box.
After it copied the image to factory (and wanted to reboot), I clicked "Download" and got "Request error, please retry" (see attached). I clicked "Download" again and I watched from the serial while it rebooted. I could click download again but just get a log identical to the one that ends with "Request error, please retry".
Eventually I logged into module via ssh and could see that it had completed successfully. I logged back into the web gui and was able to download the entire log (Partition-upgrade(3).txt)
(It reminded me of the windows xp "task failed successfully" meme!)
I see in the final log that it created a /store/plugin/Partitioning directory but there are no files inside.
Craig
** WARNING: connection is not using a post-quantum key exchange algorithm. ** This session may be vulnerable to "store now, decrypt later" attacks. ** The server may need to be upgraded. See https://openssh.com/pq.html
Welcome to the Open Vehicle Monitoring System (OVMS) - SSH Console Firmware: 3.3.005-853-g014e4be1/ota_0/main Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-853-g014e4be1/ota_0/main (build idf v3.3.4-854-g9063c8662 May 1 2026 13:45:48, product v3.3-5) Partition type: v3-35 (ota1, ota2, no factory, maximized store) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-853-g014e4be1
Am 02.05.26 um 00:27 schrieb Craig Leres via OvmsDev:
On 5/1/26 14:19, Craig Leres via OvmsDev wrote:
I created /store/plugin and scp'ed partitioning.htm there. I can edit it via //#/edit
I could not get the plugin to use the html file I had scp'ed to /store/plugin so I used cut-n-paste to put it in the plugin editor plugin content box. I see in the final log that it created a /store/plugin/Partitioning directory but there are no files inside.
You probably made some mistake when copying to /store/plugin, as there normally are no directories in there.
After it copied the image to factory (and wanted to reboot), I clicked "Download" and got "Request error, please retry" (see attached). I clicked "Download" again and I watched from the serial while it rebooted. I could click download again but just get a log identical to the one that ends with "Request error, please retry".
Yes, the request error is irritating, I'll suppress that. It's the normal output of a command channel getting lost, here due to the reboot.
The download button was meant to be able to also download the initial command output, but it obviously can get misinterpreted, so I'll remove it. We won't need it anyway if we don't do the partition copy directly on command invocation.
Eventually I logged into module via ssh and could see that it had completed successfully. I logged back into the web gui and was able to download the entire log (Partition-upgrade(3).txt)
(It reminded me of the windows xp "task failed successfully" meme!)
With a little more patience, the web UI would have reloaded automatically and shown the final success page (after login).
I get from this that it's not sufficient to place the info on the update process…
The upgrade procedure runs*fully automatic*, you just need to start it. Depending on your configuration, the upgrade can take*up to 5 minutes*to finish, so please be patient.
In case something goes wrong, the module will reboot into standard mode, and you can use this page to display and download the upgrade log for further analysis.
…in the status info only. I'll split/move that into the dialog.
Regards, Michael
I see in the final log that it created a /store/plugin/Partitioning directory but there are no files inside.
Craig
** WARNING: connection is not using a post-quantum key exchange algorithm. ** This session may be vulnerable to "store now, decrypt later" attacks. ** The server may need to be upgraded. See https://openssh.com/pq.html
Welcome to the Open Vehicle Monitoring System (OVMS) - SSH Console Firmware: 3.3.005-853-g014e4be1/ota_0/main Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-853-g014e4be1/ota_0/main (build idf v3.3.4-854-g9063c8662 May 1 2026 13:45:48, product v3.3-5) Partition type: v3-35 (ota1, ota2, no factory, maximized store) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-853-g014e4be1
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 5/2/26 00:33, Michael Balzer via OvmsDev wrote:
After it copied the image to factory (and wanted to reboot), I clicked "Download" and got "Request error, please retry" (see attached). I clicked "Download" again and I watched from the serial while it rebooted. I could click download again but just get a log identical to the one that ends with "Request error, please retry".
Yes, the request error is irritating, I'll suppress that. It's the normal output of a command channel getting lost, here due to the reboot.
My only concern is that when I saw the error message I thought something had gone wrong.
The download button was meant to be able to also download the initial command output, but it obviously can get misinterpreted, so I'll remove it. We won't need it anyway if we don't do the partition copy directly on command invocation.
I actually like the download button; if something goes wrong you have more information that you might if things go really wrong. But I would prefer a unique name for each log.
Eventually I logged into module via ssh and could see that it had completed successfully. I logged back into the web gui and was able to download the entire log (Partition-upgrade(3).txt)
(It reminded me of the windows xp "task failed successfully" meme!)
With a little more patience, the web UI would have reloaded automatically and shown the final success page (after login).
I get from this that it's not sufficient to place the info on the update process…
Maybe a "don't navigate away from this page, the upgrade can take up to 5 minutes" message?
The upgrade procedure runs*fully automatic*, you just need to start it. Depending on your configuration, the upgrade can take*up to 5 minutes*to finish, so please be patient.
In case something goes wrong, the module will reboot into standard mode, and you can use this page to display and download the upgrade log for further analysis.
I saw/liked that feature.
Could we print progress messages on the serial console? (I might have been more patient if I had thought to watch the console earlier in the process.)
Craig
Everyone,
please find attached the improved version of the UI:
- Optimized wording & user guidance
- Suppress irritating request errors
- Include run time estimation & progress bar
- Include "unplug now ok" info on process timeout
- Include setup mode reconnect details with factory reset invocation
This needs to be tested in combination with a firmware including the latest commits to master.
Be aware that due to the build partition table now being updated, doing "make flash" on "master" is no longer sufficient to reset a module to the previous partitioning. To revert to the old partitioning including the latest changes, switch to the new test branch "partitioning-v3-30", and do the "make flash" from there.
The runtime estimations should normally exceed the actual time needed. I've also changed the initial time info to "up to 10 minutes", the process will normally take 5-6 minutes when starting from an ota boot (standard for normal users). With the estimation on the safe side, users shouldn't normally run into the "you may need to unplug now" state.
Feedback is welcome.
Btw, the culprit for the slow intermediate reboots is the modem off detection when no modem type has been configured (auto detect). That process may be optimizable, but that's a task for another day.
Regards, Michael
Am 02.05.26 um 20:08 schrieb Craig Leres:
On 5/2/26 00:33, Michael Balzer via OvmsDev wrote:
After it copied the image to factory (and wanted to reboot), I clicked "Download" and got "Request error, please retry" (see attached). I clicked "Download" again and I watched from the serial while it rebooted. I could click download again but just get a log identical to the one that ends with "Request error, please retry".
Yes, the request error is irritating, I'll suppress that. It's the normal output of a command channel getting lost, here due to the reboot.
My only concern is that when I saw the error message I thought something had gone wrong.
The download button was meant to be able to also download the initial command output, but it obviously can get misinterpreted, so I'll remove it. We won't need it anyway if we don't do the partition copy directly on command invocation.
I actually like the download button; if something goes wrong you have more information that you might if things go really wrong. But I would prefer a unique name for each log.
Eventually I logged into module via ssh and could see that it had completed successfully. I logged back into the web gui and was able to download the entire log (Partition-upgrade(3).txt)
(It reminded me of the windows xp "task failed successfully" meme!)
With a little more patience, the web UI would have reloaded automatically and shown the final success page (after login).
I get from this that it's not sufficient to place the info on the update process…
Maybe a "don't navigate away from this page, the upgrade can take up to 5 minutes" message?
The upgrade procedure runs*fully automatic*, you just need to start it. Depending on your configuration, the upgrade can take*up to 5 minutes*to finish, so please be patient.
In case something goes wrong, the module will reboot into standard mode, and you can use this page to display and download the upgrade log for further analysis.
I saw/liked that feature.
Could we print progress messages on the serial console? (I might have been more patient if I had thought to watch the console earlier in the process.)
Craig
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 5/3/26 06:04, Michael Balzer via OvmsDev wrote:
please find attached the improved version of the UI:
The progress bar is a nice touch (even it's only counting down an estimate).
This needs to be tested in combination with a firmware including the latest commits to master.
Could the partition page parse the running firmware version and refuse to upgrade if it is not at least the required version?
Everything went well with my test except the browser reconnect never happened (I got the "The expected upgrade time has passed" alert). I was watching the serial console so I could see when activity died down and things were quiet for awhile before the progress bar hit 100%.
I didn't look at how you actually implement the "reconnect to browser" feature but I did a couple of searches and found results where firefox (what I was using) can fail to reconnect; that might be why it failed for me with this and the previous version:
On 5/1/26 15:27, Craig Leres wrote:
Eventually I logged into module via ssh and could see that it had completed successfully.
The instructions should probably tell the user to first try manually reconnecting to the web gui (and logging in) and checking the partition upgrade page and only unplug/replug the module if the browser can't reconnect?
Be aware that due to the build partition table now being updated, doing "make flash" on "master" is no longer sufficient to reset a module to the previous partitioning. To revert to the old partitioning including the latest changes, switch to the new test branch "partitioning-v3-30", and do the "make flash" from there.
Since "edge" now uses the new partition layout I updated my ovmsinstall.sh script (attached) to use the old v3-30 when the -o flag is given. By default you end up with the new v3-35 layout.
Craig
Am 03.05.26 um 19:18 schrieb Craig Leres:
On 5/3/26 06:04, Michael Balzer via OvmsDev wrote:
This needs to be tested in combination with a firmware including the latest commits to master.
Could the partition page parse the running firmware version and refuse to upgrade if it is not at least the required version?
Once the tool is included in the firmware, the version is guaranteed to be sufficient.
Everything went well with my test except the browser reconnect never happened (I got the "The expected upgrade time has passed" alert). I was watching the serial console so I could see when activity died down and things were quiet for awhile before the progress bar hit 100%.
I didn't look at how you actually implement the "reconnect to browser" feature but I did a couple of searches and found results where firefox (what I was using) can fail to reconnect;
OK, I think I know what happened: the auto reload of the framework depends on the new monotonic time received being lower than the last received. That can fail if the reboot was started at a very early uptime and the reconnect came at a very late.
I've added an explicit reload call to the progress bar controller, that should fix this.
The instructions should probably tell the user to first try manually reconnecting to the web gui (and logging in) and checking the partition upgrade page and only unplug/replug the module if the browser can't reconnect?
Added.
Be aware that due to the build partition table now being updated, doing "make flash" on "master" is no longer sufficient to reset a module to the previous partitioning. To revert to the old partitioning including the latest changes, switch to the new test branch "partitioning-v3-30", and do the "make flash" from there.
Since "edge" now uses the new partition layout I updated my ovmsinstall.sh script (attached) to use the old v3-30 when the -o flag is given. By default you end up with the new v3-35 layout.
Thanks.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
I appear to have successfully run the partitioning upgrade. Following Mark’s correspondence, I rebooted the module just before performing the upgrade. Result from reboot after upgrade is below [and upgrade log is attached]:
 One suggestion, if the intent is for the upgrade to run, ideally, without user intervention, is to change the wording in the feedback window [see below] from “We need to copy” to “Copying”. When I first saw that feedback as the only line—I was wondering if I needed to intervene to execute the copying of the currently running partition to factory. A minor point.

Michael, and other contributors to this project, many thanks for all of your efforts on our behalf.
Chip Cangialose

On Apr 30, 2026, at 2:20 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Everyone,
please test the attached upgrade UI draft in combination with the latest firmware changes (just pushed).
Install the development version as a web plugin for now (will be included as a standard page after collecting feedback):
- Type: Page
- Page: /dev/partitioning
- Label: Partitioning (dev)
- Menu: Config
- Auth: Cookie
I've taken care to especially support the case of a failure in the store migration step. You can trigger that case by unplugging the module when in store migration.
There are now two options to recover from that situation without USB reflashing: retry the store migration or perform a factory reset.
The factory reset will automatically reboot into upgrade mode while running on the intermediate setup, and store2 will be erased to get a clean start. This should basically work in any case. The module will then boot into the upgraded system in setup mode, so the user can restore the backup or configure from scratch.
Retrying the store migration may be sufficient if the process was aborted due to some other issue (e.g. power loss), or if the user did a config restore.
Feedback is welcome.
Regards, Michael
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
OK, I’ve implemented by parts.
Firstly, I adopted Michael’s “-5” naming suffix on the version. So if you are running a production OVMS server, with firmware downloads enabled, then please make sure you enable the ‘-5’ variants (presumably with symlinks for the moment). My server has:
/firmware/ota/…
v3.0 v3.0-5 -> v3.0 v3.1 v3.1-5 -> v3.1 v3.2 v3.2-5 -> v3.2 v3.3 v3.3-5 -> v3.3
Similar on ovms.dexters-web.de now.
Later on (as 4MB firmware is obsoleted) we will have to stop deploying to the directories without the ‘-5’ suffix, but for the moment we can simply symlink them both together.
I am not happy with the implementation. Certainly not the most ugly code I’ve written, but it just seems messy at the moment. I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I've added a log option, so we can monitor via USB, and also don't lose the output parts after unmounting /store2 when monitoring via USB.
I've also added a test command, which shows how slow the continuous close/open calls make this. Shouldn't matter for the update process, but if we're going to use this for other things, that needs to be optimized.
The other hacky ugly bit is doing this startup work in housekeeping. Seemed ok at the time, and really can’t find anywhere better (tried peripherals, but that was worse). It could be moved to MyOTA, I guess.
Done so. I also added a reboot on failures, as the module would otherwise need a manual reboot, and changed the main call to the locking version.
I've done a migration test run with these changes, will think about the UI next.
Regards, Michael
It is complete, functional, and works on my test bench. But very open to anyone who can suggest / implement improvements.
OVMS# ota partitions list Partition table: Label Type Subtype Address Size
nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 passOVMS# vfs df /store: Size: 1936 kB Avail: 1800 kB Used: 8% /sd : Size: 3802 MB Avail: 3787 MB Used: 1%
Regards, Mark.
On 24 Apr 2026, at 6:33 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com> wrote:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
So let's look at each of these, and I make a proposal how to solve.
- Making it simple 1.1) Automate the steps
The current approach is overly complex for non-technical end-users. They simply either won't do it, or will mess it up.
My suggestion here is to:
a) Introduce a new 'ovms partitions upgrade auto -noconfirm' command. This command will initiate a fully automatic upgrade.
b) At the ovms_partitions.{h,cpp} level, introduce a 'ovms_partition_table_upgrade' command that will look at the current state, perform the required actions, and reboot as necessary.
c) Extend boot.{h,cpp} to support a new BR_PartitionUpgrade boot state (used by ovms_partition_table_upgrade during reboot).
d) Extend the initial boot code so that if the BR_PartitionUpgrade flag is set then re-run ovms_partition_table_upgrade and only continue with the system init if that indicates failure.
I think in that way we can reduce the boot time (by not initialising modem, etc) and also avoid the SD CARD mounting issue. It would be one command the user runs, and monitors the output on USB console (or not).
My primary issue here still is most users won't monitor the process via USB, so will not be able to see or copy details of the process log in case something goes wrong.
OTOH, these users will need to reflash the module anyway in that case, catching the details would only help us in post-mortem analysis and maybe improving the process.
I expect corruption of /store to be the primary source of severe issues, and that would now (hopefully) be detected by the store migration step. The auto upgrade process could then do a store downgrade and set a config variable to reflect the abort cause (should do so as well on other issues encountered, so we can inform the user).
1.2) Provide a user interface
Presumably this could simply be done by the admin web interface. If the ovms_partition_table_get_type() is not OVMS_FlashPartition_35 (or perhaps introduce a new ovms_partition_table_isuptodate() call), then prompt to update.
I'll take care of that, extending this by the feedback about a failure of the process.
1.3) Module informing servers of status (so we can see progress)
In OvmsOTA::AutoFlash() we build the URL as api.openvehicles.com/firmware/ota + GetOVMSProduct() + tag + ovms3.bin. That GetOVMSProduct() is in ovms_utils.{h,cpp} and returns v3.0, v3.1, or v3.3.
In ovms_version.{h,cpp} it uses GetOVMSVersion() + " (build " + GetOVMSBuild() + ")" to build the version and set as m.version.
It seems that we can quite simply:
a) Change GetOVMSProduct() to return v3.5 for a v3.3 hardware running the upgraded partition layout.
b) Change ovms_version to add "product " + GetOVMSProduct() before the build in the version string.
The server would then receive that in the version string, and we could use it for status and statistical deployment purposes. I don't think it affects the buildverscmp() comparison.
That means we'd abandon the notion of the product code reflecting the hardware version, not sure about that: v3.0 & 3.1 modules could also be upgraded to the new partitioning.
Maybe we could rather add a minor version level or suffix for the new partitioning scheme: "v3.3" = hardware 3.3, original partitioning "v3.3.5" or "v3.3-5" = hw 3.3, new partitioning That would work just as good for the server / build side differentiation, but keep hardware versions distinguishable.
- 4MB vs 7MB firmware 2.1) Server support builds
We would need to add a v3.5 directory to the servers, and presumably do new nightly 7MB builds as well as a cut-down 4MB builds. These presumably would be simple and only have different sdkconfig files (plus perhaps a sanity check to ensure that the 4MB file is <4MB before deployment).
2.2) Module downloads correct build
The GetOVMSProduct() change in 1.3(a) above should resolve this.
2.3) Firmware fitting in the partition
This seems to have already been resolved in Michael's a5843037a503a61ec116e4ab46e05883545f7a48 commit.
Yes.
- Notifying users 3.1) Developers
I suggest we prioritise developers to get this tested. Once the firmware is ready we can push on ovmsdev mailing list to deploy for wider testing.
I think we should release that on "edge", but first with an additional warning in the UI, that this isn't yet meant for end users ("proceed at own risk" + link to the factory reset guide).
When closing 3.3.006 we can remove that warning.
3.2) End users
I can think of two approaches:
a) A pop-up on web interface main screen if partition table upgrade is available.
b) Push notifications when they connect using the App.
Some users now only use V3 connections.
I suggest we generally send the push notification once after each OTA update, and additionally when an OTA update detects a firmware too large to be installable. That case wouldn't happen normally from an auto update applying the server directory scheme, but it will happen when users flash directly from a URL or a downloaded firmware, for example when told to try the current edge build.
- Versioning 4.1) Firmware versions
If we follow the approach in 1.3 above, no change to current versioning scheme is needed.
There was a suggestion that once this is ready for production deployment we make that the 3.3-006 version.
I still think we should do so, as we're already close to the 4MB limit.
- Factory 5.1) Factory production support
Once this is in production we would need to change the default partitions.csv to reflect the new arrangement, build new production firmware, and release to factory.
I am happy to do 1.1, 1.3, 2.1 (for api.openvehicles.com <http://api.openvehicles.com/>), 2.2, 3.1, 4.1, and 5.1.
I've done 2.3, will audit 1.1 when ready, will do 1.2 once the feedback variable has been defined, and 2.1 for ovms.dexters-web.de once we've agreed on the version scheme.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Bildschirmfoto vom 2026-04-30 21-47-17.png><Bildschirmfoto vom 2026-04-30 22-07-17.png><partitioning.htm>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Mark,
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?
I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Hmm.. Tried doing the upgrade (from my own build of latest master) a few times, each time I get the following:
The tail of the partition log looks like this:
Added new store2 partition to partition table Erasing store2 data (2031616 bytes)... Recalculated MD5 checksum Erasing old partition table (4096 bytes at 0x00008000)...
And there's a back-trace in the last-boot status:
support/a2l 0x4008dda6 0x4008e041 0x401e2428 0x40085126 0x4018eae7 0x4018f1e1 0x401945a7 0x4018b7b3 0x4018e08a 0x4018aeb2 0x4013eece 0x4013f309 0x4013f720 0x4013f7a1 Using elf file: /home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x4008dda6 is in invoke_abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:156). 151 #endif 152 while (1) { 153 if (esp_cpu_in_ocd_debug_mode()) { 154 __asm__ ("break 0,0"); 155 } 156 *((int *) 0) = 0; 157 } 158 } 159 160 void abort() 0x4008e041 is in abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:171). 166 * don't overwrite that. 167 */ 168 if (esp_reset_reason_get_hint() == ESP_RST_UNKNOWN) { 169 esp_reset_reason_set_hint(ESP_RST_PANIC); 170 } 171 invoke_abort(); 172 } 173 174 175 static const char *edesc[] = { 0x401e2428 is in is_safe_write_address (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:124). 119 120 static __attribute__((unused)) bool is_safe_write_address(size_t addr, size_t size) 121 { 122 bool result = true; 123 if (addr <= ESP_PARTITION_TABLE_OFFSET + ESP_PARTITION_TABLE_MAX_LEN) { 124 UNSAFE_WRITE_ADDRESS; 125 } 126 127 const esp_partition_t *p = esp_ota_get_running_partition(); 128 if (addr >= p->address && addr < p->address + p->size) { 0x40085126 is in spi_flash_erase_range (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:220). 215 return spi_flash_erase_range(sec * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE); 216 } 217 218 esp_err_t IRAM_ATTR spi_flash_erase_range(uint32_t start_addr, uint32_t size) 219 { 220 CHECK_WRITE_ADDRESS(start_addr, size); 221 if (start_addr % SPI_FLASH_SEC_SIZE != 0) { 222 return ESP_ERR_INVALID_ARG; 223 } 224 if (size % SPI_FLASH_SEC_SIZE != 0) { 0x4018eae7 is in ovms_partition_table_rewrite(ovms_esp_partition_t*, OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1239). 1234 if (writer) writer->puts("Error: Failed to find partition table"); 1235 return false; 1236 } 1237 1238 if (writer) writer->printf("Erasing old partition table (%d bytes at 0x%08zx)...\n", (int)PARTITION_TABLE_BLOCK_SIZE, offset); 1239 esp_err_t err = spi_flash_erase_range(offset, PARTITION_TABLE_BLOCK_SIZE); 1240 if (err != ESP_OK) 1241 { 1242 if (writer)writer->printf("Error: Failed to erase old partition table: %s\n", esp_err_to_name(err)); 1243 return false; 0x4018f1e1 is in ovms_partition_table_upgrade_store(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:407). 402 p->magic_id = PARTITION_MD5_MAGIC; 403 OVMS_MD5_Final(p->data.md5.digest, &md5_ctx); 404 writer->puts("Recalculated MD5 checksum"); 405 406 // Now re-write the partition table to flash 407 if (ovms_partition_table_rewrite(table, writer)) 408 { 409 writer->puts("Partition table upgraded successfully - reboot required"); 410 partition_table_type = OVMS_FlashPartition_34; 411 } 0x401945a7 is in ovms_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1166). 1161 } 1162 } 1163 else 1164 { 1165 writer->puts("v3-30 The currently running partition is factory, so we can upgrade the partition table, and then reboot"); 1166 if (ovms_partition_table_upgrade_store(writer)) 1167 { 1168 writer->puts("v3-30 Partition table store upgraded successfully, rebooting now..."); 1169 writer->puts("v3-30 !! Do not power off the module while the upgrade is running !!"); 1170 vTaskDelay(2000/portTICK_PERIOD_MS); 0x4018b7b3 is in ota_perform_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:893). 888 { 889 writer->puts("Error: Flash operation already in progress - cannot upgrade partition table"); 890 return false; 891 } 892 893 return ovms_partition_table_upgrade_autocont(writer); 894 } 895 896 bool ota_partition_table_upgrade_autocont_yesno(OvmsWriter* writer, void* ctx, char ch) 897 { 0x4018e08a is in OvmsOTA::SystemStart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:1229). 1224 ESP_LOGI(TAG, "Checking for partition update..."); 1225 if (!ovms_partition_table_isuptodate()) 1226 { 1227 ESP_LOGI(TAG, "Partition table is not up to date, continuing to upgrade the partition table"); 1228 FileWriter writer("/store/partition-update.log", true); 1229 success = ota_perform_partition_table_upgrade_autocont(&writer); 1230 } 1231 else 1232 { 1233 success = true; 0x4018aeb2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (OvmsOTA::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (OvmsOTA*, std::_Placeholder<1>, std::_Placeholder<2>)> >::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600). 595 template<typename... _Args, typename _Req 596 = _Require<typename _Traits::__lvalue, 597 _CheckArgs<_Pack<_Args...>>>> 598 result_type 599 operator()(_Class* __object, _Args&&... __args) const 600 { return (__object->*_M_pmf)(std::forward<_Args>(__args)...); } 601 602 // Handle smart pointers, references and pointers to derived 603 template<typename _Tp, typename... _Args, typename _Req 604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class*, _Tp>, 0x4013eece is in std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271). 2266 function<_Res(_ArgTypes...)>:: 2267 operator()(_ArgTypes... __args) const 2268 { 2269 if (_M_empty()) 2270 __throw_bad_function_call(); 2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 2272 } 2273 2274 #if __cpp_rtti 2275 template<typename _Res, typename... _ArgTypes> 0x4013f309 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:345). 340 { 341 m_current_started = monotonictime; 342 m_current_callback = *itc; 343 if (m_current_callback->m_callback) 344 { 345 m_current_callback->m_callback(m_current_event, msg->body.signal.data); 346 } 347 m_current_callback = NULL; 348 } 349 } 0x4013f720 is in OvmsEvents::EventTask() (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:278). 273 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 274 m_current_event.clear(); 275 break; 276 case EVENT_phasedsignal: 277 m_current_event = msg.body.phasedsignal.event; 278 HandleQueueSignalEvent(&msg); 279 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 280 m_current_event.clear(); 281 break; 282 default: 0x4013f7a1 is in EventLaunchTask(void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:93). 88 89 void EventLaunchTask(void *pvParameters) 90 { 91 OvmsEvents* me = (OvmsEvents*)pvParameters; 92 93 me->EventTask(); 94 } 95 96 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) 97 { michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$
On Fri, 8 May 2026 at 05:43, Michael Balzer via OvmsDev < ovmsdev@lists.openvehicles.com> wrote:
Mark,
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson:
I did implement a FileWriter, which seemed neat as it simply records to disk and supports the command Writer interface - and even hooked it in so it correctly and neatly swaps from /store/… to /store2/… during the store migration. But it just doesn’t seem very elegant at the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Michael,
you missed setting…
CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y
…in your build config. That's necessary to allow the application to write the partition table.
Regards, Michael
Am 17.05.26 um 08:19 schrieb Michael Geddes:
Hmm.. Tried doing the upgrade (from my own build of latest master) a few times, each time I get the following:
The tail of the partition log looks like this:
Added new store2 partition to partition table Erasing store2 data (2031616 bytes)... Recalculated MD5 checksum Erasing old partition table (4096 bytes at 0x00008000)... And there's a back-trace in the last-boot status:
support/a2l 0x4008dda6 0x4008e041 0x401e2428 0x40085126 0x4018eae7 0x4018f1e1 0x401945a7 0x4018b7b3 0x4018e08a 0x4018aeb2 0x4013eece 0x4013f309 0x4013f720 0x4013f7a1 Using elf file: /home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x4008dda6 is in invoke_abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:156). 151 #endif 152 while (1) { 153 if (esp_cpu_in_ocd_debug_mode()) { 154 __asm__ ("break 0,0"); 155 } 156 *((int *) 0) = 0; 157 } 158 } 159 160 void abort() 0x4008e041 is in abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:171). 166 * don't overwrite that. 167 */ 168 if (esp_reset_reason_get_hint() == ESP_RST_UNKNOWN) { 169 esp_reset_reason_set_hint(ESP_RST_PANIC); 170 } 171 invoke_abort(); 172 } 173 174 175 static const char *edesc[] = { 0x401e2428 is in is_safe_write_address (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:124). 119 120 static __attribute__((unused)) bool is_safe_write_address(size_t addr, size_t size) 121 { 122 bool result = true; 123 if (addr <= ESP_PARTITION_TABLE_OFFSET + ESP_PARTITION_TABLE_MAX_LEN) { 124 UNSAFE_WRITE_ADDRESS; 125 } 126 127 const esp_partition_t *p = esp_ota_get_running_partition(); 128 if (addr >= p->address && addr < p->address + p->size) { 0x40085126 is in spi_flash_erase_range (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:220). 215 return spi_flash_erase_range(sec * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE); 216 } 217 218 esp_err_t IRAM_ATTR spi_flash_erase_range(uint32_t start_addr, uint32_t size) 219 { 220 CHECK_WRITE_ADDRESS(start_addr, size); 221 if (start_addr % SPI_FLASH_SEC_SIZE != 0) { 222 return ESP_ERR_INVALID_ARG; 223 } 224 if (size % SPI_FLASH_SEC_SIZE != 0) { 0x4018eae7 is in ovms_partition_table_rewrite(ovms_esp_partition_t*, OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1239). 1234 if (writer) writer->puts("Error: Failed to find partition table"); 1235 return false; 1236 } 1237 1238 if (writer) writer->printf("Erasing old partition table (%d bytes at 0x%08zx)...\n", (int)PARTITION_TABLE_BLOCK_SIZE, offset); 1239 esp_err_t err = spi_flash_erase_range(offset, PARTITION_TABLE_BLOCK_SIZE); 1240 if (err != ESP_OK) 1241 { 1242 if (writer)writer->printf("Error: Failed to erase old partition table: %s\n", esp_err_to_name(err)); 1243 return false; 0x4018f1e1 is in ovms_partition_table_upgrade_store(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:407). 402 p->magic_id = PARTITION_MD5_MAGIC; 403 OVMS_MD5_Final(p->data.md5.digest, &md5_ctx); 404 writer->puts("Recalculated MD5 checksum"); 405 406 // Now re-write the partition table to flash 407 if (ovms_partition_table_rewrite(table, writer)) 408 { 409 writer->puts("Partition table upgraded successfully - reboot required"); 410 partition_table_type = OVMS_FlashPartition_34; 411 } 0x401945a7 is in ovms_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1166). 1161 } 1162 } 1163 else 1164 { 1165 writer->puts("v3-30 The currently running partition is factory, so we can upgrade the partition table, and then reboot"); 1166 if (ovms_partition_table_upgrade_store(writer)) 1167 { 1168 writer->puts("v3-30 Partition table store upgraded successfully, rebooting now..."); 1169 writer->puts("v3-30 !! Do not power off the module while the upgrade is running !!"); 1170 vTaskDelay(2000/portTICK_PERIOD_MS); 0x4018b7b3 is in ota_perform_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:893). 888 { 889 writer->puts("Error: Flash operation already in progress - cannot upgrade partition table"); 890 return false; 891 } 892 893 return ovms_partition_table_upgrade_autocont(writer); 894 } 895 896 bool ota_partition_table_upgrade_autocont_yesno(OvmsWriter* writer, void* ctx, char ch) 897 { 0x4018e08a is in OvmsOTA::SystemStart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:1229). 1224 ESP_LOGI(TAG, "Checking for partition update..."); 1225 if (!ovms_partition_table_isuptodate()) 1226 { 1227 ESP_LOGI(TAG, "Partition table is not up to date, continuing to upgrade the partition table"); 1228 FileWriter writer("/store/partition-update.log", true); 1229 success = ota_perform_partition_table_upgrade_autocont(&writer); 1230 } 1231 else 1232 { 1233 success = true; 0x4018aeb2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (OvmsOTA::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (OvmsOTA*, std::_Placeholder<1>, std::_Placeholder<2>)> >::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600). 595 template<typename... _Args, typename _Req 596 = _Require<typename _Traits::__lvalue, 597 _CheckArgs<_Pack<_Args...>>>> 598 result_type 599 operator()(_Class* __object, _Args&&... __args) const 600 { return (__object->*_M_pmf)(std::forward<_Args>(__args)...); } 601 602 // Handle smart pointers, references and pointers to derived 603 template<typename _Tp, typename... _Args, typename _Req 604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class*, _Tp>, 0x4013eece is in std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271). 2266 function<_Res(_ArgTypes...)>:: 2267 operator()(_ArgTypes... __args) const 2268 { 2269 if (_M_empty()) 2270 __throw_bad_function_call(); 2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 2272 } 2273 2274 #if __cpp_rtti 2275 template<typename _Res, typename... _ArgTypes> 0x4013f309 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:345). 340 { 341 m_current_started = monotonictime; 342 m_current_callback = *itc; 343 if (m_current_callback->m_callback) 344 { 345 m_current_callback->m_callback(m_current_event, msg->body.signal.data); 346 } 347 m_current_callback = NULL; 348 } 349 } 0x4013f720 is in OvmsEvents::EventTask() (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:278). 273 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 274 m_current_event.clear(); 275 break; 276 case EVENT_phasedsignal: 277 m_current_event = msg.body.phasedsignal.event; 278 HandleQueueSignalEvent(&msg); 279 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 280 m_current_event.clear(); 281 break; 282 default: 0x4013f7a1 is in EventLaunchTask(void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:93). 88 89 void EventLaunchTask(void *pvParameters) 90 { 91 OvmsEvents* me = (OvmsEvents*)pvParameters; 92 93 me->EventTask(); 94 } 95 96 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) 97 { michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$
On Fri, 8 May 2026 at 05:43, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Mark, Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev: > Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson: >> I did implement a FileWriter, which seemed neat as it simply records >> to disk and supports the command Writer interface - and even hooked >> it in so it correctly and neatly swaps from /store/… to /store2/… >> during the store migration. But it just doesn’t seem very elegant at >> the moment. > > I see what you mean, but don't have a better idea yet. Maybe something > like `GetPath()` & `SetPath()`? I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926 _______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
More precisely:
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
Am 17.05.26 um 08:35 schrieb Michael Balzer via OvmsDev:
Michael,
you missed setting…
CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y
…in your build config. That's necessary to allow the application to write the partition table.
Regards, Michael
Am 17.05.26 um 08:19 schrieb Michael Geddes:
Hmm.. Tried doing the upgrade (from my own build of latest master) a few times, each time I get the following:
The tail of the partition log looks like this:
Added new store2 partition to partition table Erasing store2 data (2031616 bytes)... Recalculated MD5 checksum Erasing old partition table (4096 bytes at 0x00008000)... And there's a back-trace in the last-boot status:
support/a2l 0x4008dda6 0x4008e041 0x401e2428 0x40085126 0x4018eae7 0x4018f1e1 0x401945a7 0x4018b7b3 0x4018e08a 0x4018aeb2 0x4013eece 0x4013f309 0x4013f720 0x4013f7a1 Using elf file: /home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x4008dda6 is in invoke_abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:156). 151 #endif 152 while (1) { 153 if (esp_cpu_in_ocd_debug_mode()) { 154 __asm__ ("break 0,0"); 155 } 156 *((int *) 0) = 0; 157 } 158 } 159 160 void abort() 0x4008e041 is in abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:171). 166 * don't overwrite that. 167 */ 168 if (esp_reset_reason_get_hint() == ESP_RST_UNKNOWN) { 169 esp_reset_reason_set_hint(ESP_RST_PANIC); 170 } 171 invoke_abort(); 172 } 173 174 175 static const char *edesc[] = { 0x401e2428 is in is_safe_write_address (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:124). 119 120 static __attribute__((unused)) bool is_safe_write_address(size_t addr, size_t size) 121 { 122 bool result = true; 123 if (addr <= ESP_PARTITION_TABLE_OFFSET + ESP_PARTITION_TABLE_MAX_LEN) { 124 UNSAFE_WRITE_ADDRESS; 125 } 126 127 const esp_partition_t *p = esp_ota_get_running_partition(); 128 if (addr >= p->address && addr < p->address + p->size) { 0x40085126 is in spi_flash_erase_range (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:220). 215 return spi_flash_erase_range(sec * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE); 216 } 217 218 esp_err_t IRAM_ATTR spi_flash_erase_range(uint32_t start_addr, uint32_t size) 219 { 220 CHECK_WRITE_ADDRESS(start_addr, size); 221 if (start_addr % SPI_FLASH_SEC_SIZE != 0) { 222 return ESP_ERR_INVALID_ARG; 223 } 224 if (size % SPI_FLASH_SEC_SIZE != 0) { 0x4018eae7 is in ovms_partition_table_rewrite(ovms_esp_partition_t*, OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1239). 1234 if (writer) writer->puts("Error: Failed to find partition table"); 1235 return false; 1236 } 1237 1238 if (writer) writer->printf("Erasing old partition table (%d bytes at 0x%08zx)...\n", (int)PARTITION_TABLE_BLOCK_SIZE, offset); 1239 esp_err_t err = spi_flash_erase_range(offset, PARTITION_TABLE_BLOCK_SIZE); 1240 if (err != ESP_OK) 1241 { 1242 if (writer)writer->printf("Error: Failed to erase old partition table: %s\n", esp_err_to_name(err)); 1243 return false; 0x4018f1e1 is in ovms_partition_table_upgrade_store(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:407). 402 p->magic_id = PARTITION_MD5_MAGIC; 403 OVMS_MD5_Final(p->data.md5.digest, &md5_ctx); 404 writer->puts("Recalculated MD5 checksum"); 405 406 // Now re-write the partition table to flash 407 if (ovms_partition_table_rewrite(table, writer)) 408 { 409 writer->puts("Partition table upgraded successfully - reboot required"); 410 partition_table_type = OVMS_FlashPartition_34; 411 } 0x401945a7 is in ovms_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1166). 1161 } 1162 } 1163 else 1164 { 1165 writer->puts("v3-30 The currently running partition is factory, so we can upgrade the partition table, and then reboot"); 1166 if (ovms_partition_table_upgrade_store(writer)) 1167 { 1168 writer->puts("v3-30 Partition table store upgraded successfully, rebooting now..."); 1169 writer->puts("v3-30 !! Do not power off the module while the upgrade is running !!"); 1170 vTaskDelay(2000/portTICK_PERIOD_MS); 0x4018b7b3 is in ota_perform_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:893). 888 { 889 writer->puts("Error: Flash operation already in progress
- cannot upgrade partition table"); 890 return false; 891 } 892 893 return ovms_partition_table_upgrade_autocont(writer); 894 } 895 896 bool ota_partition_table_upgrade_autocont_yesno(OvmsWriter* writer, void* ctx, char ch) 897 { 0x4018e08a is in OvmsOTA::SystemStart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:1229). 1224 ESP_LOGI(TAG, "Checking for partition update..."); 1225 if (!ovms_partition_table_isuptodate()) 1226 { 1227 ESP_LOGI(TAG, "Partition table is not up to date, continuing to upgrade the partition table"); 1228 FileWriter writer("/store/partition-update.log", true); 1229 success = ota_perform_partition_table_upgrade_autocont(&writer); 1230 } 1231 else 1232 { 1233 success = true; 0x4018aeb2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (OvmsOTA::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (OvmsOTA*, std::_Placeholder<1>, std::_Placeholder<2>)> >::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600). 595 template<typename... _Args, typename _Req 596 = _Require<typename _Traits::__lvalue, 597 _CheckArgs<_Pack<_Args...>>>> 598 result_type 599 operator()(_Class* __object, _Args&&... __args) const 600 { return (__object->*_M_pmf)(std::forward<_Args>(__args)...); } 601 602 // Handle smart pointers, references and pointers to derived 603 template<typename _Tp, typename... _Args, typename _Req 604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class*, _Tp>, 0x4013eece is in std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271). 2266 function<_Res(_ArgTypes...)>:: 2267 operator()(_ArgTypes... __args) const 2268 { 2269 if (_M_empty()) 2270 __throw_bad_function_call(); 2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 2272 } 2273 2274 #if __cpp_rtti 2275 template<typename _Res, typename... _ArgTypes> 0x4013f309 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:345). 340 { 341 m_current_started = monotonictime; 342 m_current_callback = *itc; 343 if (m_current_callback->m_callback) 344 { 345 m_current_callback->m_callback(m_current_event, msg->body.signal.data); 346 } 347 m_current_callback = NULL; 348 } 349 } 0x4013f720 is in OvmsEvents::EventTask() (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:278). 273 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 274 m_current_event.clear(); 275 break; 276 case EVENT_phasedsignal: 277 m_current_event = msg.body.phasedsignal.event; 278 HandleQueueSignalEvent(&msg); 279 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 280 m_current_event.clear(); 281 break; 282 default: 0x4013f7a1 is in EventLaunchTask(void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:93). 88 89 void EventLaunchTask(void *pvParameters) 90 { 91 OvmsEvents* me = (OvmsEvents*)pvParameters; 92 93 me->EventTask(); 94 } 95 96 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) 97 { michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$
On Fri, 8 May 2026 at 05:43, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Mark, Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev: > Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson: >> I did implement a FileWriter, which seemed neat as it simply records >> to disk and supports the command Writer interface - and even hooked >> it in so it correctly and neatly swaps from /store/… to /store2/… >> during the store migration. But it just doesn’t seem very elegant at >> the moment. > > I see what you mean, but don't have a better idea yet. Maybe something > like `GetPath()` & `SetPath()`? I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926 _______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Got a first user log covering a typical store corruption, attached. The copy process correctly identified the errors and signaled a failure.
Of course the user has no backup, so needs to reconfigure now.
Regards, Michael
Am 17.05.26 um 08:37 schrieb Michael Balzer via OvmsDev:
More precisely:
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
Am 17.05.26 um 08:35 schrieb Michael Balzer via OvmsDev:
Michael,
you missed setting…
CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y
…in your build config. That's necessary to allow the application to write the partition table.
Regards, Michael
Am 17.05.26 um 08:19 schrieb Michael Geddes:
Hmm.. Tried doing the upgrade (from my own build of latest master) a few times, each time I get the following:
The tail of the partition log looks like this:
Added new store2 partition to partition table Erasing store2 data (2031616 bytes)... Recalculated MD5 checksum Erasing old partition table (4096 bytes at 0x00008000)... And there's a back-trace in the last-boot status:
support/a2l 0x4008dda6 0x4008e041 0x401e2428 0x40085126 0x4018eae7 0x4018f1e1 0x401945a7 0x4018b7b3 0x4018e08a 0x4018aeb2 0x4013eece 0x4013f309 0x4013f720 0x4013f7a1 Using elf file: /home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x4008dda6 is in invoke_abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:156). 151 #endif 152 while (1) { 153 if (esp_cpu_in_ocd_debug_mode()) { 154 __asm__ ("break 0,0"); 155 } 156 *((int *) 0) = 0; 157 } 158 } 159 160 void abort() 0x4008e041 is in abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:171). 166 * don't overwrite that. 167 */ 168 if (esp_reset_reason_get_hint() == ESP_RST_UNKNOWN) { 169 esp_reset_reason_set_hint(ESP_RST_PANIC); 170 } 171 invoke_abort(); 172 } 173 174 175 static const char *edesc[] = { 0x401e2428 is in is_safe_write_address (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:124). 119 120 static __attribute__((unused)) bool is_safe_write_address(size_t addr, size_t size) 121 { 122 bool result = true; 123 if (addr <= ESP_PARTITION_TABLE_OFFSET + ESP_PARTITION_TABLE_MAX_LEN) { 124 UNSAFE_WRITE_ADDRESS; 125 } 126 127 const esp_partition_t *p = esp_ota_get_running_partition(); 128 if (addr >= p->address && addr < p->address + p->size) { 0x40085126 is in spi_flash_erase_range (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:220). 215 return spi_flash_erase_range(sec * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE); 216 } 217 218 esp_err_t IRAM_ATTR spi_flash_erase_range(uint32_t start_addr, uint32_t size) 219 { 220 CHECK_WRITE_ADDRESS(start_addr, size); 221 if (start_addr % SPI_FLASH_SEC_SIZE != 0) { 222 return ESP_ERR_INVALID_ARG; 223 } 224 if (size % SPI_FLASH_SEC_SIZE != 0) { 0x4018eae7 is in ovms_partition_table_rewrite(ovms_esp_partition_t*, OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1239). 1234 if (writer) writer->puts("Error: Failed to find partition table"); 1235 return false; 1236 } 1237 1238 if (writer) writer->printf("Erasing old partition table (%d bytes at 0x%08zx)...\n", (int)PARTITION_TABLE_BLOCK_SIZE, offset); 1239 esp_err_t err = spi_flash_erase_range(offset, PARTITION_TABLE_BLOCK_SIZE); 1240 if (err != ESP_OK) 1241 { 1242 if (writer)writer->printf("Error: Failed to erase old partition table: %s\n", esp_err_to_name(err)); 1243 return false; 0x4018f1e1 is in ovms_partition_table_upgrade_store(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:407). 402 p->magic_id = PARTITION_MD5_MAGIC; 403 OVMS_MD5_Final(p->data.md5.digest, &md5_ctx); 404 writer->puts("Recalculated MD5 checksum"); 405 406 // Now re-write the partition table to flash 407 if (ovms_partition_table_rewrite(table, writer)) 408 { 409 writer->puts("Partition table upgraded successfully - reboot required"); 410 partition_table_type = OVMS_FlashPartition_34; 411 } 0x401945a7 is in ovms_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1166). 1161 } 1162 } 1163 else 1164 { 1165 writer->puts("v3-30 The currently running partition is factory, so we can upgrade the partition table, and then reboot"); 1166 if (ovms_partition_table_upgrade_store(writer)) 1167 { 1168 writer->puts("v3-30 Partition table store upgraded successfully, rebooting now..."); 1169 writer->puts("v3-30 !! Do not power off the module while the upgrade is running !!"); 1170 vTaskDelay(2000/portTICK_PERIOD_MS); 0x4018b7b3 is in ota_perform_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:893). 888 { 889 writer->puts("Error: Flash operation already in progress
- cannot upgrade partition table"); 890 return false; 891 } 892 893 return ovms_partition_table_upgrade_autocont(writer); 894 } 895 896 bool ota_partition_table_upgrade_autocont_yesno(OvmsWriter* writer, void* ctx, char ch) 897 { 0x4018e08a is in OvmsOTA::SystemStart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:1229). 1224 ESP_LOGI(TAG, "Checking for partition update..."); 1225 if (!ovms_partition_table_isuptodate()) 1226 { 1227 ESP_LOGI(TAG, "Partition table is not up to date, continuing to upgrade the partition table"); 1228 FileWriter writer("/store/partition-update.log", true); 1229 success = ota_perform_partition_table_upgrade_autocont(&writer); 1230 } 1231 else 1232 { 1233 success = true; 0x4018aeb2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (OvmsOTA::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (OvmsOTA*, std::_Placeholder<1>, std::_Placeholder<2>)>
::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600). 595 template<typename... _Args, typename _Req 596 = _Require<typename _Traits::__lvalue, 597 _CheckArgs<_Pack<_Args...>>>> 598 result_type 599 operator()(_Class* __object, _Args&&... __args) const 600 { return (__object->*_M_pmf)(std::forward<_Args>(__args)...); } 601 602 // Handle smart pointers, references and pointers to derived 603 template<typename _Tp, typename... _Args, typename _Req 604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class*, _Tp>, 0x4013eece is in std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271). 2266 function<_Res(_ArgTypes...)>:: 2267 operator()(_ArgTypes... __args) const 2268 { 2269 if (_M_empty()) 2270 __throw_bad_function_call(); 2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 2272 } 2273 2274 #if __cpp_rtti 2275 template<typename _Res, typename... _ArgTypes> 0x4013f309 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:345). 340 { 341 m_current_started = monotonictime; 342 m_current_callback = *itc; 343 if (m_current_callback->m_callback) 344 { 345 m_current_callback->m_callback(m_current_event, msg->body.signal.data); 346 } 347 m_current_callback = NULL; 348 } 349 } 0x4013f720 is in OvmsEvents::EventTask() (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:278). 273 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 274 m_current_event.clear(); 275 break; 276 case EVENT_phasedsignal: 277 m_current_event = msg.body.phasedsignal.event; 278 HandleQueueSignalEvent(&msg); 279 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 280 m_current_event.clear(); 281 break; 282 default: 0x4013f7a1 is in EventLaunchTask(void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:93). 88 89 void EventLaunchTask(void *pvParameters) 90 { 91 OvmsEvents* me = (OvmsEvents*)pvParameters; 92 93 me->EventTask(); 94 } 95 96 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) 97 { michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$
On Fri, 8 May 2026 at 05:43, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Mark, Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev: > Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson: >> I did implement a FileWriter, which seemed neat as it simply records >> to disk and supports the command Writer interface - and even hooked >> it in so it correctly and neatly swaps from /store/… to /store2/… >> during the store migration. But it just doesn’t seem very elegant at >> the moment. > > I see what you mean, but don't have a better idea yet. Maybe something > like `GetPath()` & `SetPath()`? I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926 _______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Can the user rollback?
On 18 May 2026, at 6:42 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Got a first user log covering a typical store corruption, attached. The copy process correctly identified the errors and signaled a failure.
Of course the user has no backup, so needs to reconfigure now.
Regards, Michael
Am 17.05.26 um 08:37 schrieb Michael Balzer via OvmsDev:
More precisely:
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
Am 17.05.26 um 08:35 schrieb Michael Balzer via OvmsDev:
Michael,
you missed setting…
CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y
…in your build config. That's necessary to allow the application to write the partition table.
Regards, Michael
Am 17.05.26 um 08:19 schrieb Michael Geddes:
Hmm.. Tried doing the upgrade (from my own build of latest master) a few times, each time I get the following:
The tail of the partition log looks like this:
Added new store2 partition to partition table Erasing store2 data (2031616 bytes)... Recalculated MD5 checksum Erasing old partition table (4096 bytes at 0x00008000)... And there's a back-trace in the last-boot status:
support/a2l 0x4008dda6 0x4008e041 0x401e2428 0x40085126 0x4018eae7 0x4018f1e1 0x401945a7 0x4018b7b3 0x4018e08a 0x4018aeb2 0x4013eece 0x4013f309 0x4013f720 0x4013f7a1 Using elf file: /home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x4008dda6 is in invoke_abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:156). 151 #endif 152 while (1) { 153 if (esp_cpu_in_ocd_debug_mode()) { 154 __asm__ ("break 0,0"); 155 } 156 *((int *) 0) = 0; 157 } 158 } 159 160 void abort() 0x4008e041 is in abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:171). 166 * don't overwrite that. 167 */ 168 if (esp_reset_reason_get_hint() == ESP_RST_UNKNOWN) { 169 esp_reset_reason_set_hint(ESP_RST_PANIC); 170 } 171 invoke_abort(); 172 } 173 174 175 static const char *edesc[] = { 0x401e2428 is in is_safe_write_address (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:124). 119 120 static __attribute__((unused)) bool is_safe_write_address(size_t addr, size_t size) 121 { 122 bool result = true; 123 if (addr <= ESP_PARTITION_TABLE_OFFSET + ESP_PARTITION_TABLE_MAX_LEN) { 124 UNSAFE_WRITE_ADDRESS; 125 } 126 127 const esp_partition_t *p = esp_ota_get_running_partition(); 128 if (addr >= p->address && addr < p->address + p->size) { 0x40085126 is in spi_flash_erase_range (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:220). 215 return spi_flash_erase_range(sec * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE); 216 } 217 218 esp_err_t IRAM_ATTR spi_flash_erase_range(uint32_t start_addr, uint32_t size) 219 { 220 CHECK_WRITE_ADDRESS(start_addr, size); 221 if (start_addr % SPI_FLASH_SEC_SIZE != 0) { 222 return ESP_ERR_INVALID_ARG; 223 } 224 if (size % SPI_FLASH_SEC_SIZE != 0) { 0x4018eae7 is in ovms_partition_table_rewrite(ovms_esp_partition_t*, OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1239). 1234 if (writer) writer->puts("Error: Failed to find partition table"); 1235 return false; 1236 } 1237 1238 if (writer) writer->printf("Erasing old partition table (%d bytes at 0x%08zx)...\n", (int)PARTITION_TABLE_BLOCK_SIZE, offset); 1239 esp_err_t err = spi_flash_erase_range(offset, PARTITION_TABLE_BLOCK_SIZE); 1240 if (err != ESP_OK) 1241 { 1242 if (writer)writer->printf("Error: Failed to erase old partition table: %s\n", esp_err_to_name(err)); 1243 return false; 0x4018f1e1 is in ovms_partition_table_upgrade_store(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:407). 402 p->magic_id = PARTITION_MD5_MAGIC; 403 OVMS_MD5_Final(p->data.md5.digest, &md5_ctx); 404 writer->puts("Recalculated MD5 checksum"); 405 406 // Now re-write the partition table to flash 407 if (ovms_partition_table_rewrite(table, writer)) 408 { 409 writer->puts("Partition table upgraded successfully - reboot required"); 410 partition_table_type = OVMS_FlashPartition_34; 411 } 0x401945a7 is in ovms_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1166). 1161 } 1162 } 1163 else 1164 { 1165 writer->puts("v3-30 The currently running partition is factory, so we can upgrade the partition table, and then reboot"); 1166 if (ovms_partition_table_upgrade_store(writer)) 1167 { 1168 writer->puts("v3-30 Partition table store upgraded successfully, rebooting now..."); 1169 writer->puts("v3-30 !! Do not power off the module while the upgrade is running !!"); 1170 vTaskDelay(2000/portTICK_PERIOD_MS); 0x4018b7b3 is in ota_perform_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:893). 888 { 889 writer->puts("Error: Flash operation already in progress - cannot upgrade partition table"); 890 return false; 891 } 892 893 return ovms_partition_table_upgrade_autocont(writer); 894 } 895 896 bool ota_partition_table_upgrade_autocont_yesno(OvmsWriter* writer, void* ctx, char ch) 897 { 0x4018e08a is in OvmsOTA::SystemStart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:1229). 1224 ESP_LOGI(TAG, "Checking for partition update..."); 1225 if (!ovms_partition_table_isuptodate()) 1226 { 1227 ESP_LOGI(TAG, "Partition table is not up to date, continuing to upgrade the partition table"); 1228 FileWriter writer("/store/partition-update.log", true); 1229 success = ota_perform_partition_table_upgrade_autocont(&writer); 1230 } 1231 else 1232 { 1233 success = true; 0x4018aeb2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (OvmsOTA::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (OvmsOTA*, std::_Placeholder<1>, std::_Placeholder<2>)> >::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600). 595 template<typename... _Args, typename _Req 596 = _Require<typename _Traits::__lvalue, 597 _CheckArgs<_Pack<_Args...>>>> 598 result_type 599 operator()(_Class* __object, _Args&&... __args) const 600 { return (__object->*_M_pmf)(std::forward<_Args>(__args)...); } 601 602 // Handle smart pointers, references and pointers to derived 603 template<typename _Tp, typename... _Args, typename _Req 604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class*, _Tp>, 0x4013eece is in std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271). 2266 function<_Res(_ArgTypes...)>:: 2267 operator()(_ArgTypes... __args) const 2268 { 2269 if (_M_empty()) 2270 __throw_bad_function_call(); 2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 2272 } 2273 2274 #if __cpp_rtti 2275 template<typename _Res, typename... _ArgTypes> 0x4013f309 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:345). 340 { 341 m_current_started = monotonictime; 342 m_current_callback = *itc; 343 if (m_current_callback->m_callback) 344 { 345 m_current_callback->m_callback(m_current_event, msg->body.signal.data); 346 } 347 m_current_callback = NULL; 348 } 349 } 0x4013f720 is in OvmsEvents::EventTask() (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:278). 273 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 274 m_current_event.clear(); 275 break; 276 case EVENT_phasedsignal: 277 m_current_event = msg.body.phasedsignal.event; 278 HandleQueueSignalEvent(&msg); 279 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 280 m_current_event.clear(); 281 break; 282 default: 0x4013f7a1 is in EventLaunchTask(void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:93). 88 89 void EventLaunchTask(void *pvParameters) 90 { 91 OvmsEvents* me = (OvmsEvents*)pvParameters; 92 93 me->EventTask(); 94 } 95 96 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) 97 { michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$ <mailto:michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$> On Fri, 8 May 2026 at 05:43, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com>> wrote:
Mark,
Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev:
Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson: > I did implement a FileWriter, which seemed neat as it simply records > to disk and supports the command Writer interface - and even hooked > it in so it correctly and neatly swaps from /store/… to /store2/… > during the store migration. But it just doesn’t seem very elegant at > the moment.
I see what you mean, but don't have a better idea yet. Maybe something like
GetPath()&SetPath()?I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API.
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com <mailto:ovmsdev-leave@lists.openvehicles.com>
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Partition-upgrade01.txt>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
He already did the factory reset as offered by the UI.
I did not include the rollback option into the UI, to keep things simple. If the store migration fails, the module still is operational without rollback, and the user needs to fix the store corruption anyway to finish the upgrade.
I described the troubleshooting options in the user guide: https://docs.openvehicles.com/en/latest/userguide/partitioning.html#troubles...
Regards, Michael
Am 19.05.26 um 01:57 schrieb Mark Webb-Johnson:
Can the user rollback?
On 18 May 2026, at 6:42 PM, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Got a first user log covering a typical store corruption, attached. The copy process correctly identified the errors and signaled a failure.
Of course the user has no backup, so needs to reconfigure now.
Regards, Michael
Am 17.05.26 um 08:37 schrieb Michael Balzer via OvmsDev:
More precisely:
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
Am 17.05.26 um 08:35 schrieb Michael Balzer via OvmsDev:
Michael,
you missed setting…
CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y
…in your build config. That's necessary to allow the application to write the partition table.
Regards, Michael
Am 17.05.26 um 08:19 schrieb Michael Geddes:
Hmm.. Tried doing the upgrade (from my own build of latest master) a few times, each time I get the following:
The tail of the partition log looks like this:
Added new store2 partition to partition table Erasing store2 data (2031616 bytes)... Recalculated MD5 checksum Erasing old partition table (4096 bytes at 0x00008000)... And there's a back-trace in the last-boot status:
support/a2l 0x4008dda6 0x4008e041 0x401e2428 0x40085126 0x4018eae7 0x4018f1e1 0x401945a7 0x4018b7b3 0x4018e08a 0x4018aeb2 0x4013eece 0x4013f309 0x4013f720 0x4013f7a1 Using elf file: /home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x4008dda6 is in invoke_abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:156). 151 #endif 152 while (1) { 153 if (esp_cpu_in_ocd_debug_mode()) { 154 __asm__ ("break 0,0"); 155 } 156 *((int *) 0) = 0; 157 } 158 } 159 160 void abort() 0x4008e041 is in abort (/home/michaelg/esp/esp-idf/components/esp32/panic.c:171). 166 * don't overwrite that. 167 */ 168 if (esp_reset_reason_get_hint() == ESP_RST_UNKNOWN) { 169 esp_reset_reason_set_hint(ESP_RST_PANIC); 170 } 171 invoke_abort(); 172 } 173 174 175 static const char *edesc[] = { 0x401e2428 is in is_safe_write_address (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:124). 119 120 static __attribute__((unused)) bool is_safe_write_address(size_t addr, size_t size) 121 { 122 bool result = true; 123 if (addr <= ESP_PARTITION_TABLE_OFFSET + ESP_PARTITION_TABLE_MAX_LEN) { 124 UNSAFE_WRITE_ADDRESS; 125 } 126 127 const esp_partition_t *p = esp_ota_get_running_partition(); 128 if (addr >= p->address && addr < p->address + p->size) { 0x40085126 is in spi_flash_erase_range (/home/michaelg/esp/esp-idf/components/spi_flash/flash_ops.c:220). 215 return spi_flash_erase_range(sec * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE); 216 } 217 218 esp_err_t IRAM_ATTR spi_flash_erase_range(uint32_t start_addr, uint32_t size) 219 { 220 CHECK_WRITE_ADDRESS(start_addr, size); 221 if (start_addr % SPI_FLASH_SEC_SIZE != 0) { 222 return ESP_ERR_INVALID_ARG; 223 } 224 if (size % SPI_FLASH_SEC_SIZE != 0) { 0x4018eae7 is in ovms_partition_table_rewrite(ovms_esp_partition_t*, OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1239). 1234 if (writer) writer->puts("Error: Failed to find partition table"); 1235 return false; 1236 } 1237 1238 if (writer) writer->printf("Erasing old partition table (%d bytes at 0x%08zx)...\n", (int)PARTITION_TABLE_BLOCK_SIZE, offset); 1239 esp_err_t err = spi_flash_erase_range(offset, PARTITION_TABLE_BLOCK_SIZE); 1240 if (err != ESP_OK) 1241 { 1242 if (writer)writer->printf("Error: Failed to erase old partition table: %s\n", esp_err_to_name(err)); 1243 return false; 0x4018f1e1 is in ovms_partition_table_upgrade_store(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:407). 402 p->magic_id = PARTITION_MD5_MAGIC; 403 OVMS_MD5_Final(p->data.md5.digest, &md5_ctx); 404 writer->puts("Recalculated MD5 checksum"); 405 406 // Now re-write the partition table to flash 407 if (ovms_partition_table_rewrite(table, writer)) 408 { 409 writer->puts("Partition table upgraded successfully
- reboot required"); 410 partition_table_type = OVMS_FlashPartition_34; 411 } 0x401945a7 is in ovms_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_partitions.cpp:1166). 1161 } 1162 } 1163 else 1164 { 1165 writer->puts("v3-30 The currently running partition is factory, so we can upgrade the partition table, and then reboot"); 1166 if (ovms_partition_table_upgrade_store(writer)) 1167 { 1168 writer->puts("v3-30 Partition table store upgraded successfully, rebooting now..."); 1169 writer->puts("v3-30 !! Do not power off the module while the upgrade is running !!"); 1170 vTaskDelay(2000/portTICK_PERIOD_MS); 0x4018b7b3 is in ota_perform_partition_table_upgrade_autocont(OvmsWriter*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:893). 888 { 889 writer->puts("Error: Flash operation already in progress - cannot upgrade partition table"); 890 return false; 891 } 892 893 return ovms_partition_table_upgrade_autocont(writer); 894 } 895 896 bool ota_partition_table_upgrade_autocont_yesno(OvmsWriter* writer, void* ctx, char ch) 897 { 0x4018e08a is in OvmsOTA::SystemStart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_ota/src/ovms_ota.cpp:1229). 1224 ESP_LOGI(TAG, "Checking for partition update..."); 1225 if (!ovms_partition_table_isuptodate()) 1226 { 1227 ESP_LOGI(TAG, "Partition table is not up to date, continuing to upgrade the partition table"); 1228 FileWriter writer("/store/partition-update.log", true); 1229 success = ota_perform_partition_table_upgrade_autocont(&writer); 1230 } 1231 else 1232 { 1233 success = true; 0x4018aeb2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (OvmsOTA::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (OvmsOTA*, std::_Placeholder<1>, std::_Placeholder<2>)>
::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600). 595 template<typename... _Args, typename _Req 596 = _Require<typename _Traits::__lvalue, 597 _CheckArgs<_Pack<_Args...>>>> 598 result_type 599 operator()(_Class* __object, _Args&&... __args) const 600 { return (__object->*_M_pmf)(std::forward<_Args>(__args)...); } 601 602 // Handle smart pointers, references and pointers to derived 603 template<typename _Tp, typename... _Args, typename _Req 604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class*, _Tp>, 0x4013eece is in std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const (/home/michaelg/.espressif/tools/xtensa-esp32-elf/1.22.0-97-gc752ad5-5.2.0/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271). 2266 function<_Res(_ArgTypes...)>:: 2267 operator()(_ArgTypes... __args) const 2268 { 2269 if (_M_empty()) 2270 __throw_bad_function_call(); 2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 2272 } 2273 2274 #if __cpp_rtti 2275 template<typename _Res, typename... _ArgTypes> 0x4013f309 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:345). 340 { 341 m_current_started = monotonictime; 342 m_current_callback = *itc; 343 if (m_current_callback->m_callback) 344 { 345 m_current_callback->m_callback(m_current_event, msg->body.signal.data); 346 } 347 m_current_callback = NULL; 348 } 349 } 0x4013f720 is in OvmsEvents::EventTask() (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:278). 273 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 274 m_current_event.clear(); 275 break; 276 case EVENT_phasedsignal: 277 m_current_event = msg.body.phasedsignal.event; 278 HandleQueueSignalEvent(&msg); 279 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task 280 m_current_event.clear(); 281 break; 282 default: 0x4013f7a1 is in EventLaunchTask(void*) (/home/michaelg/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:93). 88 89 void EventLaunchTask(void *pvParameters) 90 { 91 OvmsEvents* me = (OvmsEvents*)pvParameters; 92 93 me->EventTask(); 94 } 95 96 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) 97 { michaelg@Boggart:~/esp/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3$
On Fri, 8 May 2026 at 05:43, Michael Balzer via OvmsDev <ovmsdev@lists.openvehicles.com> wrote:
Mark, Am 28.04.26 um 19:22 schrieb Michael Balzer via OvmsDev: > Am 27.04.26 um 07:38 schrieb Mark Webb-Johnson: >> I did implement a FileWriter, which seemed neat as it simply records >> to disk and supports the command Writer interface - and even hooked >> it in so it correctly and neatly swaps from /store/… to /store2/… >> during the store migration. But it just doesn’t seem very elegant at >> the moment. > > I see what you mean, but don't have a better idea yet. Maybe something > like `GetPath()` & `SetPath()`? I had to rework the log copying/handover to make sure we can still get details of the migrate step when it encounters corruption on /store, took that opportunity to also rework the FileWriter API. Regards, Michael -- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926 _______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
OvmsDev mailing list --ovmsdev@lists.openvehicles.com To unsubscribe send an email toovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
<Partition-upgrade01.txt>_______________________________________________ OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
The partition upgrade web UI & user notifications are now included in build 3.3.005-857-g650f26406.
Notifications will not be sent/shown until version 3.3.006 by default.
To test these with a current 3.3.005 build, do:
config set ota parttype.notifyversion 3.3.005
Only a single info notification will be sent, this is bound to the system start and user connections, to catch cases of modules running without server connections by default, and modules not being used actively.
To resend the single info notification on next connect/reboot:
config rm ota parttype.notified
The web UI's home page generally displays a warning if the version condition is fulfilled, and if an OTA update fails due to an outdated partitioning scheme, an alert will be sent on each try.
In terms of the plan, we should be ready for release now:
Am 22.04.26 um 08:22 schrieb Mark Webb-Johnson:
Making it simple 1.1) Automate the steps 1.2) Provide a user interface 1.3) Module informing servers of status (so we can see progress)
4MB vs 7MB firmware 2.1) Server support builds 2.2) Module downloads correct build 2.3) Firmware fitting in the partition
Notifying users 3.1) Developers 3.2) End users
Versioning 4.1) Firmware versions
Factory 5.1) Factory production support
Anything missing?
Regards, Michael
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Worked for me as well, thank you for the great work!
But should we somehow notify users that action is necessary? I specifically looked for the item in the menu, but non-devs wouldn't know to do that?
sharkcow
Am 04.05.26 um 23:16 schrieb Craig Leres via OvmsDev:
On 5/4/26 06:35, Michael Balzer via OvmsDev wrote:
The partition upgrade web UI & user notifications are now included in build 3.3.005-857-g650f26406.
Worked sweet including the browser reconnect. Nice job!
Craig
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
Thanks for the test & feedback.
As for the user notification, these will activate on release 3.3.006. Please see my post on how to test them now -- as you already did the upgrade, you'll need to flash back to an old partitioning to see them.
Regards, Michael
Am 05.05.26 um 08:45 schrieb sharkcow via OvmsDev:
Worked for me as well, thank you for the great work!
But should we somehow notify users that action is necessary? I specifically looked for the item in the menu, but non-devs wouldn't know to do that?
sharkcow
Am 04.05.26 um 23:16 schrieb Craig Leres via OvmsDev:
On 5/4/26 06:35, Michael Balzer via OvmsDev wrote:
The partition upgrade web UI & user notifications are now included in build 3.3.005-857-g650f26406.
Worked sweet including the browser reconnect. Nice job!
Craig
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
This is the user notification, sent via all channels:
The module is running an outdated flash partitioning scheme. OTA updates will soon stop working due to its 4MB size limit. Upgrading is recommended to leverage the limit to 7MB. To upgrade; open the web UI; menu Config -> Partitioning.
It changes to type alert & "have now stopped working" when trying to install a firmware image that doesn't fit.
The notification on the web UI's home page is basically the same, but with a button "Open Partitioning Tool" instead of the last line.
Regards, Michael
Am 05.05.26 um 09:12 schrieb Michael Balzer via OvmsDev:
Thanks for the test & feedback.
As for the user notification, these will activate on release 3.3.006. Please see my post on how to test them now -- as you already did the upgrade, you'll need to flash back to an old partitioning to see them.
Regards, Michael
Am 05.05.26 um 08:45 schrieb sharkcow via OvmsDev:
Worked for me as well, thank you for the great work!
But should we somehow notify users that action is necessary? I specifically looked for the item in the menu, but non-devs wouldn't know to do that?
sharkcow
Am 04.05.26 um 23:16 schrieb Craig Leres via OvmsDev:
On 5/4/26 06:35, Michael Balzer via OvmsDev wrote:
The partition upgrade web UI & user notifications are now included in build 3.3.005-857-g650f26406.
Worked sweet including the browser reconnect. Nice job!
Craig
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
As the smart car community mostly uses the latest edge builds, some folks there have taken the leap already.
No real issues so far. Some user feedback:
…so: I puffed up my cheeks, held my breath, and just started the new partitioning with the latest Edge 857. It took a few minutes, rebooted smoothly, login as usual, message "0 errors and 0 problems."
The factory partition is now gone, ota0 and 1 are still there, but apparently larger now…
Looks great.
Thanks to the developers for this feat, for digging so deep into the module and cleaning things up so there's more space in the future 🙏🙏☺️
Partition upgrade performed without errors or warnings, followed immediately by the update to Edge 869. However, I had to reset the module after the partition upgrade by performing a power reset. Everything is running very stably. Many thanks to the developers! 👍
It worked for me too: first the update to version 869, then the partition upgrade without any problems or error messages. 👍👍👍👍
Regarding remaining space, we've got 39456 bytes left now.
On my list of things to potentially include in 3.3.006 are now only PR #1365 & #1381 adding support for the AWS IoT MQTT hub & the Amazon root cert, the latter also meant to solve the issues with ABRP. That's 4736 bytes combined.
Everyone please check if anything else needs to or should be included in 3.3.006.
Thanks & regards, Michael
Am 05.05.26 um 13:38 schrieb Michael Balzer via OvmsDev:
This is the user notification, sent via all channels:
The module is running an outdated flash partitioning scheme. OTA updates will soon stop working due to its 4MB size limit. Upgrading is recommended to leverage the limit to 7MB. To upgrade; open the web UI; menu Config -> Partitioning.
It changes to type alert & "have now stopped working" when trying to install a firmware image that doesn't fit.
The notification on the web UI's home page is basically the same, but with a button "Open Partitioning Tool" instead of the last line.
Regards, Michael
Am 05.05.26 um 09:12 schrieb Michael Balzer via OvmsDev:
Thanks for the test & feedback.
As for the user notification, these will activate on release 3.3.006. Please see my post on how to test them now -- as you already did the upgrade, you'll need to flash back to an old partitioning to see them.
Regards, Michael
Am 05.05.26 um 08:45 schrieb sharkcow via OvmsDev:
Worked for me as well, thank you for the great work!
But should we somehow notify users that action is necessary? I specifically looked for the item in the menu, but non-devs wouldn't know to do that?
sharkcow
Am 04.05.26 um 23:16 schrieb Craig Leres via OvmsDev:
On 5/4/26 06:35, Michael Balzer via OvmsDev wrote:
The partition upgrade web UI & user notifications are now included in build 3.3.005-857-g650f26406.
Worked sweet including the browser reconnect. Nice job!
Craig
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
Craig,
Am 22.04.26 um 01:52 schrieb Craig Leres via OvmsDev:
On 4/21/26 15:04, Mark Webb-Johnson wrote:
I recall when I tried it I found the unmount function wasn’t atomic, so not trivial to do.
I guess we could issue the shell command (which is atomic), but perhaps that would cause problems (with recursion into the shell)?
I think what is needed is a way to poll sdcard::m_unmounting or even a new sdcard entry point that starts the unmount and waits for it to complete (or fail).
That's what the shell command does.
A problem I see is that if there are files open on the sdcard (e.g. a log file or can log), the unmount attempt will fail. So you probably want equivalent to "log close" before trying to unmount the sd.
That's the purpose of the synchronous "sd.unmounting" event, which framework components like the file logger listen to.
If you find a component that doesn't properly react to this event, that's a bug that needs to be fixed.
In your use case of writing huge CAN logs, I'd first check if you just
need to pass a longer timeout to "sd unmount", the default is 5 seconds.
The generic ESP_FAIL code is returned by unmount() if the
"sd.unmounting" process has been started but hasn't finished yet.
Regards, Michael
Craig
OVMS# sd unmount Error: SD unmount failed: ESP_FAIL OVMS# log status Log listeners : 3 File logging status: inactive Log file path : /sd/logs/ovms-dev.log Current size : 550.5 kB Cycle size : 1024 kB Cycle count : 0 Dropped messages : 0 Messages logged : 3227 Total fsync time : 6.0 s
OvmsDev mailing list -- ovmsdev@lists.openvehicles.com To unsubscribe send an email to ovmsdev-leave@lists.openvehicles.com
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On a second thought, maybe it's better to just guide the user, but keep all steps and their log output visible for the user in the web UI. Making the steps explicit user actions keeps the user in control (as far as possible here).
We can also ask the user to copy each step's log output, so in case something goes wrong we can get that info.
If something goes totally wrong in an automated process, the module may no longer boot. All diagnostic info would be lost in that case, as the user can then only reinstall the system via USB.
Opinions?
Regards, Michael
Am 17.04.26 um 15:05 schrieb Michael Balzer via OvmsDev:
And yes, we need to automate the upgrade, a normal user should only need to trigger the process, not perform the steps needed manually. We should even think about initiating the upgrade automatically once we gain some trust in it.
As part of the automation, I think we should disable the auto init on the intermediary reboots, so the system runs on a minimal setup when performing the critical steps.
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 4/18/26 07:14, Michael Balzer via OvmsDev wrote:
On a second thought, maybe it's better to just guide the user, but keep all steps and their log output visible for the user in the web UI. Making the steps explicit user actions keeps the user in control (as far as possible here).
We can also ask the user to copy each step's log output, so in case something goes wrong we can get that info.
If something goes totally wrong in an automated process, the module may no longer boot. All diagnostic info would be lost in that case, as the user can then only reinstall the system via USB.
I have some ideas.
Does "ota flash" detect when the incoming image is too large to fit in the designated partition? (I'm guessing yes.)
I think a one-step process is fine. If it was done via the web gui it could use a <progress> bar with a "continue" button after each step. And when the user clicks on the button that would trigger a download of a file (config or partial log).
I would insert this to the start of the upgrade process:
continue: download backup
On 4/17/26 01:27, Mark Webb-Johnson wrote:
ota partitions store upgrade
continue: download log1
> # module reset (aka ‘reboot’)
sd unmount
ota partitions store migrate
continue: download log2
> # ota partitions upgrade factory
continue: download log3
> # module reset (aka ‘reboot’)
I believe currently a sd card is required to create a backup. How hard would it be to add a "download backup" button that didn't require a sd card? (But I guess the follow-on problem is how to restore a config without a sd card... So maybe just warn the user to create a backup on sd in case they run into trouble.)
I figured I would want to reset my dev module back to the 3 partition setup (for future testing) so I wrote an installation script that will also be useful for recovering from a failed upgrade. I didn't see an obvious place for it in the Open-Vehicle-Monitoring-System-3 tree so I'm attaching a copy instead of a PR. Right now it only knows how to install the old 3 partition layout. I'm guessing it could go straight to the 2 partition layout with the correct partitions.bin? And /store is auto-magically populated when the code detects it has been zero'ed?
My script has a flag to erase the flash which I tested; I had assumed I clobbered the module pw but it appears to still be there. Is there somewhere aside from flash where it lives? (Just curious)
Appended is the log from when I converted my dev module back to the 3 partition layout.
Craig
ice 1898 % ovmsinstall.sh -v /dev/tty.SLAB_USBtoUART
- wget -q --show-progress --progress=bar:force:noscroll -O /tmp/ovmsinstall.sh.bootloader.PAQhaV https://ovms.dexters-web.de/firmware/ota/v3.3/edge/bootloader.bin /tmp/ovmsinstall.sh 100%[===================>] 22.42K 141KB/s in 0.2s
- wget -q --show-progress --progress=bar:force:noscroll -O /tmp/ovmsinstall.sh.firmware.VXmLEG https://ovms.dexters-web.de/firmware/ota/v3.3/edge/ovms3.bin /tmp/ovmsinstall.sh 100%[===================>] 3.95M 410KB/s in 10s
- wget -q --show-progress --progress=bar:force:noscroll -O /tmp/ovmsinstall.sh.partitions.haefnt https://ovms.dexters-web.de/firmware/ota/v3.3/edge/partitions.bin /tmp/ovmsinstall.sh 100%[===================>] 3.00K --.-KB/s in 0s Warning: Are you sure you want to reinstall the module? y Please enter yes or no Warning: Are you sure you want to reinstall the module? yes
- esptool.py --chip esp32 --port /dev/tty.SLAB_USBtoUART --baud 921600 --before default_reset --after hard_reset write_flash --compress --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 /tmp/ovmsinstall.sh.bootloader.PAQhaV 0x10000 /tmp/ovmsinstall.sh.firmware.VXmLEG 0x8000 /tmp/ovmsinstall.sh.partitions.haefnt esptool.py v4.10.0 Serial port /dev/tty.SLAB_USBtoUART Connecting..... Chip is ESP32-D0WD-V3 (revision v3.0) Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None Crystal is 40MHz MAC: 94:b9:7e:41:88:00 Uploading stub... Running stub... Stub running... Changing baud rate to 921600 Changed. Configuring flash size... Auto-detected Flash size: 16MB Flash will be erased from 0x00001000 to 0x00006fff... Flash will be erased from 0x00010000 to 0x00402fff... Flash will be erased from 0x00008000 to 0x00008fff... SHA digest in image updated Compressed 22960 bytes to 14682... Wrote 22960 bytes (14682 compressed) at 0x00001000 in 0.5 seconds (effective 367.4 kbit/s)... Hash of data verified. Compressed 4138432 bytes to 2411643... Wrote 4138432 bytes (2411643 compressed) at 0x00010000 in 43.4 seconds (effective 763.7 kbit/s)... Hash of data verified. Compressed 3072 bytes to 152... Wrote 3072 bytes (152 compressed) at 0x00008000 in 0.0 seconds (effective 575.5 kbit/s)... Hash of data verified.
Leaving... Hard resetting via RTS pin...
Success!
On 4/17/26 01:27, Mark Webb-Johnson wrote:
The sequence that currently works is as follows:
- ota partitions store upgrade
- module reset (aka ‘reboot’)
- sd unmount
- ota partitions store migrate
- ota partitions upgrade factory
- module reset (aka ‘reboot’)
Worked fine on my dev module. First I rolled back to the 3 partition setup and made sure I was running gb7d9ad0d from the factory partition. Then I ejected the SD and followed your steps.
Craig
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-826-gb7d9ad0d/ota_0/main (build idf v3.3.4-854-g9063c8662 Apr 17 2026 13:59:08) Partition type: v3-35 (ota1, ota2, no factory, maximized store) Partition table: 0x8000 Running partition: ota_0 Boot partition: ota_0 OTA_O image: 3.3.005-826-gb7d9ad0d OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB ota_0 app ota_0 0x00010000 7 MB ota_1 app ota_1 0x00710000 7 MB store data fat 0x00e10000 1984 KB Digest: 520877a00f2b1f7506844246b950ab23 pass
I tried 3.3.005-714-gbcf42700. Unfortunately the upgrade step results in a crash.
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-714-gbcf42700-dirty/factory/main (build idf v3.3.4-854-g9063c8662 Feb 25 2026 10:36:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-714-gbcf42700-dirty OTA_O image: 3.3.005-714-gbcf42700-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass OVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)...
sea 4 % ./backtrace.sh 0x400891b7:0x3ffebbd0 0x40089451:0x3ffebbf0 0x401c82cc:0x3ffebc10 0x40084cfa:0x3ffebc30 0x4018397d:0x3ffebc60 0x40180c03:0x3ffebcf0 0x40180c3f:0x3ffebd20 0x40146bdd:0x3ffebd40 0x40289911:0x3ffebd60 0x400e1091:0x3ffebd80 0x40137f55:0x3ffebe40 0x400e077d:0x3ffebe70 0x400e0a10:0x3ffebea0 0x400e0a43:0x3ffebf50 0x401566d1:0x3ffebf70 0x40157ed7:0x3ffebfb0 0x401582e1:0x3ffebfe0 0x40158593:0x3ffec050 0x40154aa2:0x3ffec0b0 0x4015a2fe:0x3ffec0d0 0x401442ec:0x3ffec100 0x40144415:0x3ffec160
- xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x400891b7:0x3ffebbd0 0x40089451:0x3ffebbf0 0x401c82cc:0x3ffebc10 0x40084cfa:0x3ffebc30 0x4018397d:0x3ffebc60 0x40180c03:0x3ffebcf0 0x40180c3f:0x3ffebd20 0x40146bdd:0x3ffebd40 0x40289911:0x3ffebd60 0x400e1091:0x3ffebd80 0x40137f55:0x3ffebe40 0x400e077d:0x3ffebe70 0x400e0a10:0x3ffebea0 0x400e0a43:0x3ffebf50 0x401566d1:0x3ffebf70 0x40157ed7:0x3ffebfb0 0x401582e1:0x3ffebfe0 0x40158593:0x3ffec050 0x40154aa2:0x3ffec0b0 0x4015a2fe:0x3ffec0d0 0x401442ec:0x3ffec100 0x40144415:0x3ffec160 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/spi_flash/flash_ops.c:124 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/spi_flash/flash_ops.c:220 (discriminator 2) components/ovms_ota/src/ovms_partitions.cpp:394 components/ovms_ota/src/ovms_ota.cpp:1088 components/ovms_ota/src/ovms_ota.cpp:1088 main/ovms_shell.cpp:67 main/ovms_shell.cpp:80 (discriminator 2) components/console_ssh/src/console_ssh.cpp:527 main/ovms_console.cpp:184 components/console_ssh/src/console_ssh.cpp:420 components/console_ssh/src/console_ssh.cpp:420 components/console_ssh/src/console_ssh.cpp:420 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/src/mongoose_client.cpp:52 main/ovms_netmanager.cpp:1000 main/ovms_netmanager.cpp:980
Craig,
your backtrace is in is_safe_write_address() -- is it possible you forgot to change your sdkconfig?
One big ‘gotcha’ I found was that we need to set CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ABORT= and CONFIG_SPI_FLASH_WRITING_DANGEROUS_REGIONS_ALLOWED=y in our sdkconfig - otherwise the partition table cannot be re-flashed.
Regards, Michael
Am 25.02.26 um 20:09 schrieb Craig Leres:
I tried 3.3.005-714-gbcf42700. Unfortunately the upgrade step results in a crash.
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-714-gbcf42700-dirty/factory/main (build idf v3.3.4-854-g9063c8662 Feb 25 2026 10:36:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: factory Boot partition: factory Factory image: 3.3.005-714-gbcf42700-dirty OTA_O image: 3.3.005-714-gbcf42700-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota partitions list Partition table: Label Type Subtype Address Size nvs data nvs 0x00009000 16 KB otadata data ota 0x0000d000 8 KB phy_init data phy 0x0000f000 4 KB factory app factory 0x00010000 4 MB ota_0 app ota_0 0x00410000 4 MB ota_1 app ota_1 0x00810000 4 MB store data fat 0x00c10000 1 MB Digest: cfe36765a6bfe1b802a2abd4ec9f6851 pass OVMS# ota partitions upgrade Upgrade partition table to new format (y/n): y 0x00009000 Skipping over data/nvs partition 0x0000d000 Skipping over data/ota partition 0x0000f000 Skipping over data/phy partition 0x00010000 Converted factory partition to 6MB OTA 0 0x00610000 Converted OTA 0 partition to 6MB OTA 1 0x00c10000 Moved data/fat partition up one position Recalculated MD5 checksum Clearing trailing old MD5 checksum record Erasing old partition table (4096 bytes at 0x00008000)...
sea 4 % ./backtrace.sh 0x400891b7:0x3ffebbd0 0x40089451:0x3ffebbf0 0x401c82cc:0x3ffebc10 0x40084cfa:0x3ffebc30 0x4018397d:0x3ffebc60 0x40180c03:0x3ffebcf0 0x40180c3f:0x3ffebd20 0x40146bdd:0x3ffebd40 0x40289911:0x3ffebd60 0x400e1091:0x3ffebd80 0x40137f55:0x3ffebe40 0x400e077d:0x3ffebe70 0x400e0a10:0x3ffebea0 0x400e0a43:0x3ffebf50 0x401566d1:0x3ffebf70 0x40157ed7:0x3ffebfb0 0x401582e1:0x3ffebfe0 0x40158593:0x3ffec050 0x40154aa2:0x3ffec0b0 0x4015a2fe:0x3ffec0d0 0x401442ec:0x3ffec100 0x40144415:0x3ffec160
- xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x400891b7:0x3ffebbd0 0x40089451:0x3ffebbf0 0x401c82cc:0x3ffebc10 0x40084cfa:0x3ffebc30 0x4018397d:0x3ffebc60 0x40180c03:0x3ffebcf0 0x40180c3f:0x3ffebd20 0x40146bdd:0x3ffebd40 0x40289911:0x3ffebd60 0x400e1091:0x3ffebd80 0x40137f55:0x3ffebe40 0x400e077d:0x3ffebe70 0x400e0a10:0x3ffebea0 0x400e0a43:0x3ffebf50 0x401566d1:0x3ffebf70 0x40157ed7:0x3ffebfb0 0x401582e1:0x3ffebfe0 0x40158593:0x3ffec050 0x40154aa2:0x3ffec0b0 0x4015a2fe:0x3ffec0d0 0x401442ec:0x3ffec100 0x40144415:0x3ffec160 /home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/spi_flash/flash_ops.c:124
/home/sea/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/spi_flash/flash_ops.c:220 (discriminator 2) components/ovms_ota/src/ovms_partitions.cpp:394 components/ovms_ota/src/ovms_ota.cpp:1088 components/ovms_ota/src/ovms_ota.cpp:1088 main/ovms_shell.cpp:67 main/ovms_shell.cpp:80 (discriminator 2) components/console_ssh/src/console_ssh.cpp:527 main/ovms_console.cpp:184 components/console_ssh/src/console_ssh.cpp:420 components/console_ssh/src/console_ssh.cpp:420 components/console_ssh/src/console_ssh.cpp:420 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/src/mongoose_client.cpp:52 main/ovms_netmanager.cpp:1000 main/ovms_netmanager.cpp:980
-- Michael Balzer * Am Rahmen 5 * D-58313 Herdecke Fon 02330 9104094 * Handy 0176 20698926
On 2/24/26 04:40, Mark Webb-Johnson via OvmsDev wrote:
OK, I have just committed this. The two new commands are:
- ota partitions list
- ota partitions upgrade [-noconfirm]
Rollback is via:
- wget http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin <http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin>
- esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin
General approach to manual upgrade:
- Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later)
- Copy running ota firmware to factory
- Set to boot from flash and reboot
- Use âota partitions upgradeâ to upgrade
- Reboot
Items still to be addressed:
- Documentation (including rollback procedure).
- Modify partitions.csv to use this new format (when ready for production).
- Web interface (in particular concept of factory vs ota), if necessary (havenât checked this).
- OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The âota flashâ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months).
- Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot).
For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
I took a run at this with my dev module:
OVMS# ota
Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
Firmware: 3.3.005-711-g4feca695-dirty/ota_1/main (build
idf v3.3.4-854-g9063c8662 Feb 24 2026 15:08:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: ota_1 Boot partition: ota_1 Factory image: 3.3.005-20-g60be058e OTA_O image: 3.3.005-700-g0a804e40-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota copy ota_1 factory OTA copy ota_1 (08454144) -> factory (00010000) size 4194304 Error: ESP32 error #258 starting OTA operation OVMS#
I couldn't find 258 so I internet searched it and learned that it is ESP_ERR_NOT_SUPPORTED (from components/esp32/include/esp_err.h, might be worth displaying in hex given that's how it's defined).
I checked and my esp-idf is up to date.
Craig
sea 12 % pwd /home/sea/u0/leres/esp/esp-idf sea 13 % git log | head commit 9063c8662ca5d67b5490c1503bd4377b380feed3 Author: Michael Balzer <balzer@expeedo.de> Date: Tue Jan 27 17:34:34 2026 +0100
Fix mutex priority inheritance bug in spi_flash component:
backport of esp-idf commit aea2fe08162e1ba86ad0189178c1f1cf29238bbd,
see https://github.com/espressif/esp-idf/issues/5116
and https://github.com/espressif/esp-idf/issues/7580
commit 75bdad1e05dbbfe2ef79e8398749ff6aaa7deb92 sea 17 % pwd /home/sea/u0/leres/src/OVMS.V3 sea 18 % git log | head commit 4feca695d922f26b5f3864fe43c6855b8f223675 Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Tue Feb 24 20:21:30 2026 +0800
Support for ota 0+1 partition table format, and 6MB firmware
commit 1241912b48628e89947e1600f86365b4b174c6c8 Merge: 92f93554 141bad50 Author: Michael Balzer <dexter@dexters-web.de> Date: Sun Feb 22 15:57:13 2026 +0100
Craig,
Thanks for discovering this. It is nasty.
Error 258 is 0x102 and that is ESP_ERR_INVALID_ARG (not ESP_ERR_NOT_SUPPORTED).
I find it esp_ota_begin (components/app_update/esp_ota_ops.c):
if (!is_ota_partition(partition)) {
return ESP_ERR_INVALID_ARG;
}
and that is:
static bool is_ota_partition(const esp_partition_t *p) { return (p != NULL && p->type == ESP_PARTITION_TYPE_APP && p->subtype >= ESP_PARTITION_SUBTYPE_APP_OTA_0 && p->subtype < ESP_PARTITION_SUBTYPE_APP_OTA_MAX); }
It seems we can’t target the factory partition with those ota operations. I guess the problem is the whole ota rollback thing (where if it can’t boot into an ota partition, it falls back to the previous one).
It is not too onerous to workaround, though. We can simply change to use esp_partition_erase_range and esp_partition_write when targeting the factory. I will try to implement it that way tonight.
Regards, Mark.
On 25 Feb 2026, at 7:34 AM, Craig Leres <leres@xse.com> wrote:
On 2/24/26 04:40, Mark Webb-Johnson via OvmsDev wrote:
OK, I have just committed this. The two new commands are:
- ota partitions list
- ota partitions upgrade [-noconfirm] Rollback is via:
- wget http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin <http://api.openvehicles.com/firmware/ota/v3.1/main/ partitions.bin>
- esptool.py -p <path-to-usb> write_flash 0x8000 partitions.bin General approach to manual upgrade:
- Upgrade to firmware supporting this feature (3.3.005-711-g4feca695 or later)
- Copy running ota firmware to factory
- Set to boot from flash and reboot
- Use âota partitions upgradeâ to upgrade
- Reboot Items still to be addressed:
- Documentation (including rollback procedure).
- Modify partitions.csv to use this new format (when ready for production).
- Web interface (in particular concept of factory vs ota), if necessary (havenât checked this).
- OTA flash builds. We will need a way to support 6MB builds for those that can use them. I suggest to change GetOVMSProduct() to return v3.5 for these modules that are running this new 6MB capable partition table. Then on server we can build a production release final 4MB firmware including this support, and then use v3.5 tree to build future 6MB only builds. The âota flashâ system would automatically support that and give people time to upgrade (as well as new users with 4MB partition modules for many months).
- Investigate any simple way to make this a simple one-command (current ota->factory, boot factory, reboot, partitions upgrade, reboot). For the moment, please try this out and let me know if you find any problems. We can then decide on the items still to be addressed.
I took a run at this with my dev module:
OVMS# ota Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 Firmware: 3.3.005-711-g4feca695-dirty/ota_1/main (build idf v3.3.4-854-g9063c8662 Feb 24 2026 15:08:43) Partition type: v3-f12 (factory, ota1, ota2) Partition table: 0x8000 Running partition: ota_1 Boot partition: ota_1 Factory image: 3.3.005-20-g60be058e OTA_O image: 3.3.005-700-g0a804e40-dirty OTA_1 image: 3.3.005-711-g4feca695-dirty OVMS# ota copy ota_1 factory OTA copy ota_1 (08454144) -> factory (00010000) size 4194304 Error: ESP32 error #258 starting OTA operation OVMS#
I couldn't find 258 so I internet searched it and learned that it is ESP_ERR_NOT_SUPPORTED (from components/esp32/include/esp_err.h, might be worth displaying in hex given that's how it's defined).
I checked and my esp-idf is up to date.
Craigsea 12 % pwd /home/sea/u0/leres/esp/esp-idf sea 13 % git log | head commit 9063c8662ca5d67b5490c1503bd4377b380feed3 Author: Michael Balzer <balzer@expeedo.de> Date: Tue Jan 27 17:34:34 2026 +0100
Fix mutex priority inheritance bug in spi_flash component: backport of esp-idf commit aea2fe08162e1ba86ad0189178c1f1cf29238bbd, see https://github.com/espressif/esp-idf/issues/5116 and https://github.com/espressif/esp-idf/issues/7580
commit 75bdad1e05dbbfe2ef79e8398749ff6aaa7deb92 sea 17 % pwd /home/sea/u0/leres/src/OVMS.V3 sea 18 % git log | head commit 4feca695d922f26b5f3864fe43c6855b8f223675 Author: Mark Webb-Johnson <mark@webb-johnson.net> Date: Tue Feb 24 20:21:30 2026 +0800
Support for ota 0+1 partition table format, and 6MB firmware
commit 1241912b48628e89947e1600f86365b4b174c6c8 Merge: 92f93554 141bad50 Author: Michael Balzer <dexter@dexters-web.de> Date: Sun Feb 22 15:57:13 2026 +0100
participants (7)
-
Carsten Schmiemann -
Charles B Cangialose -
Craig Leres -
Mark Webb-Johnson -
Michael Balzer -
Michael Geddes -
sharkcow