[Ovmsdev] Firmware size approaching 4 MB limit
Mark Webb-Johnson
mark at webb-johnson.net
Thu Feb 26 08:47:11 HKT 2026
Michael,
I think you are correct. The safest way of doing this is to split as you suggest:
For #1, I think it is done now and should be ok.
For #2, I will work on that. I can provide both an upgrade and a downgrade option. My only concern is renaming the existing store to store-old (or something like that) while it is mounted. But that will be necessary to ensure that the new store is accessible directly after reboot.
Then #3 will need just a small tweak on the partition sizes to complete it.
Trying to do it all in one command may be too complex.
I will firstly update the typedef enum ovms_flashpartition_t so that you/we can know what state we are in. Suggest to just change it to unknown, v30 (existing arrangement), v34 (store resized and moved), v35 (flash partitions resized and factory removed).
I think I can provide a rollback if store partition is not as expected following reboot. I suggest:
ota partition upgrade store
ota partition downgrade store
ota partition upgrade factory
I’m not sure how long #2 will take me. Perhaps over the weekend, depending on the success of the recursive filesystem copy.
In the meantime, I suggest people don’t play with this too much as I’m only going to handle v30<->v34->v35 formats. Anything else will have to be manually done with esptools.
Regards, Mark.
> On 26 Feb 2026, at 2:07 AM, Michael Balzer via OvmsDev <ovmsdev at lists.openvehicles.com> wrote:
>
> Signed PGP part
> Thinking about the web UI:
>
> I intend to add a fourth tab "Partitioning" to Config→Firmware that checks the current partitioning scheme and guides through the upgrade process, so this can be done with just a couple of clicks (+ reboots).
>
> The UI will provide info & links on each step, what to do if something fails.
>
> So the process will now become this?
> Copy running ota image to "factory", reboot into "factory"
> Upgrade store partition, reboot
> Upgrade OTA partitions, reboot
> If step 2 fails, can we provide a fallback, i.e. re-enabling the old store partition, so the user can retry after another reboot?
>
> Regards,
> Michael
>
>
> Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
>> Mark,
>>
>> I don't think the recursive copy operation is an issue. We already do a similar, but much more memory hungry operation with the config backup & restore operations (zip/unzip).
>>
>> The config backup doesn't add all directories, but includes "usr" meant for anything not fitting our standard /store layout:
>>
>> backup_dir[] =
>> {
>> { "ovms_config", false },
>> { "events", true },
>> { "scripts", true },
>> { "obd2ecu", true },
>> { "dbc", true },
>> { "plugin", true },
>> { "tls", true },
>> { "trustedca", true },
>> { "usr", true },
>> { NULL, false }
>> };
>>
>> We could say that's also what we copy over to the new store, anything else is left to the user, if you think that's necessary.
>>
>> But I doubt anyone has been using multi level custom directory structures on /store, due to the limited space and the frequent corruption issues we had with /store that would have turned out impractical very soon. Could be an option now, as I didn't hear about any corruption since introducing the config transaction scheme, but that's still new and only on edge, but I'd still recommend using /sd for that kind of storage.
>>
>> Regards,
>> Michael
>>
>>
>> Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
>>> Craig, Michael,
>>>
>>> I have fixed the (a) partition type in ‘ota’ after upgrade, and (b) copy to factory. Both commits pushed.
>>>
>>> I also experimented with store.
>>>
>>> I can modify the partition table, rename old store to xstore, and create a new store at 0xe10000 same size 1MB, copy over the store partition. Reboot and all comes up fine (with repositioned store accessible).
>>>
>>> If I do the same, but set new store size to 0x1f0000, then the store can’t be mounted (corrupt, presumably). I guess the wear-leveling thing, or maybe internal fat filesystem metadata is messing things up.
>>>
>>> So it seems that either we leave store at 1MB (move it to the end of flash and make the code partitions as big as we can), or we resize it to the 0x1f0000 and handle the complexities of first modifying the partition table for the new store, then mounting two fat file systems, recursively copying over the files, and then adjusting the partition tables again. The recursive copy worries me (memory consumption and if deeply nested then all those directory handles open at the same time - one for each level).
>>>
>>> Thoughts?
>>>
>>> Regards, Mark.
>>>
>>>> On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter at expeedo.de> <mailto:dexter at expeedo.de> wrote:
>>>>
>>>> @Craig: nice find!
>>>>
>>>> Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
>>>>> Michael,
>>>>>
>>>>>> AFAICT "ota partitions upgrade" currently does not verify the system has booted from "factory" before allowing the operation -- I think that should be added as a safety measure.
>>>>>
>>>>> I thought it did that check. Maybe not working?
>>>>
>>>> No, you're right, should be OK that way. But as Craig's result shows, I still should test it ;-)
>>>>
>>>>>
>>>>> Here are the three partition table options:
>>>>>
>>>>> Original (f12):
>>>>>
>>>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec)
>>>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>>>> partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096
>>>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>>>> factory app factory 0x10000 0x400000 410000 65536 4194304
>>>>> ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304
>>>>> ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304
>>>>> store data fat 0xC10000 0x100000 D10000 12648448 1048576
>>>>> *unused*
>>>>>
>>>>> 0xD10000 0x2F0000 1000000 13697024 3080192
>>>>>
>>>>> Current (12):
>>>>>
>>>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec)
>>>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>>>> partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096
>>>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>>>> ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456
>>>>> ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456
>>>>> store data fat 0xC10000 0x100000 D10000 12648448 1048576
>>>>> *unused*
>>>>>
>>>>> 0xD10000 0x2F0000 1000000 13697024 3080192
>>>>>
>>>>> Suggested (12, 7MB code, more store):
>>>>>
>>>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex) Offset (Dec) Size (Dec)
>>>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>>>> partition_table partition_table partition_table 0x8000 0x1000 9000 32768 4096
>>>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>>>> ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032
>>>>> ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032
>>>>> store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
>>>>>
>>>>> It would make sense to resize store *now* if we are going to do it, but I am wary of requiring a factory reset. Also need to double check that the unused space at the end is truly unused. I am assuming the boot loader is in the first 32KB, but haven’t actually checked.
>>>>>
>>>>> Looking at the third (resize store) arrangement, it would be moving 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to overlap. I wonder what simply copy that partition over (without reformatting flash) would do? Either corrupt the filesystem, ignore the extra space, or magically have the new space available to FAT? I suspect corruption. So, there is another (more complex) alternative:
>>>>> Create a 2nd fat store partition (0xE10000-0xFFFFFF)
>>>>> Format and mount that as FAT
>>>>> Copy files from old to new
>>>>> Unmount and drop the old store
>>>>> Finish the rest of the partition upgrade
>>>>>
>>>>> I suspect #3 would be the most complex step.
>>>>>
>>>>> Or the store move and resize could be done first as a separate step.
>>>>>
>>>>> Thoughts? Much more complex and risky...
>>>>
>>>> There's the fatresize tool, didn't check the code, but given we can simply create the new partition cleanly and copy the files there, we should go that way.
>>>>
>>>> I second the store resize should be the first upgrade step:
>>>>
>>>> Unmounting /store while running may be risky, that was never needed and up to now only occurrs during shutdown as a final step.
>>>>
>>>> I suggest we just relable the old "store" partition and do a reboot instead. esp_vfs_fat_spiflash_mount() searches for the partition by name, so the system should automatically mount & use the new "store" on boot.
>>>>
>>>> Regards,
>>>> Michael
>>>>
>>>> --
>>>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
>>>> Fon 02330 9104094 * Handy 0176 20698926
>>>>
>>>
>>
>> --
>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
>> Fon 02330 9104094 * Handy 0176 20698926
>>
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
> --
> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
> Fon 02330 9104094 * Handy 0176 20698926
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260226/9df679a2/attachment-0001.htm>
More information about the OvmsDev
mailing list