[Ovmsdev] Firmware size approaching 4 MB limit
Michael Balzer
dexter at expeedo.de
Thu Feb 26 02:07:05 HKT 2026
Thinking about the web UI:
I intend to add a fourth tab "Partitioning" to Config→Firmware that
checks the current partitioning scheme and guides through the upgrade
process, so this can be done with just a couple of clicks (+ reboots).
The UI will provide info & links on each step, what to do if something
fails.
So the process will now become this?
1. Copy running ota image to "factory", reboot into "factory"
2. Upgrade store partition, reboot
3. Upgrade OTA partitions, reboot
If step 2 fails, can we provide a fallback, i.e. re-enabling the old
store partition, so the user can retry after another reboot?
Regards,
Michael
Am 25.02.26 um 17:18 schrieb Michael Balzer via OvmsDev:
> Mark,
>
> I don't think the recursive copy operation is an issue. We already do
> a similar, but much more memory hungry operation with the config
> backup & restore operations (zip/unzip).
>
> The config backup doesn't add all directories, but includes "usr"
> meant for anything not fitting our standard /store layout:
>
> backup_dir[] =
> {
> { "ovms_config", false },
> { "events", true },
> { "scripts", true },
> { "obd2ecu", true },
> { "dbc", true },
> { "plugin", true },
> { "tls", true },
> { "trustedca", true },
> { "usr", true },
> { NULL, false }
> };
>
> We could say that's also what we copy over to the new store, anything
> else is left to the user, if you think that's necessary.
>
> But I doubt anyone has been using multi level custom directory
> structures on /store, due to the limited space and the frequent
> corruption issues we had with /store that would have turned out
> impractical very soon. Could be an option now, as I didn't hear about
> any corruption since introducing the config transaction scheme, but
> that's still new and only on edge, but I'd still recommend using /sd
> for that kind of storage.
>
> Regards,
> Michael
>
>
> Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
>> Craig, Michael,
>>
>> I have fixed the (a) partition type in ‘ota’ after upgrade, and (b)
>> copy to factory. Both commits pushed.
>>
>> I also experimented with store.
>>
>> * I can modify the partition table, rename old store to xstore, and
>> create a new store at 0xe10000 same size 1MB, copy over the store
>> partition. Reboot and all comes up fine (with repositioned store
>> accessible).
>>
>>
>> * If I do the same, but set new store size to 0x1f0000, then the
>> store can’t be mounted (corrupt, presumably). I guess the
>> wear-leveling thing, or maybe internal fat filesystem metadata is
>> messing things up.
>>
>>
>> So it seems that either we leave store at 1MB (move it to the end of
>> flash and make the code partitions as big as we can), or we resize it
>> to the 0x1f0000 and handle the complexities of first modifying the
>> partition table for the new store, then mounting two fat file
>> systems, recursively copying over the files, and then adjusting the
>> partition tables again. The recursive copy worries me (memory
>> consumption and if deeply nested then all those directory handles
>> open at the same time - one for each level).
>>
>> Thoughts?
>>
>> Regards, Mark.
>>
>>> On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter at expeedo.de> wrote:
>>>
>>> @Craig: nice find!
>>>
>>> Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
>>>> Michael,
>>>>
>>>>> AFAICT "ota partitions upgrade" currently does not verify the
>>>>> system has booted from "factory" before allowing the operation --
>>>>> I think that should be added as a safety measure.
>>>>
>>>> I thought it did that check. Maybe not working?
>>>
>>> No, you're right, should be OK that way. But as Craig's result
>>> shows, I still should test it ;-)
>>>
>>>>
>>>> Here are the three partition table options:
>>>>
>>>> Original (f12):
>>>>
>>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex)
>>>> Offset (Dec) Size (Dec)
>>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>>> partition_table partition_table partition_table 0x8000
>>>> 0x1000 9000 32768 4096
>>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>>> factory app factory 0x10000 0x400000 410000 65536 4194304
>>>> ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304
>>>> ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304
>>>> store data fat 0xC10000 0x100000 D10000 12648448 1048576
>>>> *unused*
>>>>
>>>> 0xD10000 0x2F0000 1000000 13697024 3080192
>>>>
>>>>
>>>> Current (12):
>>>>
>>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex)
>>>> Offset (Dec) Size (Dec)
>>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>>> partition_table partition_table partition_table 0x8000
>>>> 0x1000 9000 32768 4096
>>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>>> ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456
>>>> ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456
>>>> store data fat 0xC10000 0x100000 D10000 12648448 1048576
>>>> *unused*
>>>>
>>>> 0xD10000 0x2F0000 1000000 13697024 3080192
>>>>
>>>>
>>>> Suggested (12, 7MB code, more store):
>>>>
>>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex)
>>>> Offset (Dec) Size (Dec)
>>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>>> partition_table partition_table partition_table 0x8000
>>>> 0x1000 9000 32768 4096
>>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>>> ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032
>>>> ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032
>>>> store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
>>>>
>>>>
>>>> It would make sense to resize store *now* if we are going to do it,
>>>> but I am wary of requiring a factory reset. Also need to double
>>>> check that the unused space at the end is truly unused. I am
>>>> assuming the boot loader is in the first 32KB, but haven’t actually
>>>> checked.
>>>>
>>>> Looking at the third (resize store) arrangement, it would be moving
>>>> 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to
>>>> overlap. I wonder what simply copy that partition over (without
>>>> reformatting flash) would do? Either corrupt the filesystem, ignore
>>>> the extra space, or magically have the new space available to FAT?
>>>> I suspect corruption. So, there is another (more complex) alternative:
>>>>
>>>> 1. Create a 2nd fat store partition (0xE10000-0xFFFFFF)
>>>> 2. Format and mount that as FAT
>>>> 3. Copy files from old to new
>>>> 4. Unmount and drop the old store
>>>> 5. Finish the rest of the partition upgrade
>>>>
>>>>
>>>> I suspect #3 would be the most complex step.
>>>>
>>>> Or the store move and resize could be done first as a separate step.
>>>>
>>>> Thoughts? Much more complex and risky...
>>>
>>> There's the fatresize tool, didn't check the code, but given we can
>>> simply create the new partition cleanly and copy the files there, we
>>> should go that way.
>>>
>>> I second the store resize should be the first upgrade step:
>>>
>>> Unmounting /store while running may be risky, that was never needed
>>> and up to now only occurrs during shutdown as a final step.
>>>
>>> I suggest we just relable the old "store" partition and do a reboot
>>> instead. esp_vfs_fat_spiflash_mount() searches for the partition by
>>> name, so the system should automatically mount & use the new "store"
>>> on boot.
>>>
>>> Regards,
>>> Michael
>>>
>>> --
>>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
>>> Fon 02330 9104094 * Handy 0176 20698926
>>>
>>
>
> --
> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
> Fon 02330 9104094 * Handy 0176 20698926
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
--
Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
Fon 02330 9104094 * Handy 0176 20698926
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260225/1de211c7/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260225/1de211c7/attachment-0001.sig>
More information about the OvmsDev
mailing list