[Ovmsdev] Firmware size approaching 4 MB limit
Michael Balzer
dexter at expeedo.de
Thu Feb 26 00:18:36 HKT 2026
Mark,
I don't think the recursive copy operation is an issue. We already do a
similar, but much more memory hungry operation with the config backup &
restore operations (zip/unzip).
The config backup doesn't add all directories, but includes "usr" meant
for anything not fitting our standard /store layout:
backup_dir[] =
{
{ "ovms_config", false },
{ "events", true },
{ "scripts", true },
{ "obd2ecu", true },
{ "dbc", true },
{ "plugin", true },
{ "tls", true },
{ "trustedca", true },
{ "usr", true },
{ NULL, false }
};
We could say that's also what we copy over to the new store, anything
else is left to the user, if you think that's necessary.
But I doubt anyone has been using multi level custom directory
structures on /store, due to the limited space and the frequent
corruption issues we had with /store that would have turned out
impractical very soon. Could be an option now, as I didn't hear about
any corruption since introducing the config transaction scheme, but
that's still new and only on edge, but I'd still recommend using /sd for
that kind of storage.
Regards,
Michael
Am 25.02.26 um 14:13 schrieb Mark Webb-Johnson:
> Craig, Michael,
>
> I have fixed the (a) partition type in ‘ota’ after upgrade, and (b)
> copy to factory. Both commits pushed.
>
> I also experimented with store.
>
> * I can modify the partition table, rename old store to xstore, and
> create a new store at 0xe10000 same size 1MB, copy over the store
> partition. Reboot and all comes up fine (with repositioned store
> accessible).
>
>
> * If I do the same, but set new store size to 0x1f0000, then the
> store can’t be mounted (corrupt, presumably). I guess the
> wear-leveling thing, or maybe internal fat filesystem metadata is
> messing things up.
>
>
> So it seems that either we leave store at 1MB (move it to the end of
> flash and make the code partitions as big as we can), or we resize it
> to the 0x1f0000 and handle the complexities of first modifying the
> partition table for the new store, then mounting two fat file systems,
> recursively copying over the files, and then adjusting the partition
> tables again. The recursive copy worries me (memory consumption and if
> deeply nested then all those directory handles open at the same time -
> one for each level).
>
> Thoughts?
>
> Regards, Mark.
>
>> On Feb 25, 2026, at 3:59 PM, Michael Balzer <dexter at expeedo.de> wrote:
>>
>> @Craig: nice find!
>>
>> Am 25.02.26 um 02:58 schrieb Mark Webb-Johnson:
>>> Michael,
>>>
>>>> AFAICT "ota partitions upgrade" currently does not verify the
>>>> system has booted from "factory" before allowing the operation -- I
>>>> think that should be added as a safety measure.
>>>
>>> I thought it did that check. Maybe not working?
>>
>> No, you're right, should be OK that way. But as Craig's result shows,
>> I still should test it ;-)
>>
>>>
>>> Here are the three partition table options:
>>>
>>> Original (f12):
>>>
>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex)
>>> Offset (Dec) Size (Dec)
>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>> partition_table partition_table partition_table 0x8000
>>> 0x1000 9000 32768 4096
>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>> factory app factory 0x10000 0x400000 410000 65536 4194304
>>> ota_0 app ota_0 0x410000 0x400000 810000 4259840 4194304
>>> ota_1 app ota_1 0x810000 0x400000 C10000 8454144 4194304
>>> store data fat 0xC10000 0x100000 D10000 12648448 1048576
>>> *unused*
>>>
>>> 0xD10000 0x2F0000 1000000 13697024 3080192
>>>
>>>
>>> Current (12):
>>>
>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex)
>>> Offset (Dec) Size (Dec)
>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>> partition_table partition_table partition_table 0x8000
>>> 0x1000 9000 32768 4096
>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>> ota_0 app ota_0 0x10000 0x600000 610000 65536 6291456
>>> ota_1 app ota_1 0x610000 0x600000 C10000 6356992 6291456
>>> store data fat 0xC10000 0x100000 D10000 12648448 1048576
>>> *unused*
>>>
>>> 0xD10000 0x2F0000 1000000 13697024 3080192
>>>
>>>
>>> Suggested (12, 7MB code, more store):
>>>
>>> Label Type Subtype Offset (Hex) Size (Hex) Next (Hex)
>>> Offset (Dec) Size (Dec)
>>> bootloader bootloader bootloader 0x0000 0x8000 8000 0 32768
>>> partition_table partition_table partition_table 0x8000
>>> 0x1000 9000 32768 4096
>>> nvs data nvs 0x9000 0x4000 D000 36864 16384
>>> otadata data ota 0xD000 0x2000 F000 53248 8192
>>> phy_init data phy 0xF000 0x1000 10000 61440 4096
>>> ota_0 app ota_0 0x10000 0x700000 710000 65536 7340032
>>> ota_1 app ota_1 0x710000 0x700000 E10000 7405568 7340032
>>> store data fat 0xE10000 0x1F0000 1000000 14745600 2031616
>>>
>>>
>>> It would make sense to resize store *now* if we are going to do it,
>>> but I am wary of requiring a factory reset. Also need to double
>>> check that the unused space at the end is truly unused. I am
>>> assuming the boot loader is in the first 32KB, but haven’t actually
>>> checked.
>>>
>>> Looking at the third (resize store) arrangement, it would be moving
>>> 0xC10000-0xD0FFFF to 0xE10000-0xFFFFFF, which doesn’t seem to
>>> overlap. I wonder what simply copy that partition over (without
>>> reformatting flash) would do? Either corrupt the filesystem, ignore
>>> the extra space, or magically have the new space available to FAT? I
>>> suspect corruption. So, there is another (more complex) alternative:
>>>
>>> 1. Create a 2nd fat store partition (0xE10000-0xFFFFFF)
>>> 2. Format and mount that as FAT
>>> 3. Copy files from old to new
>>> 4. Unmount and drop the old store
>>> 5. Finish the rest of the partition upgrade
>>>
>>>
>>> I suspect #3 would be the most complex step.
>>>
>>> Or the store move and resize could be done first as a separate step.
>>>
>>> Thoughts? Much more complex and risky...
>>
>> There's the fatresize tool, didn't check the code, but given we can
>> simply create the new partition cleanly and copy the files there, we
>> should go that way.
>>
>> I second the store resize should be the first upgrade step:
>>
>> Unmounting /store while running may be risky, that was never needed
>> and up to now only occurrs during shutdown as a final step.
>>
>> I suggest we just relable the old "store" partition and do a reboot
>> instead. esp_vfs_fat_spiflash_mount() searches for the partition by
>> name, so the system should automatically mount & use the new "store"
>> on boot.
>>
>> Regards,
>> Michael
>>
>> --
>> Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
>> Fon 02330 9104094 * Handy 0176 20698926
>>
>
--
Michael Balzer * Am Rahmen 5 * D-58313 Herdecke
Fon 02330 9104094 * Handy 0176 20698926
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260225/3f9c5b7b/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20260225/3f9c5b7b/attachment-0001.sig>
More information about the OvmsDev
mailing list