ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage. ESP IDF 3.x now has SPIFFS as an option. SPIFFS is obviously more suited to embedded flash. What do people think about making the switch to SPIFFS for /store now? Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup). Doing it later is not really an option. Thoughts? Mark
This is not a problem for me. I have very few files in /store. The scp implementation does support -r (recursive), so the copy out and back would be straightforward. -- Steve On Fri, 19 Jan 2018, Mark Webb-Johnson wrote:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Assuming the new file system is stable, we should move to it. Remind me the syntax for scp... we're using v1, right? Just warn us when the change is checked in... Greg Mark Webb-Johnson wrote:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OK, I looked into this is more detail… Notes Presently, spiffs does not support directories. It produces a flat structure. If SPIFFS is mounted under /spiffs creating a file with path /spiffs/tmp/myfile.txt will create a file called /tmp/myfile.txt in SPIFFS, instead of myfile.txt under directory /spiffs/tmp. It is not a realtime stack. One write operation might last much longer than another. Presently, it does not detect or handle bad blocks. Maybe not… It seems that for our use-case, the wear levelled FAT maybe more suitable. Mark
On 19 Jan 2018, at 11:23 AM, Greg D. <gregd2350@gmail.com> wrote:
Assuming the new file system is stable, we should move to it. Remind me the syntax for scp... we're using v1, right?
Just warn us when the change is checked in...
Greg
Mark Webb-Johnson wrote:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
My quick reaction on the second caveat is: How can any Flash file system be real-time? Flash writes that span blocks take longer than those that don’t. The only solution is RAM buffering and threading, but those have their impacts on embedded systems due to limited RAM and limited CPU for threads. I’ve used JFFS2 on Linux, but that’s hardly a real-time situation, even though we were getting near-real-time performance. Seems like JFFS2 might be too complex, unless it is built in layers where we could perhaps shed the high-level Posix features and just borrow the low-level, Flash-specific features. All in all, SPIFFS does seem a bit limited. However, FAT is rather limited by today’s standards… I’d like to see some sort of wear leveling, although the nature of log files is that they might be nearly ideal for wear leveling. It’s the non-log settings files that are more of a challenge. How many files are we talking about? A flat (non-directory) structure might not be so bad with only a few files. Brian On Jan 19, 2018, at 10:32 AM, Greg D. <gregd2350@gmail.com> wrote:
Yeah, that seems like a killer. Wear-leveled FAT, it is. They don't support something like JFFS2, do they? Probably overkill...
Greg
Mark Webb-Johnson wrote:
OK, I looked into this is more detail…
Notes
• Presently, spiffs does not support directories. It produces a flat structure. If SPIFFS is mounted under /spiffs creating a file with path /spiffs/tmp/myfile.txt will create a file called /tmp/myfile.txt in SPIFFS, instead of myfile.txt under directory /spiffs/tmp. • It is not a realtime stack. One write operation might last much longer than another. • Presently, it does not detect or handle bad blocks. Maybe not… It seems that for our use-case, the wear levelled FAT maybe more suitable.
Mark
Mark Webb-Johnson wrote:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
I’d like to see some sort of wear leveling, although the nature of log files is that they might be nearly ideal for wear leveling. It’s the non-log settings files that are more of a challenge.
The current system we use is FAT over a wear-levelling layer (using same 4KB block size as the FAT layer, for simplicity).
How many files are we talking about? A flat (non-directory) structure might not be so bad with only a few files.
Not too many. Part of it is the user config, which we want to keep separate from generic user data. Using directories makes that simple. Regards, Mark.
On 21 Jan 2018, at 7:17 AM, HONDA S-2000 <s2000@audiobanshee.com> wrote:
My quick reaction on the second caveat is: How can any Flash file system be real-time? Flash writes that span blocks take longer than those that don’t. The only solution is RAM buffering and threading, but those have their impacts on embedded systems due to limited RAM and limited CPU for threads.
I’ve used JFFS2 on Linux, but that’s hardly a real-time situation, even though we were getting near-real-time performance. Seems like JFFS2 might be too complex, unless it is built in layers where we could perhaps shed the high-level Posix features and just borrow the low-level, Flash-specific features.
All in all, SPIFFS does seem a bit limited. However, FAT is rather limited by today’s standards…
I’d like to see some sort of wear leveling, although the nature of log files is that they might be nearly ideal for wear leveling. It’s the non-log settings files that are more of a challenge.
How many files are we talking about? A flat (non-directory) structure might not be so bad with only a few files.
Brian
On Jan 19, 2018, at 10:32 AM, Greg D. <gregd2350@gmail.com> wrote:
Yeah, that seems like a killer. Wear-leveled FAT, it is. They don't support something like JFFS2, do they? Probably overkill...
Greg
Mark Webb-Johnson wrote:
OK, I looked into this is more detail…
Notes
• Presently, spiffs does not support directories. It produces a flat structure. If SPIFFS is mounted under /spiffs creating a file with path /spiffs/tmp/myfile.txt will create a file called /tmp/myfile.txt in SPIFFS, instead of myfile.txt under directory /spiffs/tmp. • It is not a realtime stack. One write operation might last much longer than another. • Presently, it does not detect or handle bad blocks. Maybe not… It seems that for our use-case, the wear levelled FAT maybe more suitable.
Mark
Mark Webb-Johnson wrote:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
I just came across this on MickMake: The little filesystem (https://github.com/geky/littlefs) Seems to have support for the most important things like directories (I think), wear leveling, power-loss handling and they claim a low RAM and ROM footprint. Also it seems to do a great better than SPIFFS when it comes to performance: https://github.com/RIOT-OS/RIOT/pull/8316 Could it be an alternative? Best regards, Geir
22. jan. 2018 kl. 04:16 skrev Mark Webb-Johnson <mark@webb-johnson.net>:
I’d like to see some sort of wear leveling, although the nature of log files is that they might be nearly ideal for wear leveling. It’s the non-log settings files that are more of a challenge.
The current system we use is FAT over a wear-levelling layer (using same 4KB block size as the FAT layer, for simplicity).
How many files are we talking about? A flat (non-directory) structure might not be so bad with only a few files.
Not too many. Part of it is the user config, which we want to keep separate from generic user data. Using directories makes that simple.
Regards, Mark.
On 21 Jan 2018, at 7:17 AM, HONDA S-2000 <s2000@audiobanshee.com> wrote:
My quick reaction on the second caveat is: How can any Flash file system be real-time? Flash writes that span blocks take longer than those that don’t. The only solution is RAM buffering and threading, but those have their impacts on embedded systems due to limited RAM and limited CPU for threads.
I’ve used JFFS2 on Linux, but that’s hardly a real-time situation, even though we were getting near-real-time performance. Seems like JFFS2 might be too complex, unless it is built in layers where we could perhaps shed the high-level Posix features and just borrow the low-level, Flash-specific features.
All in all, SPIFFS does seem a bit limited. However, FAT is rather limited by today’s standards…
I’d like to see some sort of wear leveling, although the nature of log files is that they might be nearly ideal for wear leveling. It’s the non-log settings files that are more of a challenge.
How many files are we talking about? A flat (non-directory) structure might not be so bad with only a few files.
Brian
On Jan 19, 2018, at 10:32 AM, Greg D. <gregd2350@gmail.com> wrote:
Yeah, that seems like a killer. Wear-leveled FAT, it is. They don't support something like JFFS2, do they? Probably overkill...
Greg
Mark Webb-Johnson wrote:
OK, I looked into this is more detail…
Notes
• Presently, spiffs does not support directories. It produces a flat structure. If SPIFFS is mounted under /spiffs creating a file with path /spiffs/tmp/myfile.txt will create a file called /tmp/myfile.txt in SPIFFS, instead of myfile.txt under directory /spiffs/tmp. • It is not a realtime stack. One write operation might last much longer than another. • Presently, it does not detect or handle bad blocks. Maybe not… It seems that for our use-case, the wear levelled FAT maybe more suitable.
Mark
Mark Webb-Johnson wrote:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
No problem for me either. Geir Sendt fra min iPhone
19. jan. 2018 kl. 02:29 skrev Mark Webb-Johnson <mark@webb-johnson.net>:
ESP IDF 2.1 only had FAT support as a filesystem. We used this for the /store vfs mount, for config and script storage.
ESP IDF 3.x now has SPIFFS as an option.
SPIFFS is obviously more suited to embedded flash.
What do people think about making the switch to SPIFFS for /store now?
Doing so would probably not be hard (just a few lines of code change in the mounting of that /store partition), but would lose all config and script data in that partition (developers would have to disable vfs protection in menuconfig, then scp out first, to backup).
Doing it later is not really an option.
Thoughts?
Mark
_______________________________________________ OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
participants (5)
-
Geir Øyvind Vælidalo -
Greg D. -
HONDA S-2000 -
Mark Webb-Johnson -
Stephen Casner