<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Ludovic,<br>
<br>
most of your thoughts were also mine on designing the file logging
scheme, I suggest taking that as a reference & inspiration.<br>
<br>
I suggest controlling this through a similar config scheme,
introducing more can config param instances as needed. There's
currently only the queue size:<br>
<br>
<font face="monospace">OVMS# con lis can<br>
can (readable writeable)<br>
log.queuesize: 300</font><br>
<br>
The file logging sync period feature is what you need to not lose a
complete file on crash, it also enables following a file by "vfs
tail" and transferring the file content written so far without
stopping the logger.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 09.11.22 um 22:52 schrieb Ludovic
LANGE:<br>
</div>
<blockquote type="cite"
cite="mid:3a3a4714-e439-56a9-885f-f8098b21f603@lange.nom.fr">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<p>Hi,<br>
</p>
<p>I'm planning to implement an "automatic naming" of the CAN log
files for the VFS target. It would also take care of the file
name rotation according to some criterion (file size, and/or log
duration).</p>
<p>I've created an issue to follow it here
<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/748"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/748</a>
(and some others also) but let me first share what I have in
mind:<br>
</p>
<p><br>
</p>
<p>The use case is the following: I want to setup OVMS as a CAN
bus datalogger. It should be always-on, always and continuously
logging, event after a restart.<br>
</p>
<p>(For that, my current solution is to use event scripts, cf
<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/766"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/766</a>
)<br>
</p>
<p>Logging to the same file, in that case, present some
challenges:</p>
<ul>
<li>file size would grow indefinitely, which could be then make
it difficult to copy / network send somewhere<br>
</li>
<li>file would always be opened, which could prevent any action
on it (if we want to send it somewhere else, or remove it)</li>
<li>in case of crash / corruption the whole file could be lost</li>
<li>in case of reboot you'd end overwriting the file.<br>
</li>
</ul>
<p>Of course we should change the file name, but it's not easy to
do it in a script.</p>
<p>It may be possible to do this with Duktape and `<code
class="docutils literal notranslate"><span class="pre">OvmsCommand.Exec</span></code>`
but I fear that we may miss events if we stop / start logging on
a highly loaded bus.<br>
</p>
<p><br>
</p>
<p>A proposal to solve this would be to add a new mode in
`canlog_vfs.cpp` that would cater for this automatic naming
(inspiration for this feature coming from another product).</p>
<p><br>
In this mode you don't pass the path as a parameter, but instead
a `<font face="monospace">log prefix</font>` that will be used
to construct the file path. <br>
<br>
Syntax could be something like : `<font face="monospace">can log
start vfs-auto crtd <log prefix> [filter1] ... [filterN]</font>`
(no filename).<br>
<br>
It would automatically generate a log file name, taking into
account the splits / rotations that can be configured:<br>
</p>
<ul>
<li>if asked for by configuration - rotate file name (=
increment file name) after a file size is reached</li>
<li>if asked for by configuration - rotate file name (=
increment file name) after a certain logging duration has been
reached</li>
</ul>
<p><br>
Naming would be 'automatic' with something like : <br>
`<font face="monospace">/sd/log/<vehicleid>/<#power
on>/<log prefix>/<#rotations>.<log
extension></font>`<br>
<br>
where:<br>
* `<font face="monospace">/sd/log/</font>` is arbitrary. Could
be coming from configuration if wanted (esp. for module without
sd card)<br>
* `<font face="monospace"><vehicleid></font>` is there to
help to declutter your sdcard (if you share it with multiple
modules), or to declutter your storage server (when
<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/749"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/749</a>
has been completed). We may have to make it a valid filename.<br>
* `<font face="monospace"><#power on></font>` is the
number of time the module has powered on ; it is an always
incrementing number expressed as an integer on 8 digits.
Depending on the configuration, some modules will be always-on ;
others will stop more frequently - the number will overflow
after 99.999.999 power on, which will certainly see a change of
sd card, so no fear of unwanted overwriting of files<br>
* `<font face="monospace"><log prefix></font>` is a way to
distinguish between multiple concurrent can logs active.<br>
* `<font face="monospace"><#rotations></font>` is a
counter of rotations for this file and this power on ; it is an
integer on 8 digits ; reset to `<font face="monospace">00000001</font>`
at each poweroff and that will increment each time there is a
file name rotation.<br>
<br>
some configuration items cater for the file name rotation:<br>
* `<font face="monospace">split_size_mb</font>` : when the file
has reached this size, the file is closed, the element `<font
face="monospace"><#rotations></font>` is incremented,
and a new file with the 'automatic' name is opened.<br>
* `<font face="monospace">split_duration_s</font>` : when the
file has logged for this duration, the file is closed, the
element `<font face="monospace"><#rotations></font>` is
incremented, and a new file with the 'automatic' name is opened.</p>
<p><br>
<br>
example of file name: `<font face="monospace">/sd/log/mycar/00000024/can1-25k/00000005.crtd</font>`</p>
<p><br>
</p>
<p>(Of course we need a way to either stop logging once the VFS is
full, or to auto-remove some log files under this path)<br>
</p>
<p><br>
</p>
<p>Let me know if you have any thoughts / concerns about this.</p>
<p><br>
</p>
<p>Regards,</p>
<p>Ludovic</p>
<pre class="moz-quote-pre" wrap=""></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</body>
</html>