[Ovmsdev] Automatic naming of CAN log files

[Ludovic LANGE]

Hi,

I'm planning to implement an "automatic naming" of the CAN log files for 
the VFS target. It would also take care of the file name rotation 
according to some criterion (file size, and/or log duration).

I've created an issue to follow it here 
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/748 
(and some others also) but let me first share what I have in mind:


The use case is the following: I want to setup OVMS as a CAN bus 
datalogger. It should be always-on, always and continuously logging, 
event after a restart.

(For that, my current solution is to use event scripts, cf 
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/766 
)

Logging to the same file, in that case, present some challenges:

  * file size would grow indefinitely, which could be then make it
    difficult to copy / network send somewhere
  * file would always be opened, which could prevent any action on it
    (if we want to send it somewhere else, or remove it)
  * in case of crash / corruption the whole file could be lost
  * in case of reboot you'd end overwriting the file.

Of course we should change the file name, but it's not easy to do it in 
a script.

It may be possible to do this with Duktape and `|OvmsCommand.Exec|` but 
I fear that we may miss events if we stop / start logging on a highly 
loaded bus.


A proposal to solve this would be to add a new mode in `canlog_vfs.cpp` 
that would cater for this automatic naming (inspiration for this feature 
coming from another product).


In this mode you don't pass the path as a parameter, but instead a `log 
prefix` that will be used to construct the file path.

Syntax could be something like : `can log start vfs-auto crtd <log 
prefix> [filter1] ... [filterN]` (no filename).

It would automatically generate a log file name, taking into account the 
splits / rotations that can be configured:

  * if asked for by configuration - rotate file name (= increment file
    name) after a file size is reached
  * if asked for by configuration - rotate file name (= increment file
    name) after a certain logging duration has been reached


Naming would be 'automatic' with something like :
`/sd/log/<vehicleid>/<#power on>/<log prefix>/<#rotations>.<log extension>`

where:
* `/sd/log/` is arbitrary. Could be coming from configuration if wanted 
(esp. for module without sd card)
* `<vehicleid>` is there to help to declutter your sdcard (if you share 
it with multiple modules), or to declutter your storage server (when 
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/749 
has been completed). We may have to make it a valid filename.
* `<#power on>` is the number of time the module has powered on ; it is 
an always incrementing number expressed as an integer on 8 digits. 
Depending on the configuration, some modules will be always-on ; others 
will stop more frequently - the number will overflow after 99.999.999 
power on, which will certainly see a change of sd card, so no fear of 
unwanted overwriting of files
* `<log prefix>` is a way to distinguish between multiple concurrent can 
logs active.
* `<#rotations>` is a counter of rotations for this file and this power 
on ; it is an integer on 8 digits ; reset to `00000001` at each poweroff 
and that will increment each time there is a file name rotation.

some configuration items cater for the file name rotation:
* `split_size_mb` : when the file has reached this size, the file is 
closed, the element `<#rotations>` is incremented, and a new file with 
the 'automatic' name is opened.
* `split_duration_s` : when the file has logged for this duration, the 
file is closed, the element `<#rotations>` is incremented, and a new 
file with the 'automatic' name is opened.



example of file name:  `/sd/log/mycar/00000024/can1-25k/00000005.crtd`


(Of course we need a way to either stop logging once the VFS is full, or 
to auto-remove some log files under this path)


Let me know if you have any thoughts / concerns about this.


Regards,

Ludovic

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20221109/7f948228/attachment.htm>