[Ovmsdev] Getting data off the OVMS unit and archived

[Michael Balzer]

Steve,

for mid to low time resolution/volume data, use "historical" messages 
(notification class "data"). You're totally free on the structure of 
"historical" messages, see the Twizy module for some examples:

https://docs.openvehicles.com/en/latest/components/vehicle_renaulttwizy/docs/notifications.html

For high resolution/volume data, using historical messages isn't 
recommended (the notification hub has only the SPIRAM cache, so ~ 3 MB). 
It's better to write this to a file on the SD card. Files can be fetched 
from SD via scp, http or by carrying the SD card to your PC.

An example for this is the Twizy's controller log, implemented in this 
module: 
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/blob/master/vehicle/OVMS.V3/components/vehicle_renaulttwizy/src/rt_sevcon_mon.cpp

The SEVCON monitor writes a CSV file to SD at a rate of 10 rows per second.

The third option is streaming the data to a connected WebSocket or TCP 
client. Streaming via TCP needs to be implemented via Mongoose, see RE 
tools for an example. Streaming via WebSocket is integrated into the 
notification framework (class "stream"), see again RE tools for an example.

Server V2 / MP records also aren't limited to once per minute, even some 
standard messages as the location updates will be sent in short 
intervals if requested by the user. Historical records are sent as soon 
as possible, but are throttled to ~ 5 per second to not overload the PPP 
channel.

The notification system buffers messages automatically (see above), if 
at least one registered listener is started. So you can for example 
start the V2 server but shutdown the modem connection. Then the system 
will buffer all data until you're connected via Wifi again.

Regards,
Michael


Am 11.01.21 um 17:09 schrieb Steve Davies:
> Hi,
>
> What do people do to retain data from OVMS for later review and 
> analysis?  Are there any projects around for longer term analysis?
>
> I see the ovms_historicalmessages table and I assume I can find docs 
> on those record types and what all the fields are.
>
> But I have Influxdb and Homeassistant / openenergymonitor / NodeRed so 
> my inclination would be to push all the data in there since InfluxDB 
> can do nice time analysis of data.
>
> It looks like data is only sent to the server once per minute - so I'm 
> not going to be getting things like motor power, speed, throttle 
> position at a high resolution?
>
> On the mqtt side is it sent at the same rate?
>
> Is there any existing method to collect high resolution samples but 
> send in a chunk each minute, or dump once back in wifi range?
>
> Thanks,
> Steve

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26


-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 203 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20210111/00817b8a/attachment.sig>