Steve,

Am 15.01.21 um 10:04 schrieb Steve Davies:

This seems more manageable than the continuous streaming, a sensible compromise.

As I said, the streaming mode wasn't meant to stream all metrics. The V3 server lacks a means to control metrics update rates. Feel free to add one, also take into account what has been discussed on this before, i.e. https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/344

But should I be working with MQTT? Right now Mark and Michael you are by far the biggest contributors so I'd definitely take a steer from you about this. I don't think it helps the project to have two different approaches "competing" in the same code base.

It's not a competition, it's supporting two different approaches. We even support a third approach in form of our WebSocket protocol, which can be used for direct local connections to the module.

V2 / MP is a custom protocol and more of a legacy support. But it has undeniable advantages over the current MQTT implementation.

You say:

> The V3 server needs further development from actual users like you. I don't use MQTT because of the protocol overhead. So go ahead, refine it.

So the blunt question is whether this MQTT / V3 approach is dying on the vine or if it is worth investing time into.

Looks like Mark wrote most of it 3 years ago.

For me I'm used to MQTT, I like it and use it in my house control systems, energy monitoring, OpenEVSE etc. It's exactly designed for the IOT world. NodeRed, Homeassistant and other tools support it well.

But OVMS is already invested in the V2 protocol and you are still developing it.

As implemented, the V3/MQTT approach will definitely have more overhead - and indeed sending every metric in a separate message will use a lot more data.

But nothing says that we can't have a "compact" format similar to the data messages that V2 sends. MQTT adds just a few bytes per published message for the MQTT protocol so that would end up being nearly as efficient unless I've missed something?

It does mean that consumers of the MQTT messages need to work harder to parse them - they would need to know the order of the values. We couldn't use a self-describing format like JSON since that has plenty of overhead of its own.

I think you're describing a valid approach to reduce the MQTT overhead, basically something like sending the V2 MP records over MQTT instead of single metrics.

I just don't know if that's what people expect from MQTT. It removes all the simplicity from receiving metric values via MQTT. Will standard MQTT clients & tools be able to parse such records? And even if they do, it would still introduce a hurdle for users trying to get into telemetry processing. Part of the appeal of MQTT is being able to use a broad selection of generic standard clients.

Another approach is - instead of having the V3/MQTT "server" on OVMS, to rather have a two way gateway that runs server-side that connects to the V2 server and proxies back and forth from MQTT. I would see this as an "app" side gateway - so metric records coming from the car get published to mqtt (v2->v3) and commands are translated back the other way (v3->v2). Then the extra protocol overhead for the existing MQTT format doesn't matter since we aren't going to be sending over an "IOT" SIM over 3G.

What are your thoughts?

That would introduce the need for a second server / gateway instance just to attach to MP via MQTT. Possible, but then, MP isn't that hard to connect directly to. There are many example clients. The gateway would still have a benefit if MQTT clients can parse MP records though.

How about eliminating the need for that gateway by adding optional standard & configurable custom topic record structures to the V3 server? In the most simple form, a record structure is simply a list of metric names to join under a definable topic. A little bit of number formatting would be useful, e.g. using printf style format strings.

Example: the MP "L" record structure could be described by some string like this:

{v.p.latitude},{v.p.longitude},{v.p.direction},{v.p.altitude},{v.p.gpslock},{v.p.speed},{v.p.trip},{v.e.drivemode%x},{v.b.power%.3f},{v.b.energy.used%.3f},…

JSON format could be supported by some keyword to request JSON value encoding. You could even introduce a way to get the value from a Javascript function, like Greg does for the obd2ecu system.

Combine that with a decent metric filter & update rate configuration (see issue link above – btw, we already have topic pattern support from Mongoose…), and you get the best of both approaches, without any added transport complexity or points of failure.

Standard users will be able to subscribe to single metric values, and power users will be able to easily create custom records.

Just my 2 cents as a yet-to-become MQTT user.

Thanks,

Steve

Regards,
Michael

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26