<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Regarding MQTT, I definitely think it is the correct direction for the future, and hope it can replace v2 protocol in the long run. The advantages are many, and the only disadvantage seems to be data size on the wire.<div class=""><br class=""></div><div class="">To address that, I have been waiting for v5 MQTT to become more widespread. As well as some other desirable features, that adds topic aliases. This means we can replace the topic name with a 4 byte integer. Along with the protocol overhead that is still slightly more than the single byte comma field separator of v2 protocol, but has the advantage that a single changed metric can be transmitted without having to transmit the entire v2 protocol message.</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class="">Implementation is relatively straightforward. The first time a particular message is sent, the full topic name and value are used, along with a 4 byte integer topic alias. Subsequent updates (on the same connection session) can leave the full topic name blank and just provide the 4 byte integer topic alias (with the expansion done at the server side). There is also a bunch of stuff in v5 regarding session resumption, to further reduce the overhead.</div></blockquote><div class=""><br class=""></div><div class="">There is now a development version of mosquito that supports v5, and hivemq has become sort of open source. Our current networking library doesn’t support it, however. Perhaps now is the time to re-visit this (in particular if others are willing to work on it)?</div><div class=""><br class=""></div><div class="">Regards, Mark.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 16 Jan 2021, at 3:50 PM, Michael Balzer <<a href="mailto:dexter@expeedo.de" class="">dexter@expeedo.de</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="content-isolator__container"><div class="protected-part"><div class="protected-title">Signed PGP part</div><div class="protected-content">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div class="">
Steve,<br class="">
<br class="">
forgot to mention: the V3 server already supports "data"
notifications. These are published under the topic scheme
"<prefix>/notify/data/<subtype>/<id>/<timestamp>".<br class="">
<br class="">
So you can already send compact custom records. V2 "history" records
are sent that way, see <font face="monospace" class="">OvmsVehicleRenaultTwizy::SendTripLog()</font>
for an example.<br class="">
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<br class="">
<div class="moz-cite-prefix">Am 15.01.21 um 13:55 schrieb Michael
Balzer:<br class="">
</div>
<blockquote type="cite" cite="mid:0e8b32cd-d4f5-263a-b391-21e173acddf5@expeedo.de" class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
Steve,<br class="">
<br class="">
<div class="moz-cite-prefix">Am 15.01.21 um 10:04 schrieb Steve
Davies:<br class="">
</div>
<blockquote type="cite" cite="mid:CABFTEGU6q3H2=PsPTiyreoWUyuixxPge=aD3BDipxfe2StkZCg@mail.gmail.com" class="">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8" class="">
<div dir="ltr" class="">This seems more manageable than the continuous
streaming, a sensible compromise.</div>
</blockquote>
<br class="">
As I said, the streaming mode wasn't meant to stream all metrics.
The V3 server lacks a means to control metrics update rates. Feel
free to add one, also take into account what has been discussed on
this before, i.e.
<a class="moz-txt-link-freetext" href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/344" moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/344</a><br class="">
<br class="">
<blockquote type="cite" cite="mid:CABFTEGU6q3H2=PsPTiyreoWUyuixxPge=aD3BDipxfe2StkZCg@mail.gmail.com" class="">
<div dir="ltr" class="">
<div class="">But should I be working with MQTT? Right now Mark and
Michael you are by far the biggest contributors so I'd
definitely take a steer from you about this. I don't think
it helps the project to have two different approaches
"competing" in the same code base.</div>
</div>
</blockquote>
<br class="">
It's not a competition, it's supporting two different approaches.
We even support a third approach in form of our WebSocket
protocol, which can be used for direct local connections to the
module.<br class="">
<br class="">
V2 / MP is a custom protocol and more of a legacy support. But it
has undeniable advantages over the current MQTT implementation.<br class="">
<br class="">
<blockquote type="cite" cite="mid:CABFTEGU6q3H2=PsPTiyreoWUyuixxPge=aD3BDipxfe2StkZCg@mail.gmail.com" class="">
<div dir="ltr" class="">
<div class="gmail_quote">
<div class="">You say:</div>
<div class=""><br class="">
</div>
<div class=""> > The V3 server needs further development from
actual users like you. I don't use MQTT because of the
protocol overhead. So go ahead, refine it.</div>
<div class=""><br class="">
</div>
<div class="">So the blunt question is whether this MQTT / V3
approach is dying on the vine or if it is worth investing
time into. </div>
<div class=""><br class="">
</div>
<div class="">Looks like Mark wrote most of it 3 years ago.</div>
<div class=""><br class="">
</div>
<div class="">For me I'm used to MQTT, I like it and use it in my
house control systems, energy monitoring, OpenEVSE etc.
It's exactly designed for the IOT world. NodeRed,
Homeassistant and other tools support it well.</div>
<div class=""><br class="">
</div>
<div class="">But OVMS is already invested in the V2 protocol and you
are still developing it.</div>
<div class=""><br class="">
</div>
<div class="">As implemented, the V3/MQTT approach will definitely
have more overhead - and indeed sending every metric in a
separate message will use a lot more data.</div>
<div class=""><br class="">
</div>
<div class="">But nothing says that we can't have a "compact" format
similar to the data messages that V2 sends. MQTT adds
just a few bytes per published message for the MQTT
protocol so that would end up being nearly as efficient
unless I've missed something?</div>
<div class=""><br class="">
</div>
<div class="">It does mean that consumers of the MQTT messages need
to work harder to parse them - they would need to know
the order of the values. We couldn't use a
self-describing format like JSON since that has plenty of
overhead of its own.</div>
</div>
</div>
</blockquote>
<br class="">
I think you're describing a valid approach to reduce the MQTT
overhead, basically something like sending the V2 MP records over
MQTT instead of single metrics.<br class="">
<br class="">
I just don't know if that's what people expect from MQTT. It
removes all the simplicity from receiving metric values via MQTT.
Will standard MQTT clients & tools be able to parse such
records? And even if they do, it would still introduce a hurdle
for users trying to get into telemetry processing. Part of the
appeal of MQTT is being able to use a broad selection of generic
standard clients.<br class="">
<br class="">
<blockquote type="cite" cite="mid:CABFTEGU6q3H2=PsPTiyreoWUyuixxPge=aD3BDipxfe2StkZCg@mail.gmail.com" class="">
<div dir="ltr" class="">
<div class="gmail_quote">
<div class="">Another approach is - instead of having the V3/MQTT
"server" on OVMS, to rather have a two way gateway that
runs server-side that connects to the V2 server and
proxies back and forth from MQTT. I would see this as an
"app" side gateway - so metric records coming from the car
get published to mqtt (v2->v3) and commands are
translated back the other way (v3->v2). Then the extra
protocol overhead for the existing MQTT format doesn't
matter since we aren't going to be sending over an "IOT"
SIM over 3G.</div>
<div class=""><br class="">
</div>
<div class="">What are your thoughts?</div>
</div>
</div>
</blockquote>
<br class="">
That would introduce the need for a second server / gateway
instance just to attach to MP via MQTT. Possible, but then, MP
isn't that hard to connect directly to. There are many example
clients. The gateway would still have a benefit if MQTT clients
can parse MP records though.<br class="">
<br class="">
How about eliminating the need for that gateway by adding optional
standard & configurable custom topic record structures to the
V3 server? In the most simple form, a record structure is simply a
list of metric names to join under a definable topic. A little bit
of number formatting would be useful, e.g. using printf style
format strings.<br class="">
<br class="">
Example: the MP "L" record structure could be described by some
string like this:<br class="">
<br class="">
<font face="monospace" class="">{v.p.latitude},{v.p.longitude},{v.p.direction},{v.p.altitude},{v.p.gpslock},{v.p.speed},{v.p.trip},{v.e.drivemode%x},{v.b.power%.3f},{v.b.energy.used%.3f},…</font><br class="">
<br class="">
JSON format could be supported by some keyword to request JSON
value encoding. You could even introduce a way to get the value
from a Javascript function, like Greg does for the obd2ecu system.<br class="">
<br class="">
Combine that with a decent metric filter & update rate
configuration (see issue link above – btw, we already have topic
pattern support from Mongoose…), and you get the best of both
approaches, without any added transport complexity or points of
failure.<br class="">
<br class="">
Standard users will be able to subscribe to single metric values,
and power users will be able to easily create custom records.<br class="">
<br class="">
Just my 2 cents as a yet-to-become MQTT user.<br class="">
<br class="">
<blockquote type="cite" cite="mid:CABFTEGU6q3H2=PsPTiyreoWUyuixxPge=aD3BDipxfe2StkZCg@mail.gmail.com" class="">
<div dir="ltr" class="">
<div class="gmail_quote">
<div class="">Thanks,</div>
<div class="">Steve</div>
</div>
</div>
</blockquote>
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
<br class="">
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br class="">
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</div>
</div></div><br class=""><iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" width="200" height="10" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="></iframe></div></div></blockquote></div><br class=""></div></body></html>