[Ovmsdev] MQTT and Ovms Server v3

[Mark Webb-Johnson]

I am far from an expert in MQTT. Not even a novice. So, the work below is ‘best efforts’. Any help / comments / suggestions would be most appreciated. In particular, a big thanks to Jakob for his contributions to this so far.

With yesterday’s merge, and commits, we have a very very basic OVMS server v3 implementation. It sends the metrics, and doesn’t crash (much). The overall design is as follows:

We use the mongoose mqtt library. We don’t do anything special, and everything is following the mqtt 3.1 standard.

MQTT has the concept of topics. Our default prefix for everything is:

ovms/<mqtt-username>/<vehicleid>/

(that can be customised with config server.v3 topic.prefix)

Metrics are sent as topics. The metric name is added to the topic prefix + “m/“ suffix, and “.” characters converted to “/“ to match the MQTT conventions. The value is simply the metric value (as a string). With our current arrangement, this adds m/m/, m/s/, and m/v/ sub-trees to the the MQTT topic hierarchy. Clients can subscribe to <prefix>/m/# to receive all metrics. The ‘retained’ flag is set on these metrics, at QOS level 0 (so subscribing clients will get a copy of the last known values for these metrics, even with a disconnected vehicle).

The metric s.v3.connected is maintained by the ServerV3 code. When a successful MQTT connection is made, and login successful, that is set true (yes). A MQTT last-will-testament is set so that if the OVMS module disconnects the server will automatically update that to false (no). The ‘retained’ flag is set on this metric, at QOS level 0 (so subscribing clients will get a copy of the last known state). Clients can use this to see if the vehicle is connected to the server.

Connecting clients are expected to write a “1” value to the <prefix>/c/<clientid> topic, and to repeat that write once a minute. They are also expected to use a last-will-testament on that same topic with value “0”. QOS 1 should be used, and these topics should not be retained. The server V3 code subscribes to this <prefix>/c/# topic, so it gets informed of all connected clients. It can then update the s.v3.peers metric appropriately. Clients are expected to monitor the <prefix>/s/v3/connected topic, so that if it becomes ‘yes’ (true) the client should send <prefix>/c/<clientid> “1” immediately. This mechanism allows a newly connected vehicle to immediately know if one or more clients is connected.

The Server v3 sets a timeout for <prefix>/c/<clientid> connections of 2 minutes. If that mqtt topic is not sent again within that time, it is expired (and that client is treated as disconnected).

Similar to the v2 code, the server v3 transmits modified metrics once a minute if there are one or more clients connected, or once every ten minutes if there are no clients connected.

All the above has been implemented. To reach parity with v2, and exceed it’s functionality in places, we have a few more things to do:

Notifications

On the vehicle side, I am proposing to use the <prefix>/notify/<type> namespace for these, using QOS 2 messages without retention. Clients can listen to those, if necessary. We can also have a central daemon running that listens to ovms/+/+/n/# topic pattern to receive these notifications and handle appropriately. Using QOS 2 we can confirm the reception of the notification / historical data, and mark it as delivered, appropriately. However, that would only confirm delivery to the MQTT server, not to the central daemon; if the daemon was not running, the notification would be lost.

Textual Commands

I am proposing to use the <prefix>/cmd/<clientid>/c/ and <prefix>/cmd/<clientid>/r/ namespaces for this, using QOS 2 messages without retention. The value in the /c/ would be the command, and the response would be written to matching the /r/ topic. The commands and responses could be prefixed by an identifier (like in imap protocol) so responses can be matched to commands by the clients). The client side can simply subscribe to itself, and the vehicle side subscribes to <prefix>/cmd/#. In this way, commands cannot be duplicated, and clients don’t see responses to commands they didn’t initiate (which was an issue with v2).

Numeric Commands

I am not sure if we need to implement the numeric commands, as used in the v2 protocol. It seems to me that we can use textual commands.

Config Access

I am not sure if we need this, beyond the command processor. If we do, we could expose a /config/ namespace.

Events

It would be nice to expose events (except for the ticker.* ones, of course). This could be done by a <prefix>/events topic, using QOS 2 and no retention.

Logs

It would be nice to expose logs. This could be done by a <prefix>/logs topic, using QOS 1 and no retention.

Security

We need to add SSL support. I am trying to get an authentication plugin for mosquitto / vernemq written so that we can authenticate straight from the OVMS database that is already running on the servers, and give each user their own ovms/<userid>/# namespace. That way, the configuration for v3 on the vehicle/apps/clients is simple - just put in the server username and password (no separate vehicle passwords necessary).

I think that is it. The above would form the basis of the specification for this. As this is the basis for future work and direction in OVMS, it is important that we get it right, so all comments / suggestions most welcome.

Regards, Mark.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20180705/d0dff904/attachment-0001.html>