Hey,
Regarding naming:
I suppose the following topic names:
metrics: <prefix>/metric/#
events: <prefix>/event/#
notifications: <prefix>/notify/#
config: <prefix>/config/#
logs: <prefix>/log/<tag>
active: <prefix>/client/<clientid>/active
requests: <prefix>/client/<clientid>/request/#
commands: <prefix>/client/<clientid>/command/<command id>
cmd responses: <prefix>/client/<clientid>/response/<command id>
Filtering:
In order to decrease used data a blacklist of metrics which are not
send (like m.freeram or m.monotonic) would be cool.
heartbeats:
I've seen you implemented that clients have to send a message every two
minutes. A topic like the above .../<clientid>/active which is set to 1
when a client connects and to 0 by its last will would have the same
effect but not require heartbeats to be sent every two minutes and thus
use less of our precious cellular data.
Requesting values:
In order to decrease data usage even more allowing clients to request
the value of a metric using <prefix>/client/<clientid>/request/metric
and the metric name in the value or using
<prefix>/client/<clientid>/request/config and the config name in the
value. Allowing wildcards like
m.net.* or just * allows to request
multiple or all metrics with a single request. Then e.g. the app would
request all metrics it wants to display.
Encryption:
TLS in general seems to be a missing thing with OVMS. Downloading
firmware updates over http and not doing any verification is kind of
meh. The problem is (as noted in my last email) with the CA certs. My
suggestion is to ship with one default CA (e.g. Let's Encrypt) and
allow the user to replace it with a different one.
I would love to implement this, but I am currently in exam phase and
won't have much time to spare until next week.
Authentification:
If you can live with moving to mosquitto[0] (MQTT broker by eclipse)
they have a very good auth plugin[1]. All you have to do is to write an
sql statement which receives the hashed passwords from the OVMS
database by username (see [2]). If you are on debian its just an apt
install mosquitto mosquitto-auth-plugin away.
It is very important to set up ACL (access control lists) which make
sure only user X can subscribe/publish to ovms/X and noone else.
Luckily this is also handeled by mosquitto_auth_plug
[0]
https://mosquitto.org/[1]
https://github.com/jpmens/mosquitto-auth-plug[2]
https://github.com/jpmens/mosquitto-auth-plug#postgresqlOn Thu, 2018-07-05 at 09:26 +0800, Mark Webb-Johnson wrote:
I am far from an expert in MQTT. Not even a novice. So, the work
below is ‘best efforts’. Any help / comments / suggestions would be
most appreciated. In particular, a big thanks to Jakob for his
contributions to this so far.
With yesterday’s merge, and commits, we have a very very basic OVMS
server v3 implementation. It sends the metrics, and doesn’t crash
(much). The overall design is as follows:
We use the mongoose mqtt library. We don’t do anything special, and
everything is following the mqtt 3.1 standard.
MQTT has the concept of topics. Our default prefix for everything is:
ovms/<mqtt-username>/<vehicleid>/
(that can be customised with config server.v3 topic.prefix)
Metrics are sent as topics. The metric name is added to the topic
prefix + “m/“ suffix, and “.” characters converted to “/“ to match
the MQTT conventions. The value is simply the metric value (as a
string). With our current arrangement, this adds m/m/, m/s/, and m/v/
sub-trees to the the MQTT topic hierarchy. Clients can subscribe to
<prefix>/m/# to receive all metrics. The ‘retained’ flag is set on
these metrics, at QOS level 0 (so subscribing clients will get a copy
of the last known values for these metrics, even with a disconnected
vehicle).
The metric s.v3.connected is maintained by the ServerV3 code. When a
successful MQTT connection is made, and login successful, that is set
true (yes). A MQTT last-will-testament is set so that if the OVMS
module disconnects the server will automatically update that to false
(no). The ‘retained’ flag is set on this metric, at QOS level 0 (so
subscribing clients will get a copy of the last known state). Clients
can use this to see if the vehicle is connected to the server.
Connecting clients are expected to write a “1” value to the
<prefix>/c/<clientid> topic, and to repeat that write once a minute.
They are also expected to use a last-will-testament on that same
topic with value “0”. QOS 1 should be used, and these topics should
not be retained. The server V3 code subscribes to this <prefix>/c/#
topic, so it gets informed of all connected clients. It can then
update the s.v3.peers metric appropriately. Clients are expected to
monitor the <prefix>/s/v3/connected topic, so that if it becomes
‘yes’ (true) the client should send <prefix>/c/<clientid> “1”
immediately. This mechanism allows a newly connected vehicle to
immediately know if one or more clients is connected.
The Server v3 sets a timeout for <prefix>/c/<clientid> connections of
2 minutes. If that mqtt topic is not sent again within that time, it
is expired (and that client is treated as disconnected).
Similar to the v2 code, the server v3 transmits modified metrics once
a minute if there are one or more clients connected, or once every
ten minutes if there are no clients connected.
All the above has been implemented. To reach parity with v2, and
exceed it’s functionality in places, we have a few more things to do:
Notifications
On the vehicle side, I am proposing to use the <prefix>/notify/<type>
namespace for these, using QOS 2 messages without retention. Clients
can listen to those, if necessary. We can also have a central daemon
running that listens to ovms/+/+/n/# topic pattern to receive these
notifications and handle appropriately. Using QOS 2 we can confirm
the reception of the notification / historical data, and mark it as
delivered, appropriately. However, that would only confirm delivery
to the MQTT server, not to the central daemon; if the daemon was not
running, the notification would be lost.
Textual Commands
I am proposing to use the <prefix>/cmd/<clientid>/c/ and
<prefix>/cmd/<clientid>/r/ namespaces for this, using QOS 2 messages
without retention. The value in the /c/ would be the command, and the
response would be written to matching the /r/ topic. The commands and
responses could be prefixed by an identifier (like in imap protocol)
so responses can be matched to commands by the clients). The client
side can simply subscribe to itself, and the vehicle side subscribes
to <prefix>/cmd/#. In this way, commands cannot be duplicated, and
clients don’t see responses to commands they didn’t initiate (which
was an issue with v2).
Numeric Commands
I am not sure if we need to implement the numeric commands, as used
in the v2 protocol. It seems to me that we can use textual commands.
Config Access
I am not sure if we need this, beyond the command processor. If we
do, we could expose a /config/ namespace.
Events
It would be nice to expose events (except for the ticker.* ones, of
course). This could be done by a <prefix>/events topic, using QOS 2
and no retention.
Logs
It would be nice to expose logs. This could be done by a
<prefix>/logs topic, using QOS 1 and no retention.
Security
We need to add SSL support. I am trying to get an authentication
plugin for mosquitto / vernemq written so that we can authenticate
straight from the OVMS database that is already running on the
servers, and give each user their own ovms/<userid>/# namespace. That
way, the configuration for v3 on the vehicle/apps/clients is simple -
just put in the server username and password (no separate vehicle
passwords necessary).
I think that is it. The above would form the basis of the
specification for this. As this is the basis for future work and
direction in OVMS, it is important that we get it right, so all
comments / suggestions most welcome.
Regards, Mark.
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.comhttp://lists.openvehicles.com/mailman/listinfo/ovmsdev