The metrics subscription scheme is an option, and the auto-subscribe
feature via a getter is a nice idea. But I wouldn't apply that to
metrics value conversions and units.
Also we would still need the current set of metrics to be subscribed
by default, as there are also non Javascript devices (e.g. smart
buttons, Wifi displays) reading the WebSocket stream.
My thoughts on this so far:
Basically the web UI, as any frontend, should adapt to unit
configurations seamlessly. The web UI includes many command outputs,
which already automatically switch units as configured.
For all practical purposes, the web UI needs to interact with users
in their preferred units. Only some plugins and functions will need
certain values in metric (native) units for calculations, and these
will also need a simple way to convert calculation results back to
user units for displaying. So I think we need to provide the unit
configuration and value conversion tools in the web framework as
well.
Proposal:
a) We provide a config option and a WebSocket command to switch the
WebSocket metrics transmission mode to user / native units. To keep
plugin compatibility, the default is 'native'.
b) We introduce a separate units dictionary object containing the
user units for both metrics and the unit groups in their code &
label representation. The units dictionary only needs to be sent
initially, when new metrics are registered, when the metrics mode is
changed for the current connection, and when some user unit
configuration is changed, keeping the bandwidth and processing
requirements low.
The units dictionary can combine both metrics and group units, as
the unit group names are fully distinct from the metrics namespace.
The transport scheme could be:
{ units: {
"v.p.speed": { code:
"kmph", label: "km/h" }, …
"units.distance": { code: "miles", label:
"M" }, …
}
c) In the web framework, accessing units should be
as simple as possible and avoid throwing exceptions for undefined
entries, so we could e.g. split these into separate code & label
objects:
units["v.p.speed"] = "km/h" // consistently
accompanies metrics["v.p.speed"]
unitcodes["v.p.speed"]
= "kmph"
units["units.distance"]
= "M"
unitcodes["units.distance"] = "miles"
…or provide a getter that tests for the key existence and returns an
object with empty fields as necessary.
With this, all metrics displays can easily be changed to display the
actual unit labels instead of using fixed strings.
d) To provide value conversion we implement UnitConvert() in
Javascript plus some wrappers that automatically look up the unit
for a given metrics/group name and do the conversion to/from native
units, something like…
var speed_kph =
toNativeValue("v.p.speed"); // optional second arg to convert
any data
var speed_kph =
metrics_native["v.p.speed"]; // using a getter
var trip_display = toUserValue("units.distance",
1234);
Plugins for scientific/technical applications that depend on native
(metric) units can use the new metrics transmission mode control
command to force native mode. Or they can choose to migrate from
"metrics[]" to "metrics_native[]".
The metrics mode config option can come with a note informing users
that there may be some old plugins not compatible with non-native
units. They can then check their plugins for this and make an
informed decision on wether to enable user units and/or wether to
install a specific plugin.
Thoughts, comments?
Regards,
Michael
Am 25.11.22 um 03:13 schrieb Michael
Geddes:
I have an idea which might reduce traffic for
maintaining the metrics[] array in the browser and cope with
the user units.
I'll start by saying I'm not a JS developer per
se.. so a newb in JS really. Still, it's mainly just another
language so .. we'll give it a go.
Firstly:
* Implement the 'changed' filters as below for the
web-socket.. for both normal and 'user' values.
* Add a function that subscribes to a value (and returns
the current value of it)..including to 'user'
value/unitlabel.
Subscribing the normal way to the metrics over the
websocket would have the normal effect.. but we would have a
new way that would subscribe in a filtered way.
I've had a little play with the Proxy object .. so at
least I know this should work:
Have a metrics_ array that is the real associative array
for metrics[] and then define a Proxy that has (at the
least) 'get' and 'has' defined (giving us the ability to
overload metrics['prop'] and "prop" in metrics
operations).
The get function would return the underlying
value if it exists in the metrics_ array (which is
maintained through the websocket from currently subscribed
values in the current manner).
If the value is not in the metrics_ array - it
would then do a subscribe+query on the websocket getting the
current value and adding it into the
metrics_ container. If it was unavailable then it
would put undefined into the array.
The 'has' would do the get() and return true if the value
was not == undefined.
For the 'query the websocket' bit, I'm assuming I would
be working with promises or futures or some such: I'll do
the research and do it properly unless somebody can help
me out with it. That's the bit I was going to work on next
for the proof-of-concept.
Any immediate thoughts? Dangers?
I also noticed there was a bit that went through html
element properties and looked for metrics .. this could be
used to bulk subscribe to any metric values required
there.
//.ichael
Yeah, ok.
I will get all the other 'user unit' stuff done as a
line in the sand, and then move to working out the web
stuff. I'm still finding my way though all the client
side javascript, which looks very cool.. but I've not
really done jQuery before (just enough to recognise it).
Subscribing to metrics with/without user units makes
a lot of sense. Obviously the default needs to be
'Subscribe to all metrics but not user units' to
maintain compatibility... but I was also thinking it
might be nice if we could filter down even the normal
subscribed events.
We could have:
* Web socket command to filter units (flag on
websocket to say 'filtered' + flag bitset on each
metric similar to 'dirty')
Then either:
* Web socket command to turn on user units (single
flag on that websocket)
or
* Web socket command to turn on user units for
specific metrics (flag bitset on each metric)
A parameter to the URI for the websocket could start
the socket in 'filtered' mode to avoid the initial rush
of metrics.
This could drastically reduce traffic and time for
the metrics command to execute. It would be possible to
also check (on a 'filtered' websocket) for any changes
to metrics for that websocket slot before queueing the
'metric update' socket command.
//.ichael
Michael,
I don't have much spare time currently, just some
quick first comments: it's important to implement
this as lightweight as possible, both in terms of
network load and client CPU & memory
requirements. Some devices already have issues,
which can be seen by the "websocket overflow"
messages. The web UI also should stay usable via
cellular.
My impression is the new scheme, while only slightly
raising the client requirements, adds substantially
to the network requirements.
An option could be to separate the units -- or more,
back when implementing this I thought about
separating the names later on. Another question is
if we normally generally need both the native and
the converted values in the web UI. We maybe could
provide an option to switch to converted values, or
add an option to retreive or subscribe to a set of
converted metrics on demand.
Standard plugins like ABRP and PwrMon rely on
getting metric (native) units, and there probably
are non-public plugins, e.g. for engineering &
scientific projects, that depend on metric units to
do their calculations and don't need anything else.
We shouldn't make life harder for these applications
without good reason.
Regards,
Michael
Am 15.11.22 um 01:26 schrieb Michael Geddes:
If you're ok with the [default]
option I'll stick with that. I mean in some
ways it would be nice to have a button choice
metric | usa | europe | asia |
custom etc and I kind of considered
something like that but figured it's only a
handful of choices.. and it's an embedded
device.. so simpler is better.
On a related note - I was thinking how it
would be nice if the dashboard (etc) had
access to the 'user' units, so went hunting
down that little rabbit hole. Quite a nice
mechanism with the web socket updating the
"metrics" object in the UI.
This is a snippet of one idea, which is
that for any metric that has the possibility
of a user unit, we set the extra values of the
metric with '#unit' and '#user' appended - see
below. (I've chosen '#' arbitrarily.. but it
could be '/' or ':' or '>' but maybe not
'.' )
v.p.odometer#unit: "M"
v.p.odometer#user: 6754.91
v.p.satcount: 13
v.p.speed: 0
v.p.speed#unit: "km/h"
v.p.speed#user: null
v.p.trip: 28
v.p.trip#unit: "M"
v.p.trip#user: 17.3984
Then we can use this in the dials to
populate the values and captions! (not that I
like Miles).
I
The other (similar) way was to have
something like the following:
"v.p.trip#user" : { "value": 17.3984,
"unit": "M" }
It wouldn't make the total message any
shorter.. soo.. dunno.
There's also some complications with
setting up the dials (for min/max values) -
like for the speed.
Notice also that I'm returning null for
undefined values. It's nice - but I'm not
sure how javascript handles null when used /
printed etc.
//.ichael
Michael,
looks good.
I think having an explicit 'default'
option is better than taking the
'Metric' equivalent for that, as in
your example you already show unit
alternatives within the metric system
to support different scalings (kW / W,
kWh / Wh). (Btw… waiting for someone
to miss Horsepower & BTU here ;-))
@Patrick, I think that also answers
your implicit question:
The default button makes it unclear what the actual setting is.
The default (native unit) is always
metric, but you may have a mix of
scalings, as we try to find the one
that fits best for the given
application when defining a metric.
For example the current driving energy
consumption is stored natively in
Wh/km, while the energy used or
regenerated is in kWh, and the
odometer & trip counters are in
km, while the altitude ist in m.
Regards,
Michael
Am 13.11.22 um 08:42 schrieb
Michael Geddes:
Greetings,
so this is my idea of being
able to select which units
various groups use (in addition
to Distance).
This can be then accessed by
the special 'user' unit code.
(or 'metrics list -u ' )
The idea of [Default]
selection below simply means
storing the value to blank -
meaning use whatever unit the
particular metric uses. The
other idea I had was to actually
default it to the equivalent of
'Metric' special unit code and
not have the [Default] button.
Currently I've made it so
that if there are more than 3
choices other than [default]
that it uses the choice/combo
box rather than the Radio
buttons. (ie this list is
auto-generated from the Metric
Units table and the Metric
Groups table).
Thoughts / comments?
//.ichael
https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/771
I'm hoping this P/R is
ok in this form (made of 5
separate commits).
I will have a look at
implementing the "user"
unit code. The base for
how it would work is
already a part of the
above pull request. I'll
just look at the module
configuration for
distance.
The 'power consumption'
is one where it's not just
a check-box.. there're 5
possible choice!
I should also add 'bar'
for pressure given that
for some reason that's
still a thing people want.
//.ichael
I think this is
pretty decent &
complete now.
I also like the approach
of the 'user' unit code.
Moving all user unit
prefs into the module
configuration is an old
todo. Currently only the
distance unit is defined
at the module side,
temperature and pressure
are App prefs.
Regards,
Michael
Am 11.11.22 um
09:54 schrieb Michael
Geddes:
Ok - so
here's what I have
implemented for
Duktape and Metrics.
(I added IsDefined()
as well).
Any thoughts on
this?
Noting
OvmsMetrics.Float(
{metric} )
-> Outputs
metric as
float (same)
OvmsMetrics.Float(
{metric},
{unit}) ->
Outputs metric
as float
converted to
given unit
(new)
OvmsMetrics.Value(
{metric} )
-> Outputs
Metric in
native value
(same)
OvmsMetrics.Value(
{metric} ,
false) ->
Outputs Metric
as string and
no units
(same)
OvmsMetrics.Value(
{metric} ,
{unit}) ->
Outputs Metric
converted to
given unit as
native value.
(new)
OvmsMetrics.Value(
{metric} ,
{unit}, false
) ->
Outputs Metric
converted to
given unit as
string
including any
unit
specifier.
(new)
also
OvmsMetric.GetValues(
{metric}
[,{unit}] [,
{converted} ]
) Adds
similar
behaviour to
Value() above.
also the
special units
'imperial'
and 'metric'
will convert
to the
associated
imperial /
metric version
of the units
as
appropriate.
(function()
{
dump =
function
(metric) {
print( metric+
" ["+(typeof
metric)+"]\n"
); }
dump_obj =
function (obj
) {
print('---
Object
----\n')
for (var
k in obj) {
xk =
obj[k];
print(
k+':'+ xk + '
['+typeof xk+
"]\n");
}
}
dump(OvmsMetrics.Value("xiq.v.trip.consumption"));
dump(OvmsMetrics.Value("xiq.v.trip.consumption",
false));
dump(OvmsMetrics.Value("xiq.v.trip.consumption","kmpkwh"));
dump(OvmsMetrics.Value("xiq.v.trip.consumption",
"mipkwh",
false));
dump(OvmsMetrics.AsFloat("xiq.v.trip.consumption"));
dump(OvmsMetrics.AsFloat("xiq.v.trip.consumption","kmpkwh"));
dump(OvmsMetrics.Value("xiq.v.trip.consumption","imperial"))
dump(OvmsMetrics.Value("xiq.v.trip.consumption","imperial",
false))
dump_obj(OvmsMetrics.GetValues("trip",
"metric"))
dump_obj(OvmsMetrics.GetValues("trip",
"imperial",
false))
})();
With this
output:
19.2308 [number]
19.2308 [string]
5.2 [number]
3.23112mi/kWh [string]
19.2308 [number]
5.2 [number]
309.49 [number]
309.49Wh/mi [string]
--- Object ----
v.p.trip:13 [number]
xiq.e.trip:0 [number]
xiq.e.trip.energy.recuperated:0 [number]
xiq.e.trip.energy.used:0 [number]
xiq.v.trip.consumption:19.2308 [number]
--- Object ----
v.p.trip:8.07781M [string]
xiq.e.trip:0M [string]
xiq.e.trip.energy.recuperated:0kWh [string]
xiq.e.trip.energy.used:0kWh [string]
xiq.v.trip.consumption:309.49Wh/mi [string]
Yeah
- I like
HasValue. I
implemented
IsDefined() but
I will rename
it.. that's a
much clearer
name.
Another
thought. How
about if we
did this (but
also with
GetValues() as
well - see the
special values
below)
OvmsMetrics.Value("xiq.v.trip.consumption",
true) -> 17.0582 (Number)
OvmsMetrics.Value("xiq.v.trip.consumption",
false)
-> 17.0582 (String)
OvmsMetrics.Value("xiq.v.trip.consumption",
"mipkwh",
true) -> 3.64264 (Number)
OvmsMetrics.Value("xiq.v.trip.consumption",
"mipkwh",
false) -> 3.64264Mi/kWh (String)
OvmsMetrics.Value("xiq.v.trip.consumption",
"native",
false) -> 17.0582km/kWh (String)
and
OvmsMetrics.Value("xiq.v.trip.consumption",
"imperial",
false) -> 3.64264Mi/kWh (String)
I have
already
implemented
the special
values
'native'
(existing),
'imperial' and
'metric'.
I was also
thinking that
in the future
you could have
'user'. Where
for each group
of values:
'temperature',
'distance',
'shortdistance',
'power' etc..
you could have
a user
preference. I
probably won't
implement it
now,.but it
could be cool
that any UI
could just ask
for the user
defined units
(rather than
having a
separate
choice).
//.ichael
Or perhaps something more specific?
HasValue()
Mark
Signed
PGP part
That's
basically a
good approach,
but be aware
'IsDefined()'
has an
ambiguous
meaning here,
as with the
API stem
"OvmsMetrics"
it would
naturally be
expected to
mean "is this
metric
defined", not
"does this
metric have a
defined
value".
An undefined
metric
currently can
be derived
from
'Values()'
returning
undefined, but
that's more an
undocumented
side effect
than intended.
Maybe
'GetDefined()'
could be a
better name,
leveraging
this
behaviour,
i.e. returning
'undefined'
for an
actually
undefined
metric, and
'null' for a
defined metric
without a
value.
Regards,
Michael
Am
08.11.22 um
13:46 schrieb
Michael
Geddes:
Ah yes.
Arrays - will
check those.
Yeah, how
about we add a
'IsDefined'
method to
metrics
instead of the
null thing (it
does sound
like it will
upset too many
applecarts).
//.
Michael,
looks all good
to me, once
again nice
find with the
decode
argument.
Adding decode
to the Value()
call was only
for symmetry
IIRC, the main
use was with
GetValues() (
https://docs.openvehicles.com/en/latest/userguide/scripting.html#ovmsmetrics).
Don't forget
to test
arrays, e.g.
"v.t.pressure"
&
"v.t.temp".
Returning null
for an
undefined
metric seems
like a natural
choice, but is
a rather deep
change, as for
consistency
not only the
Duktape
metrics API
but also the
Web UI metrics
API would need
to be changed
accordingly.
Unless you've
got a real use
case that
needs that, we
should be
careful.
Regards,
Michael
Am
07.11.22 um
15:00 schrieb
Michael
Geddes:
I
have figured
out a bunch of
stuff and have
implemented
the following:
(having done
away with
needing
AsFloatUnit)
OvmsMetrics.Value(
{metric} [,
{decode}])
OvmsMetrics.Value(
{metric},
{unit}
[,{decode}])
It turns
out that the
[decode] flag
wasn't working
anyway (since
the function
was being
registered as
only having 1
param)...
This way
it is still
really 1
function.. but
I check it the
second
parameter is a
'boolean', and
if not.. try
the second
form.
OvmsMetrics.AsFloat(
{metric}
[,{unit}] )
and add
the function
Ovms.Metrics.ValueUnit(
{metric}
[,{unit}])
This
prints the
value and the
unit.
Here's a
sample
function and
the output!
This also
shows the
types of the
output.
(function()
{
x =
OvmsMetrics.Value("xiq.v.trip.consumption");
print(
(typeof x) +
": "+ x+"\n"
);
x =
OvmsMetrics.Value("xiq.v.trip.consumption",
false);
print(
(typeof x) +
": "+ x +"\n"
);
x =
OvmsMetrics.Value("xiq.v.trip.consumption","kmpkwh")
print(
(typeof x) +
": "+ x
+"\n");
x =
OvmsMetrics.Value("xiq.v.trip.consumption",
"mipkwh",
false)
print(
(typeof x) +
": "+ x
+"\n");
x =
OvmsMetrics.ValueUnit("xiq.v.trip.consumption")
print(
(typeof x) +
": "+ x
+"\n");
x =
OvmsMetrics.ValueUnit("xiq.v.trip.consumption","mipkwh")
print(
(typeof x) +
": "+ x
+"\n");
x =
OvmsMetrics.AsFloat("xiq.v.trip.consumption")
print(
(typeof x) +
": "+ x
+"\n");
x =
OvmsMetrics.AsFloat("xiq.v.trip.consumption","kmpkwh")
print(
(typeof x) +
": "+ x
+"\n");
})();
number: 17.0582
string: 17.0582
number: 5.86227
string: 3.64264
string: 17.0582kWh/100km
string: 3.64264mi/kWh
number: 17.0582
number: 5.86227
It still
might be an
idea to use
'null' as a
return value
if the metrics
is
!IsDefined() but
that would be
changing the
existing
behaviour
slightly.
//.ichael
I've
worked out
what the
decode flag is
for and how it
works, and I
think how
optional
params work.
I'm
pretty sure I
won't need
the
'AsFloatUnit'
function; the
unit would be
an option to
AsFloat();
I'll know that
soon.
The
'Value'
function is
more
complicated
because of the
optional
decode bool. I
guess I could
add the Unit
to the end of
that.
ValueUnit
could be still
useful then to
provide a
'Value +
Unit'.
Question:
Is there a
reason we
shouldn't be
returning
with duk_push_null
if the
metric
!IsDefined()
in both
AsFloat() and
Value(metric,true) cases?
//.ichael
Right, so
I've
implemented
some stuff
that seems to
work quite
well.
The
commit that
will follow on
from that it
implements the
new Units:
kWh/100km,
km/kWh and
mi/kWh.
This is a
summary of
what I've
implemented
for scripting
- including
showing the
unit codes I
have so far.
I've
considered a
few things:
*
Should some of
the longer
unit codes be
shortened (eg
mi, mins, m,
ft, deg, perc)
* The
unit codes
could be much
more regular
and separated
by dots eg:
watthours
-> w.h
kwhp100km
->
kw.h_100km or
kw.h/100km
miph
-> mi_h or
mi/h (or
should it be
mph).
OVMS#
metric units
km :
km
miles : M
meters : m
feet :
ft
celcius :
°C
fahrenheit :
°F
kpa :
kPa
pa :
Pa
psi :
psi
volts : V
amps : A
amphours
: Ah
kw :
kW
kwh :
kWh
watts : W
watthours
: Wh
seconds :
Sec
minutes :
Min
hours :
Hour
utc :
UTC
degrees : °
kmph :
km/h
miph
: Mph
kmphps
: km/h/s
miphps
: Mph/s
mpss :
m/s²
dbm :
dBm
sq :
sq
percent : %
whpkm :
Wh/km
whpmi
: Wh/mi
kwhp100km :
kWh/100km
kmpkwh
: km/kWh
mipkwh
: mi/kWh
nm :
Nm
OVMS#
metric unit mi
miles :
M