<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Michael,<br>
<br>
"data" notifications correspond to the V2/MP "historical" message
type:<br>
<ul>
<li><a class="moz-txt-link-freetext" href="https://docs.openvehicles.com/en/latest/protocol_v2/messages.html#historical-data-update-message-0x48-h">https://docs.openvehicles.com/en/latest/protocol_v2/messages.html#historical-data-update-message-0x48-h</a></li>
</ul>
<p>So a record type of "*-LOG-Poll" would be OK, but I suggest using
"*-LOG-PollStats" to be more precise.</p>
<p>The record ID needs to be an integer, and the default V2 server
database defines this to be a 32 bit signed integer. Note that
sending a new record won't overwrite an existing one with the same
ID, as the timestamp is part of the primary key. I suggest using
your report line number.</p>
<p>You can provide a header as line 0 then, or you can leave adding
a header to the download tool (as do all the other data records up
to now). If you use my server, you can download all data from the
car UI, if you use another public server, you can still use my
download tool via this page:</p>
<ul>
<li><a class="moz-txt-link-freetext" href="https://dexters-web.de/downloadtool?lang=EN">https://dexters-web.de/downloadtool?lang=EN</a><br>
</li>
</ul>
<p>Download tools other than the ones I provide in my web UI are the
scripts in the server repository's client directory:</p>
<ul>
<li><a class="moz-txt-link-freetext" href="https://github.com/openvehicles/Open-Vehicle-Server/tree/master/clients">https://github.com/openvehicles/Open-Vehicle-Server/tree/master/clients</a></li>
</ul>
<p>The most simple form is shown by the "serverlog.sh" script, for
adding headers see e.g. "rt_fetchlogs.sh". I normally let my
server send me the logs by mail on a daily base, these include all
historical data files with headers added.</p>
Assuming record type "<font face="monospace">*-LOG-PollStats</font>",
I've just added auto headers to my tool based on your template as
follows:<br>
<ul>
<li><font face="monospace">type,count_hz,avg_util_pm,peak_util_pm,avg_time_ms,peak_time_ms</font></li>
</ul>
(keeping the header style consistent with the other logs)<br>
<br>
So you can now simply send the data rows, the tool will prepend the
header once on each download.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 19.06.24 um 11:10 schrieb Michael
Geddes:<br>
</div>
<blockquote type="cite"
cite="mid:CAH0p7uKjMJ5q-4WVzubGG1Zjhuijoy0W2+j-X1LFTmrfrPziVw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="auto">
<div>Sure, I can do that. </div>
<div>I did it this way because it was easier and could mostly
do it in one message. As soon as I added spaces for
alignment it pushed the message over 2 notifications. Also,
I wasn't sure about making a new Data type and how that
worked.</div>
<div><br>
</div>
<div>I'm assuming something like: *-LOG-Poll would work
(unless you want to suggest something else) The next 2 cols
seem to be ID and lifetime.</div>
<div>What do I use as an ID? Can it be an alpha string or
does it have to be a number? (LIke using the first column
descriptor). I'm not sure</div>
<div>how the ID column is treated. I _could_ just send a line
number for the dump group I guess?</div>
<div><br>
</div>
<div>These are the columns I have. I can force the two cols to
be always permille.</div>
<div>"Type","Count (hz)","Avg utlization (permille)","Peak
utlization (permille)", "Avg Time (ms)","Peak Time (ms)"<br>
</div>
<div><br>
</div>
<div>Type is the only alpha column.. but if I can use that for
the ID I guess that would be better?</div>
<div><br>
</div>
<div>How would I provide a header if I wanted to? Is there
some indicator saying it's a header line? I'm not sure I
want to - just asking.</div>
<div><br>
</div>
<div>//.ichael</div>
<div><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, 19 June 2024,
00:11 Michael Balzer via OvmsDev, <<a
href="mailto:ovmsdev@lists.openvehicles.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div> Michael,<br>
<br>
I've received the new drive report notifications now;
these need to be changed: please do not use text
notifications to transport CSV data. That's what
"data" notifications are meant for, which are stored
in their raw form on the server for later retrieval.
See e.g. the vehicle trip & grid logs for
reference on how to build the messages, or have a look
at the specific Twizy and UpMiiGo data messages.<br>
<br>
Data notifications also are designed to transport one
row at a time, so you normally don't run into buffer
size issues. A header can be supplied as a row, but
you normally add one when downloading the data from
the server, so tools don't need to filter these out. I
provide headers automatically on my server for known
message types, just send me your template and I'll
include that.<br>
<br>
Apart from that, the timing statistics now seem to
work pretty well, providing valuable insights.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<br>
<div>Am 14.06.24 um 08:59 schrieb Michael Geddes:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thankyou.
<div>I was more worried that we might be waiting
on each other.</div>
<div><br>
</div>
<div>I don't think I have quite the correct able
to test on my friends Leaf properly, or does it
use the standard cable?</div>
<div><br>
</div>
<div>Anyway, let me know what I can do to.</div>
<div><br>
</div>
<div>//.ichael</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, 14 Jun
2024 at 14:46, Michael Balzer via OvmsDev <<a
href="mailto:ovmsdev@lists.openvehicles.com"
rel="noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Sorry, I know I'm behind with PRs.<br>
<br>
I'll try to find some time this weekend.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div>Am 14.06.24 um 08:31 schrieb Michael
Geddes via OvmsDev:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Was this all good? I want to make
sure I get to the bottom of this whole
issue asap! </div>
<div><br>
</div>
<a
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/1018"
rel="noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/1018</a>
<div><br>
</div>
<div>Was there something else you needed
me to work on to make sure this all
works for all supported cars?<br>
<div><br>
</div>
<div>//.ichael</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun,
26 May 2024 at 21:15, Michael Balzer via
OvmsDev <<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> OK, I see now why it wouldn't send
the notification: the V2 & V3
server register for notifications up
to COMMAND_RESULT_NORMAL = 1024
characters.<br>
<br>
The report quickly becomes larger than
1024 characters, so the notifications
no longer get sent via the server
connectors.<br>
<br>
You need to either reduce the size,
split the report, or use data
notifications instead.<br>
<br>
On the reset value init: for my float
targeting smoothing helper class for
the UpMiiGo, I implemented a gradual
ramp up from 1 to the requested sample
size. You can do something similar
also with powers of 2. IOW, yes,
initialization from the first values
received is perfectly OK.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div>Am 26.05.24 um 14:35 schrieb
Michael Geddes:<br>
</div>
<blockquote type="cite">
<div>
<div>
<div>
<p class="MsoNormal"
style="margin-bottom:12pt">It
_<i>should</i>_ already be
sending a report on charge
stop.</p>
<p class="MsoNormal"
style="margin-bottom:12pt">MyEvents.RegisterEvent(TAG,
"vehicle.charge.stop",
std::bind(&OvmsPollers::VehicleChargeStop,
this, _1, _2));</p>
<p class="MsoNormal"
style="margin-bottom:12pt">Reset
on charge start/vehicle on
is a good idea.</p>
<p class="MsoNormal"
style="margin-bottom:12pt">A
question – would it be silly
if the first value after a
reset, rather than using 0
average to start with, if
the average got set to the
initial value? I’m in 2
minds about it. It would
make the average more useful
quicker.</p>
<p class="MsoNormal"
style="margin-bottom:12pt">//.ichael</p>
<div>
<div>
<p class="MsoNormal">On
Sun, 26 May 2024, 19:39
Michael Balzer via
OvmsDev, <<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<p class="MsoNormal"
style="margin-bottom:12pt">As the averages quickly decline when idle, an
automatic report
should probably also
be sent on charge
stop.<br>
<br>
Also, I think you
should automatically
reset the timer
statistics on drive
& charge start.<br>
<br>
First stats from
charging my UpMiiGo:<br>
<br>
<span
style="font-family:"Courier New"">Type | count |
Utlztn | Time <br>
| per
s | [‰] | [ms]<br>
---------------+--------+--------+---------<br>
Poll:PRI Avg|
1.00| 0.723|
0.716<br>
Peak| |
1.282| 3.822<br>
---------------+--------+--------+---------<br>
Poll:SRX Avg|
7.72| 1.246|
0.184<br>
Peak| |
3.128| 1.058<br>
---------------+--------+--------+---------<br>
RxCan1[7ae] Avg|
2.48| 0.915|
0.362<br>
Peak| |
1.217| 1.661<br>
---------------+--------+--------+---------<br>
RxCan1[7cf] Avg|
4.76| 1.928|
0.397<br>
Peak| |
2.317| 2.687<br>
---------------+--------+--------+---------<br>
RxCan1[7ed] Avg|
3.38| 1.251|
0.327<br>
Peak| |
8.154| 12.273<br>
---------------+--------+--------+---------<br>
RxCan1[7ee] Avg|
0.21| 0.066|
0.297<br>
Peak| |
0.225| 1.690<br>
---------------+--------+--------+---------<br>
TxCan1[744] Avg|
1.49| 0.022|
0.011<br>
Peak| |
0.032| 0.095<br>
---------------+--------+--------+---------<br>
TxCan1[765] Avg|
3.89| 0.134|
0.027<br>
Peak| |
0.155| 0.113<br>
---------------+--------+--------+---------<br>
TxCan1[7e5] Avg|
2.32| 0.038|
0.013<br>
Peak| |
0.295| 0.084<br>
---------------+--------+--------+---------<br>
TxCan1[7e6] Avg|
1 0.21| 0.002|
0.008<br>
Peak| |
0.010| 0.041<br>
---------------+--------+--------+---------<br>
Cmd:State Avg|
0.00| 0.000|
0.007<br>
Peak| |
0.005| 0.072<br>
===============+========+========+=========<br>
Total Avg|
27.46| 6.324|
2.349<br>
</span><br>
<br>
Overall healthy I'd
say, but let's see how
it compares.<br>
<br>
7ed is the BMS, the
peak time is probably
related to the
extended cell data
logging -- I've
enabled log intervals
for both cell voltages
& temperatures.<br>
<br>
Regards,<br>
Michael<br>
<br>
</p>
<div>
<p class="MsoNormal">Am
26.05.24 um 08:42
schrieb Michael
Balzer via OvmsDev:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<p class="MsoNormal">The
notification works
on my devices, it
only has a garbled
per mille character
-- see attached
screenshot. The same
applies to the mail
version:<br>
<br>
<br>
</p>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<pre>Poller timing is: on</pre>
<pre>Type | count | Utlztn | Time </pre>
<pre> | per s | [‰] | [ms]</pre>
<pre>---------------+--------+--------+---------</pre>
<pre>Poll:PRI Avg| 0.25| 0.119| 0.382</pre>
<pre> Peak| | 0.513| 0.678</pre>
<pre>---------------+--------+--------+---------</pre>
<pre>RxCan1[597] Avg| 0.01| 0.004| 0.021</pre>
<pre> Peak| | 0.000| 0.338</pre>
<pre>---------------+--------+--------+---------</pre>
<pre>RxCan1[59b] Avg| 0.01| 0.011| 0.053</pre>
<pre> Peak| | 0.000| 0.848</pre>
<pre>---------------+--------+--------+---------</pre>
<pre>Cmd:State Avg| 0.01| 0.002| 0.012</pre>
<pre> Peak| | 0.000| 0.120</pre>
<pre>===============+========+========+=========</pre>
<pre> Total Avg| 0.28| 0.135| 0.468</pre>
</blockquote>
<p class="MsoNormal"
style="margin-bottom:12pt"><br>
The encoding is a
general issue. The
character encoding
for text messages
via V2/MP is quite
old & clumsy,
it's got an issue
with the degree
celcius character as
well. We previously
tried to keep all
text messages within
the SMS safe
character set (which
e.g. lead to writing
just "C" instead of
"°C"). I'd say we
should head towards
UTF-8 now. If we
ever refit SMS
support, we can
recode on the fly.<br>
<br>
Regarding not seeing
the notification on
your phone:<br>
<br>
a) Check your
notification
subtype/channel
filters on the
module. See <a
href="https://docs.openvehicles.com/en/latest/userguide/notifications.html#suppress-notifications"
rel="noreferrer"
target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">https://docs.openvehicles.com/en/latest/userguide/notifications.html#suppress-notifications</a><br>
<br>
b) Check your
notification vehicle
filters on the phone
(menu on
notification tab):
if you enabled the
vehicle filter, it
will add the
messages of not
currently selected
vehicles to the list
only, but not raise
a system
notification.
(Applies to the
Android App, no idea
about the iOS
version)<br>
<br>
Regards,<br>
Michael<br>
<br>
</p>
<div>
<p class="MsoNormal">Am
26.05.24 um 06:32
schrieb Michael
Geddes:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<div>
<div>
<p
class="MsoNormal">Hi,</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">I'm trying to finalise this now .. and one last thing
is that I
don't get the
report coming
to my mobile.
I'm using the
command:</p>
</div>
<div>
<p
class="MsoNormal"> <span style="font-family:"Courier New"">
MyNotify.NotifyString("info",
"poller.report", buf.c_str());</span></p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Where the buffer string is just the same as the report
output.
Should I be
using some
other format
or command?</p>
</div>
<div>
<p
class="MsoNormal">I get "alert" types (like the ioniq5 door-open alert)
fine to my
mobile.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Michael.</p>
</div>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<div>
<p
class="MsoNormal">On Sun, 19 May 2024, 12:51 Michael Balzer via OvmsDev,
<<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<p
class="MsoNormal" style="margin-bottom:12pt">A builtin web UI for this
seems a bit
over the top.
Builtin web
config pages
should focus
on user
features, this
is clearly a
feature only
needed
during/for the
development/extension of a vehicle adapter. Development features in the
web UI are
confusing for
end users.<br>
<br>
If persistent
enabling/disabling is done by a simple config command (e.g. "config set
can
poller.trace
on"), that's
also doable by
users.<br>
<br>
Regards,<br>
Michael<br>
<br>
</p>
<div>
<p
class="MsoNormal">Am 19.05.24 um 02:06 schrieb Michael Geddes:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<div>
<div>
<p
class="MsoNormal">I was so focused on how I calculated the value that I
totally
missed that ‰
would be a
better
description.
I could also
use the
system 'Ratio'
unit... so %
or ‰. </p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">I'll make space to put 'Avg' on the row. Was trying
to limit the
width for
output on a
mobile. I
agree it would
make it easier
to understand.</p>
</div>
<div>
<p
class="MsoNormal">Totals also makes sense.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Should I make this a configuration that can be set on
the web-page?
I'd probably
use a
configuration
change
notification
so that the
very bit
setting is
sync'd with
the
'configuration'
value.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">//.ichael</p>
</div>
<div>
<p
class="MsoNormal"> </p>
<div>
<div>
<p
class="MsoNormal">On Sat, 18 May 2024, 14:05 Michael Balzer, <<a
href="mailto:dexter@expeedo.de" rel="noreferrer" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">dexter@expeedo.de</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<p
class="MsoNormal">I'm not sure whether the 'max' should be the maximum
of the
smoothed
value.. or the
maximum of the
raw value.</p>
</div>
</blockquote>
<p
class="MsoNormal" style="margin-bottom:12pt"><br>
It should
normally be
the maximum of
the raw value
I think, the
maximum of the
smoothed value
cannot tell
about how bad
the processing
of an ID can
become.<br>
<br>
The naming in
the table is a
bit confusing
I think.
(besides: I've
never seen
"ave" as the
abbreviation
for average)<br>
<br>
If I
understand you
correctly,
"time ms per
s" is the time
share in per
mille, so
something in
that direction
would be more
clear, and
"length ms"
would then be
"time [ms]".<br>
<br>
The totals for
all averages
in the table
foot would
also be nice.<br>
<br>
Maybe "Ave"
(or avg?) also
should be
placed on the
left, as the
"peak" label
now suggests
being the peak
of the
average.<br>
<br>
Btw, keep in
mind, not all
"edge" users /
testers are
developers
(e.g. the
Twizy driver
I'm in contact
with),
collecting
stats feedback
for vehicles
from testers
should be
straight
forward. Maybe
add a
data/history
record, sent
automatically
on every
drive/charge
stop when the
poller tracing
is on?<br>
<br>
Regards,<br>
Michael<br>
<br>
</p>
<div>
<p
class="MsoNormal">Am 18.05.24 um 02:28 schrieb Michael Geddes:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<div>
<p
class="MsoNormal">You did say max/pead value. I also halved the N for
both.</p>
</div>
<div>
<p
class="MsoNormal">I'm not sure whether the 'max' should be the maximum
of the
smoothed
value.. or the
maximum of the
raw value.</p>
</div>
<div>
<p
class="MsoNormal">This is currently the raw-value maximum. </p>
</div>
<div>
<p
class="MsoNormal">The problem is that the middle column is the maximum
of the {{sum
over 10s} /
(10*1000,000)</p>
</div>
<div>
<p
class="MsoNormal">I could easily change the 'period' to 1s and see how
that goes..
was just
trying to
reduce the
larger
calculations. </p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<p
class="MsoNormal"><span style="font-family:"Courier New"">Usage:
poller
[pause|resume|status|times|trace]</span>
</p>
<div>
<p
class="MsoNormal"><span style="font-family:"Courier New""><br>
OVMS# poller
time status<br>
Poller timing
is: on<br>
Type |
Count | Ave
time | Ave
length<br>
|
per s | ms
per s | ms <br>
-------------+----------+-----------+-----------<br>
Poll:PRI |
1.00|
0.559|
0.543<br>
peak |
|
0.663|
1.528<br>
-------------+----------+-----------+-----------<br>
Poll:SRX |
0.08|
0.009|
0.038<br>
peak |
|
0.068|
0.146<br>
-------------+----------+-----------+-----------<br>
CAN1 RX[778] |
0.11|
0.061|
0.280<br>
peak |
|
0.458|
1.046<br>
-------------+----------+-----------+-----------<br>
CAN1 RX[7a8] |
0.04|
0.024|
0.124<br>
peak |
|
0.160|
0.615<br>
-------------+----------+-----------+-----------<br>
CAN1 TX[770] |
0.05|
0.004|
0.016<br>
peak |
|
0.022|
0.102<br>
-------------+----------+-----------+-----------<br>
CAN1 TX[7a0] |
0.02|
0.002|
0.011<br>
peak |
|
0.010|
0.098<br>
-------------+----------+-----------+-----------<br>
CAN1 TX[7b3] |
0.01|
0.001|
0.006<br>
peak |
|
0.000|
0.099<br>
-------------+----------+-----------+-----------<br>
CAN1 TX[7e2] |
0.02|
0.002|
0.011<br>
peak |
|
0.010|
0.099<br>
-------------+----------+-----------+-----------<br>
CAN1 TX[7e4] |
0.08|
0.008|
0.048<br>
peak |
|
0.049|
0.107<br>
-------------+----------+-----------+-----------<br>
Cmd:State |
0.00|
0.000|
0.005<br>
peak |
|
0.000|
0.094</span></p>
</div>
</div>
<p
class="MsoNormal"> </p>
<div>
<div>
<p
class="MsoNormal">On Fri, 17 May 2024 at 15:26, Michael Geddes <<a
href="mailto:frog@bunyip.wheelycreek.net" rel="noreferrer"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">frog@bunyip.wheelycreek.net</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<p
class="MsoNormal">This is what I have now.</p>
</div>
<div>
<p
class="MsoNormal">The one on the end is the one MIchael B was after
using an N of
32. (up for
discussion).</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">The middle is the time spent in that even t per
second. It
accumulates
times (in
microseconds),
and then every
10s it stores
it as
smoothed (N=16)
value.</p>
</div>
<div>
<p
class="MsoNormal">The Count is similar (except that we store a value of
'100' as 1
event so it
can be still
integers and
has 2 decimal
places).</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Every received poll does a 64bit difference to 32bit
(for the
elapsed time)
and 64bit
comparison
(for
end-of-period).</p>
</div>
<div>
<p
class="MsoNormal">It also does 1x 32bit smoothing and 2x 32bit adds.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Then at the end of a 10s period, it will do a 64bit
add to get the
next
end-of-period
value, as well
as the 2x
32bit
smoothing
calcs.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">This is from the Ioniq 5 so not any big values yet.
You can
certainly see
how
insignificant
the TX
callbacks are.</p>
</div>
<div>
<p
class="MsoNormal">I'll leave it on for when the car is moving and gets
some faster
polling.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"><span style="font-family:"Courier New"">OVMS#
poll time
status<br>
Poller timing
is: on<br>
Type |
Count | Ave
time | Ave
Length<br>
|
per s | ms
per s | ms <br>
-------------+----------+-----------+-----------<br>
Poll:PRI |
1.00|
0.540|
0.539<br>
Poll:SRX |
0.03|
0.004|
0.017<br>
CAN1 RX[778] |
0.06|
0.042|
0.175<br>
CAN1 TX[770] |
0.04|
0.002|
0.008<br>
Cmd:State |
0.01|
0.001|
0.005</span></p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"><span style="font-family:"Courier New"">----------------------8<--------------------------------</span></p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"><span style="font-family:"Courier New"">Nice
smoothing
class (forces
N as a power
of 2):</span></p>
</div>
<div>
<p
class="MsoNormal"> constexpr unsigned floorlog2(unsigned x)<br>
{<br>
return x
== 1 ? 0 :
1+floorlog2(x
>> 1);<br>
}<br>
/* Maintain
a smoothed
average using
shifts for
division.<br>
* T should
be an integer
type<br>
* N needs
to be a power
of 2<br>
*/<br>
template
<typename
T, unsigned
N><br>
class
average_util_t<br>
{<br>
private:<br>
T m_ave;<br>
public:<br>
average_util_t()
: m_ave(0) {}<br>
static
const uint8_t
_BITS =
floorlog2(N);<br>
void
add( T val)<br>
{<br>
static_assert(N
== (1 <<
_BITS), "N
must be a
power of 2");<br>
m_ave
= (((N-1) *
m_ave) + val)
>>
_BITS;<br>
}<br>
T get()
{ return
m_ave; }<br>
operator
T() { return
m_ave; }<br>
};</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
</div>
<p
class="MsoNormal"> </p>
<div>
<div>
<p
class="MsoNormal">On Thu, 16 May 2024 at 10:29, Michael Geddes <<a
href="mailto:frog@bunyip.wheelycreek.net" rel="noreferrer"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">frog@bunyip.wheelycreek.net</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p
class="MsoNormal">Thanks Michael,</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<p
class="MsoNormal">My calculations give me ((2^32)-1) / (1000*1000*3600)
= only 1.2
hours of
processing
time in
32bit. The
initial
subtraction is
64bit anyway
and I can't
see a
further 64bit
addition being
a problem. I
have the
calculations
being
performed in
doubles at
print-out
where
performance is
not really an
issue anyway.
(Though
apparently
doing 64 bit
division is
worse than
floating
point). </p>
<div>
<p
class="MsoNormal"> </p>
<div>
<p
class="MsoNormal">In addition</p>
</div>
<div>
<p
class="MsoNormal">* I currently have this being able to be turned on and
off and reset
manually (only
do it when
required).</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">* For the lower volume commands, the smoothed average
is not going
to be useful -
the count is
more
interesting
for different
reasons.</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">* The total time is quite useful. Ie a high average
time doesn't
matter if the
count is low.
The things
that are
affecting
performance
are stuff with
high total
time. Stuff
which is
happening 100
times a second
needs to be a
much lower
average than
once a second.</p>
</div>
<div>
<div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">* A measure like 'time per minute/second' and possibly
count per
minute/seconds
as a
smoothed average
would
potentially be
more useful.
(or in
addition?)</p>
</div>
<div>
<p
class="MsoNormal">I think we could do _that_ in a reasonably efficient
manner using a
64 bit 'last
measured
time', a 32
bit
accumulated
value and the
stored 32 bit
rolling
average. </p>
</div>
<div>
<p
class="MsoNormal">It would boils down to some iterative (integer) sums
and
multiplications
plus a divide
by n ^ (time
periods
passed) -
which is a
shift - and
which can be
optimised to
'0' if
'time-periods-passed'
is more than
32/(bits-per-n)
- effectively
limiting the
number of
iterations.</p>
</div>
<div>
<p
class="MsoNormal">The one issue I can see is that we need to calculate
'number of
time-periods
passed' which
is a 64 bit
subtraction
followed by a
32 bit
division (not
optimisable to
a simple
shift).</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
</div>
</div>
<div>
<p
class="MsoNormal">* I'm also happy to keep a rolling (32bit) average
time.</p>
</div>
<div>
<p
class="MsoNormal"> Even if you assume averages in the 100ms, 32bit is
going to
happily
support an N
of 64 or even
128.</p>
</div>
<div>
<div>
<div>
<p
class="MsoNormal"> Am I right in thinking that the choice of N is
highly
dependent on
frequency. For
things
happening 100
times per
second, you
might want an
N like 128..
where things
happening once
per</p>
</div>
<div>
<p
class="MsoNormal"> second, you might want an N of 4 or 8. The other
things we keep
track of in
this manner we
have a better
idea of the
frequency of
the thing.</p>
</div>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
</div>
<div>
<p
class="MsoNormal">How about we have (per record type):</p>
</div>
<div>
<p
class="MsoNormal"> * total count (since last reset?) (32 bit)</p>
</div>
<div>
<p
class="MsoNormal"> * smoothed average of time per instance (32 bit)</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"> * ?xx? total accumulated time since last reset
(64bit) ??
<-- with
the below
stats this is
much less
useful</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"> * last-measured-time (64 bit) </p>
</div>
<div>
<p
class="MsoNormal"> * accumulated count since last time-period (16bit -
but maybe
32bit anyway
for byte
alignment?)</p>
</div>
<div>
<p
class="MsoNormal"> * smoothed average of count per time-period (32bit)</p>
</div>
<div>
<p
class="MsoNormal"> * accumulated time since last time-period (32bit)</p>
</div>
<div>
<p
class="MsoNormal"> * smoothed average of time per time-period (32bit)</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">It's possible to keep the </p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Is this going to be too much per record type? The
number of
'records' we
are keeping is
quite low (so
10 to 20
maybe) - so
it's not a
huge memory
burden.</p>
</div>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">Thoughts?</p>
</div>
<div>
<p
class="MsoNormal"> </p>
</div>
</div>
<p
class="MsoNormal">//.ichael</p>
<div>
<div>
<p
class="MsoNormal">On Thu, 16 May 2024 at 03:09, Michael Balzer via
OvmsDev <<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<p
class="MsoNormal" style="margin-bottom:12pt">esp_timer_get_time() is the
right choice
for precision
timing.<br>
<br>
I'd say uint32
is enough
though, even
if counting
microseconds
that can hold
a total of
more than 71
hours of
actual
processing
time. uint64
has a
significant
performance
penalty,
although I
don't recall
the overhead
for simple
additions.<br>
<br>
Also &
more
important, the
average
wouldn't be my
main focus,
but the
maximum
processing
time seen per
ID, which
seems to be
missing in
your draft.<br>
<br>
Second thought
on the
average… the
exact overall
average really
has a minor
meaning, I'd
rather see the
current
average,
adapting to
the current
mode of
operation
(drive/charge/…).
I suggest
feeding the
measurements
to a low pass
filter to get
the smoothed
average of the
last n
measurements.
Pattern:<br>
<br>
runavg =
((N-1) *
runavg +
newval) / N<br>
<br>
By using a low
power of 2 for
N (e.g. 8 or
16), you can
replace the
division by a
simple bit
shift, and
have enough
headroom to
use 32 bit
integers.<br>
<br>
Regards,<br>
Michael<br>
<br>
</p>
<div>
<p
class="MsoNormal">Am 15.05.24 um 06:51 schrieb Michael Geddes via
OvmsDev:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<p
class="MsoNormal">Formatting aside, I have implemented what I think
Michael B was
suggesting.
This is a
sample run on
the Ioniq 5
(which doesn't
have
unsolicited RX
events). </p>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal">This uses the call esp_timer_get_time() got get a
64bit <b>microseconds</b>
since started
value - and
works out the
time to
execute that
way. It's
looking at
absolute time
and not time
in the Task -
so other
things going
on at the same
time in other
tasks will
have an
effect. (The
normal tick
count doesn't
have nearly
enough
resolution to
be useful -
any other
ideas on
measurement?)
I've got total
accumulated
time
displaying in
seconds and
the average in
milliseconds
currently -
but I can
change that
easy enough.</p>
</div>
<div>
<p
class="MsoNormal">The cumulative time is stored as uint64_t which will
be plenty, as
32bit wouldn't
be nearly
enough.</p>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal"><span style="font-family:"Courier New"">OVMS#
<b>poller time
on</b><br>
Poller timing
is now on</span></p>
<div>
<p
class="MsoNormal"> </p>
</div>
<div>
<p
class="MsoNormal" style="margin-bottom:12pt"><span
style="font-family:"Courier New"">OVMS# <b>poller time status</b><br>
Poller timing
is: on<br>
Poll [PRI]
:
n=390 tot=0.2s
ave=0.586ms<br>
Poll [SRX]
:
n=316 tot=0.1s
ave=0.196ms<br>
CAN1 RX[0778]
:
n=382 tot=0.2s
ave=0.615ms<br>
CAN1 RX[07a8]
:
n=48 tot=0.0s
ave=0.510ms<br>
CAN1 RX[07bb]
:
n=162 tot=0.1s
ave=0.519ms<br>
CAN1 RX[07ce]
:
n=33 tot=0.0s
ave=0.469ms<br>
CAN1 RX[07ea]
:
n=408 tot=0.2s
ave=0.467ms<br>
CAN1 RX[07ec]
:
n=486 tot=0.2s
ave=0.477ms<br>
CAN3 RX[07df]
:
n=769 tot=0.2s
ave=0.261ms<br>
CAN1 TX[0770]
:
n=191 tot=0.0s
ave=0.054ms<br>
CAN1 TX[07a0]
:
n=16 tot=0.0s
ave=0.047ms<br>
CAN1 TX[07b3]
:
n=31 tot=0.0s
ave=0.069ms<br>
CAN1 TX[07c6]
:
n=11 tot=0.0s
ave=0.044ms<br>
CAN1 TX[07e2]
:
n=82 tot=0.0s
ave=0.067ms<br>
CAN1 TX[07e4]
:
n=54 tot=0.0s
ave=0.044ms<br>
Set State
: </span></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</body>
</html>