<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    A builtin web UI for this seems a bit over the top. Builtin web
    config pages should focus on user features, this is clearly a
    feature only needed during/for the development/extension of a
    vehicle adapter. Development features in the web UI are confusing
    for end users.<br>
    <br>
    If persistent enabling/disabling is done by a simple config command
    (e.g. "config set can poller.trace on"), that's also doable by
    users.<br>
    <br>
    Regards,<br>
    Michael<br>
    <br>
    <br>
    <div class="moz-cite-prefix">Am 19.05.24 um 02:06 schrieb Michael
      Geddes:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAH0p7uJeqg7syFFSCaDCkv0OQabfHg9A9jUCoZeFe_Ho4Z2Nsw@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div dir="auto">
          <div>I was so focused on how I calculated the value that I
            totally missed that ‰ would be a better description.  I
            could also use the  system 'Ratio' unit... so % or ‰.  </div>
          <div dir="auto"><br>
          </div>
          <div dir="auto">I'll make space to put 'Avg' on the row.  Was
            trying to limit the width for output on a mobile.  I agree
            it would make it easier to understand.</div>
          <div>Totals also makes sense.</div>
          <div dir="auto"><br>
          </div>
          <div>Should I make this a configuration that can be set on the
            web-page?   I'd probably use a configuration change
            notification so that the very bit setting is sync'd with the
            'configuration' value.</div>
          <div><br>
          </div>
          <div>//.ichael</div>
          <div dir="auto"><br>
            <div class="gmail_quote" dir="auto">
              <div dir="ltr" class="gmail_attr">On Sat, 18 May 2024,
                14:05 Michael Balzer, <<a
                  href="mailto:dexter@expeedo.de" rel="noreferrer"
                  target="_blank" moz-do-not-send="true"
                  class="moz-txt-link-freetext">dexter@expeedo.de</a>>
                wrote:<br>
              </div>
              <blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                <div>
                  <blockquote type="cite">
                    <div>I'm not sure whether the 'max' should be the
                      maximum of the smoothed value.. or the maximum of
                      the raw value.</div>
                  </blockquote>
                  <br>
                  It should normally be the maximum of the raw value I
                  think, the maximum of the smoothed value cannot tell
                  about how bad the processing of an ID can become.<br>
                  <br>
                  The naming in the table is a bit confusing I think.
                  (besides: I've never seen "ave" as the abbreviation
                  for average)<br>
                  <br>
                  If I understand you correctly, "time ms per s" is the
                  time share in per mille, so something in that
                  direction would be more clear, and "length ms" would
                  then be "time [ms]".<br>
                  <br>
                  The totals for all averages in the table foot would
                  also be nice.<br>
                  <br>
                  Maybe "Ave" (or avg?) also should be placed on the
                  left, as the "peak" label now suggests being the peak
                  of the average.<br>
                  <br>
                  Btw, keep in mind, not all "edge" users / testers are
                  developers (e.g. the Twizy driver I'm in contact
                  with), collecting stats feedback for vehicles from
                  testers should be straight forward. Maybe add a
                  data/history record, sent automatically on every
                  drive/charge stop when the poller tracing is on?<br>
                  <br>
                  Regards,<br>
                  Michael<br>
                  <br>
                  <br>
                  <div>Am 18.05.24 um 02:28 schrieb Michael Geddes:<br>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">
                      <div>You did say max/pead value.  I also halved
                        the N for both.</div>
                      <div>I'm not sure whether the 'max' should be the
                        maximum of the smoothed value.. or the maximum
                        of the raw value.</div>
                      <div>This is currently the raw-value maximum.  </div>
                      <div>The problem is that the middle column is the
                        maximum of the {{sum over 10s} / (10*1000,000)</div>
                      <div>I could easily change the 'period' to 1s and
                        see how that goes.. was just trying to reduce
                        the larger calculations. </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <font face="monospace">Usage: poller
                        [pause|resume|status|times|trace]</font>
                      <div><font face="monospace"><br>
                          OVMS# poller time status<br>
                          Poller timing is: on<br>
                          Type         | Count    | Ave time  | Ave
                          length<br>
                                       | per s    | ms per s  | ms <br>
-------------+----------+-----------+-----------<br>
                          Poll:PRI     |      1.00|      0.559|    
                           0.543<br>
                           peak        |          |      0.663|    
                           1.528<br>
-------------+----------+-----------+-----------<br>
                          Poll:SRX     |      0.08|      0.009|    
                           0.038<br>
                           peak        |          |      0.068|    
                           0.146<br>
-------------+----------+-----------+-----------<br>
                          CAN1 RX[778] |      0.11|      0.061|    
                           0.280<br>
                           peak        |          |      0.458|    
                           1.046<br>
-------------+----------+-----------+-----------<br>
                          CAN1 RX[7a8] |      0.04|      0.024|    
                           0.124<br>
                           peak        |          |      0.160|    
                           0.615<br>
-------------+----------+-----------+-----------<br>
                          CAN1 TX[770] |      0.05|      0.004|    
                           0.016<br>
                           peak        |          |      0.022|    
                           0.102<br>
-------------+----------+-----------+-----------<br>
                          CAN1 TX[7a0] |      0.02|      0.002|    
                           0.011<br>
                           peak        |          |      0.010|    
                           0.098<br>
-------------+----------+-----------+-----------<br>
                          CAN1 TX[7b3] |      0.01|      0.001|    
                           0.006<br>
                           peak        |          |      0.000|    
                           0.099<br>
-------------+----------+-----------+-----------<br>
                          CAN1 TX[7e2] |      0.02|      0.002|    
                           0.011<br>
                           peak        |          |      0.010|    
                           0.099<br>
-------------+----------+-----------+-----------<br>
                          CAN1 TX[7e4] |      0.08|      0.008|    
                           0.048<br>
                           peak        |          |      0.049|    
                           0.107<br>
-------------+----------+-----------+-----------<br>
                          Cmd:State    |      0.00|      0.000|    
                           0.005<br>
                           peak        |          |      0.000|    
                           0.094</font><br>
                      </div>
                    </div>
                    <br>
                    <div class="gmail_quote">
                      <div dir="ltr" class="gmail_attr">On Fri, 17 May
                        2024 at 15:26, Michael Geddes <<a
                          href="mailto:frog@bunyip.wheelycreek.net"
                          rel="noreferrer noreferrer" target="_blank"
                          moz-do-not-send="true"
                          class="moz-txt-link-freetext">frog@bunyip.wheelycreek.net</a>>
                        wrote:<br>
                      </div>
                      <blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                        <div dir="ltr">
                          <div>This is what I have now.</div>
                          <div>The one on the end is the one  MIchael B
                            was after using an N of 32. (up for
                            discussion).</div>
                          <div><br>
                          </div>
                          <div>The middle is the time  spent in that
                            even t per second.  It accumulates times (in
                            microseconds), and then every 10s it stores
                            it as smoothed (N=16) value.</div>
                          <div>The Count is similar (except that we
                            store a value of '100' as 1 event so it can
                            be still integers and has 2 decimal places).</div>
                          <div><br>
                          </div>
                          <div>Every received poll  does a
                            64bit difference to 32bit (for the elapsed
                            time) and 64bit comparison (for
                            end-of-period).</div>
                          <div>It also does 1x 32bit smoothing and 2x
                            32bit adds.</div>
                          <div><br>
                          </div>
                          <div>Then at the end of a 10s period, it will
                            do a 64bit add to get the next end-of-period
                            value, as well as the 2x 32bit smoothing
                            calcs.</div>
                          <div><br>
                          </div>
                          <div>This is from the Ioniq 5 so not any big
                            values yet.  You can certainly see how
                            insignificant the TX callbacks are.</div>
                          <div>I'll leave it on for when the car is
                            moving and gets some faster polling.</div>
                          <div><font face="monospace"><br>
                            </font></div>
                          <div><font face="monospace">OVMS# poll time
                              status<br>
                              Poller timing is: on<br>
                              Type         | Count    | Ave time  | Ave
                              Length<br>
                                           | per s    | ms per s  | ms <br>
-------------+----------+-----------+-----------<br>
                              Poll:PRI     |      1.00|      0.540|    
                               0.539<br>
                              Poll:SRX     |      0.03|      0.004|    
                               0.017<br>
                              CAN1 RX[778] |      0.06|      0.042|    
                               0.175<br>
                              CAN1 TX[770] |      0.04|      0.002|    
                               0.008<br>
                              Cmd:State    |      0.01|      0.001|    
                               0.005</font><br>
                          </div>
                          <div><font face="monospace"><br>
                            </font></div>
                          <div><font face="monospace">----------------------8<--------------------------------</font></div>
                          <div><font face="monospace"><br>
                            </font></div>
                          <div><font face="monospace">Nice smoothing
                              class (forces N as a power of 2):</font></div>
                          <div>  constexpr unsigned floorlog2(unsigned
                            x)<br>
                                {<br>
                                return x == 1 ? 0 : 1+floorlog2(x
                            >> 1);<br>
                                }<br>
                              /* Maintain a smoothed average using
                            shifts for division.<br>
                               * T should be an integer type<br>
                               * N needs to be a power of 2<br>
                               */<br>
                              template <typename T, unsigned N><br>
                              class average_util_t<br>
                                {<br>
                                private:<br>
                                  T m_ave;<br>
                                public:<br>
                                  average_util_t() : m_ave(0) {}<br>
                                  static const uint8_t _BITS =
                            floorlog2(N);<br>
                                  void add( T val)<br>
                                    {<br>
                                    static_assert(N == (1 <<
                            _BITS), "N must be a power of 2");<br>
                                    m_ave = (((N-1) * m_ave) + val)
                            >> _BITS;<br>
                                    }<br>
                                  T get() { return m_ave; }<br>
                                  operator T() { return m_ave; }<br>
                                };<br>
                          </div>
                          <div><font face="monospace"><br>
                            </font></div>
                          <div><font face="monospace"><br>
                            </font></div>
                        </div>
                        <br>
                        <div class="gmail_quote">
                          <div dir="ltr" class="gmail_attr">On Thu, 16
                            May 2024 at 10:29, Michael Geddes <<a
                              href="mailto:frog@bunyip.wheelycreek.net"
                              rel="noreferrer noreferrer"
                              target="_blank" moz-do-not-send="true"
                              class="moz-txt-link-freetext">frog@bunyip.wheelycreek.net</a>>
                            wrote:<br>
                          </div>
                          <blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                            <div dir="ltr">
                              <div dir="ltr">
                                <div>Thanks Michael,</div>
                                <div dir="ltr"><br>
                                </div>
                                My calculations give me ((2^32)-1) /
                                (1000*1000*3600) = only 1.2 hours of
                                processing time in 32bit.  The initial
                                subtraction is 64bit anyway and I can't
                                see a further 64bit addition being a
                                problem. I have the calculations being
                                performed in doubles at print-out where
                                performance is not really an issue
                                anyway. (Though apparently doing 64 bit
                                division is worse than floating point).
                                <div><br>
                                  <div>In addition</div>
                                  <div>* I currently have this being
                                    able to be turned on and off and
                                    reset manually (only do it when
                                    required).</div>
                                  <div><br>
                                  </div>
                                  <div>*  For the lower volume commands,
                                    the smoothed average is not going to
                                    be useful - the count is more
                                    interesting for different reasons.</div>
                                  <div><br>
                                  </div>
                                  <div>* The total time is quite useful.
                                    Ie a high average time doesn't
                                    matter if the count is low.  The
                                    things that are affecting
                                    performance are stuff with high
                                    total time. Stuff which is happening
                                    100 times a second needs to be a
                                    much lower average than once a
                                    second.</div>
                                  <div>
                                    <div>
                                      <div><br>
                                      </div>
                                      <div>* A measure like 'time per
                                        minute/second' and possibly
                                        count per minute/seconds as a
                                        smoothed average would
                                        potentially be more useful. (or
                                        in addition?)</div>
                                      <div>I think we could do _that_ in
                                        a reasonably efficient manner
                                        using a 64 bit 'last measured
                                        time', a 32 bit accumulated
                                        value and the stored 32 bit
                                        rolling average. </div>
                                      <div>It would boils down to some
                                        iterative (integer) sums and
                                        multiplications plus a divide by
                                        n ^ (time periods passed)  -
                                        which is a shift - and which can
                                        be optimised to '0' if
                                        'time-periods-passed' is more
                                        than 32/(bits-per-n) -
                                        effectively limiting the number
                                        of iterations.</div>
                                      <div>The one issue I can see is
                                        that we need to calculate
                                        'number of time-periods passed'
                                        which is a 64 bit subtraction
                                        followed by a 32 bit division
                                        (not optimisable to a simple
                                        shift).</div>
                                      <div><br>
                                      </div>
                                    </div>
                                  </div>
                                  <div>* I'm also happy to keep a
                                    rolling (32bit) average time.<br>
                                  </div>
                                  <div>   Even if you assume averages in
                                    the 100ms, 32bit is going to happily
                                    support an N of 64 or even 128.</div>
                                  <div>
                                    <div>
                                      <div>   Am I right in thinking
                                        that the choice of N is highly
                                        dependent on frequency. For
                                        things happening 100 times per
                                        second, you might want an N like
                                        128.. where things happening
                                        once per</div>
                                      <div>   second, you might want an
                                        N of 4 or 8.  The other things
                                        we keep track of in this manner
                                        we have a better idea of the
                                        frequency of the thing.<br>
                                      </div>
                                    </div>
                                    <div><br>
                                    </div>
                                  </div>
                                  <div>How about we have (per record
                                    type):</div>
                                  <div>  * total count (since last
                                    reset?) (32 bit)</div>
                                  <div>  * smoothed average of time per
                                    instance (32 bit)</div>
                                  <div><br>
                                  </div>
                                  <div>  * ?xx? total accumulated time
                                    since last reset (64bit) ?? <--
                                    with the below stats this is much
                                    less useful</div>
                                  <div><br>
                                  </div>
                                  <div>  * last-measured-time (64 bit) </div>
                                  <div>  * accumulated count since last
                                    time-period (16bit - but maybe 32bit
                                    anyway for byte alignment?)</div>
                                  <div>  * smoothed average of count per
                                    time-period (32bit)<br>
                                  </div>
                                  <div>  * accumulated time since last
                                    time-period (32bit)</div>
                                  <div>  * smoothed average of time per
                                    time-period (32bit)</div>
                                  <div><br>
                                  </div>
                                  <div>It's possible to keep the </div>
                                  <div><br>
                                  </div>
                                  <div>Is this going to be too much per
                                    record type?  The number of
                                    'records' we are keeping is quite
                                    low (so 10 to 20 maybe) - so it's
                                    not a huge memory burden.<br>
                                  </div>
                                </div>
                                <div><br>
                                </div>
                                <div>Thoughts?</div>
                                <div><br>
                                </div>
                              </div>
                              //.ichael<br>
                              <div class="gmail_quote">
                                <div dir="ltr" class="gmail_attr">On
                                  Thu, 16 May 2024 at 03:09, Michael
                                  Balzer via OvmsDev <<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer noreferrer"
                                    target="_blank"
                                    moz-do-not-send="true"
                                    class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
                                  wrote:<br>
                                </div>
                                <blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                  <div> esp_timer_get_time() is the
                                    right choice for precision timing.<br>
                                    <br>
                                    I'd say uint32 is enough though,
                                    even if counting microseconds that
                                    can hold a total of more than 71
                                    hours of actual processing time.
                                    uint64 has a significant performance
                                    penalty, although I don't recall the
                                    overhead for simple additions.<br>
                                    <br>
                                    Also & more important, the
                                    average wouldn't be my main focus,
                                    but the maximum processing time seen
                                    per ID, which seems to be missing in
                                    your draft.<br>
                                    <br>
                                    Second thought on the average… the
                                    exact overall average really has a
                                    minor meaning, I'd rather see the
                                    current average, adapting to the
                                    current mode of operation
                                    (drive/charge/…). I suggest feeding
                                    the measurements to a low pass
                                    filter to get the smoothed average
                                    of the last n measurements. Pattern:<br>
                                    <br>
                                    runavg = ((N-1) * runavg + newval) /
                                    N<br>
                                    <br>
                                    By using a low power of 2 for N
                                    (e.g. 8 or 16), you can replace the
                                    division by a simple bit shift, and
                                    have enough headroom to use 32 bit
                                    integers.<br>
                                    <br>
                                    Regards,<br>
                                    Michael<br>
                                    <br>
                                    <br>
                                    <div>Am 15.05.24 um 06:51 schrieb
                                      Michael Geddes via OvmsDev:<br>
                                    </div>
                                    <blockquote type="cite">
                                      <div dir="ltr">Formatting aside, I
                                        have implemented what I think
                                        Michael B was suggesting.  This
                                        is a sample run on the Ioniq 5
                                        (which doesn't have
                                        unsolicited RX events).
                                        <div><br>
                                        </div>
                                        <div>This uses the
                                          call esp_timer_get_time() got
                                          get a 64bit <b>microseconds</b>
                                          since started value - and
                                          works out the time to execute
                                          that way. It's looking at
                                          absolute time and not time in
                                          the Task - so other things
                                          going on at the same time in
                                          other tasks will have an
                                          effect.  (The normal tick
                                          count doesn't have nearly
                                          enough resolution to be useful
                                          - any other ideas on
                                          measurement?) I've got total
                                          accumulated time displaying in
                                          seconds and the average in
                                          milliseconds currently - but I
                                          can change that easy enough.</div>
                                        <div>The cumulative time is
                                          stored as uint64_t which will
                                          be plenty, as 32bit wouldn't
                                          be nearly enough.<br>
                                          <div><br>
                                          </div>
                                          <div><font face="monospace">OVMS#
                                              <b>poller time on</b><br>
                                              Poller timing is now on<br>
                                            </font>
                                            <div><br>
                                            </div>
                                            <div><font face="monospace">OVMS#
                                                <b>poller time status</b><br>
                                                Poller timing is: on<br>
                                                Poll [PRI]              
                                                : n=390 tot=0.2s
                                                ave=0.586ms<br>
                                                Poll [SRX]              
                                                : n=316 tot=0.1s
                                                ave=0.196ms<br>
                                                CAN1 RX[0778]          
                                                 : n=382 tot=0.2s
                                                ave=0.615ms<br>
                                                CAN1 RX[07a8]          
                                                 : n=48 tot=0.0s
                                                ave=0.510ms<br>
                                                CAN1 RX[07bb]          
                                                 : n=162 tot=0.1s
                                                ave=0.519ms<br>
                                                CAN1 RX[07ce]          
                                                 : n=33 tot=0.0s
                                                ave=0.469ms<br>
                                                CAN1 RX[07ea]          
                                                 : n=408 tot=0.2s
                                                ave=0.467ms<br>
                                                CAN1 RX[07ec]          
                                                 : n=486 tot=0.2s
                                                ave=0.477ms<br>
                                                CAN3 RX[07df]          
                                                 : n=769 tot=0.2s
                                                ave=0.261ms<br>
                                                CAN1 TX[0770]          
                                                 : n=191 tot=0.0s
                                                ave=0.054ms<br>
                                                CAN1 TX[07a0]          
                                                 : n=16 tot=0.0s
                                                ave=0.047ms<br>
                                                CAN1 TX[07b3]          
                                                 : n=31 tot=0.0s
                                                ave=0.069ms<br>
                                                CAN1 TX[07c6]          
                                                 : n=11 tot=0.0s
                                                ave=0.044ms<br>
                                                CAN1 TX[07e2]          
                                                 : n=82 tot=0.0s
                                                ave=0.067ms<br>
                                                CAN1 TX[07e4]          
                                                 : n=54 tot=0.0s
                                                ave=0.044ms<br>
                                                Set State              
                                                 : n=7 tot=0.0s
                                                ave=0.104ms<br>
                                              </font><br>
                                            </div>
                                          </div>
                                          <div>This is probably going to
                                            be quite useful in general! 
                                            The TX call-backs don't seem
                                            to be significant here. (oh,
                                            I should probably implement
                                            a reset of the values too).</div>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div>//.ichael</div>
                                      </div>
                                      <br>
                                      <div class="gmail_quote">
                                        <div dir="ltr"
                                          class="gmail_attr">On Sun, 12
                                          May 2024 at 22:58, Michael
                                          Geddes <<a
href="mailto:frog@bunyip.wheelycreek.net" rel="noreferrer noreferrer"
                                            target="_blank"
                                            moz-do-not-send="true"
class="moz-txt-link-freetext">frog@bunyip.wheelycreek.net</a>> wrote:<br>
                                        </div>
                                        <blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                          <div dir="ltr">
                                            <div dir="ltr">Yeah - I
                                              certainly wasn't going to
                                              put a hard limit. Just a
                                              log above a certain time,
                                              that being said, the idea
                                              of just collecting stats
                                              (being able to turn it on
                                              via a  "poller timer" set
                                              of commands) would be much
                                              more useful.  I'll look
                                              into that.</div>
                                            <div dir="ltr">
                                              <div><br>
                                              </div>
                                              <div>Average time is
                                                probably a good stat -
                                                and certainly what we
                                                care about.</div>
                                              <div><br>
                                              </div>
                                              <div>I actually am hopeful
                                                that those couple of
                                                things I did might help
                                                reduce that average time
                                                quite a bit (that
                                                short-cutting the isotp
                                                protocol handling
                                                especially). </div>
                                              <div><br>
                                              </div>
                                              <div>That p/r with logging
                                                changes might help
                                                reduce the unproductive
                                                log time further, but
                                                also makes it possible
                                                to turn on the poller
                                                logging without the RX
                                                task logs kicking in.</div>
                                              <div><br>
                                              </div>
                                              <div>//.ichael</div>
                                              <div><br>
                                              </div>
                                            </div>
                                            <br>
                                            <div class="gmail_quote">
                                              <div dir="ltr"
                                                class="gmail_attr">On
                                                Sun, 12 May 2024 at
                                                22:29, Michael Balzer
                                                via OvmsDev <<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer noreferrer"
                                                  target="_blank"
                                                  moz-do-not-send="true"
class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
                                                wrote:<br>
                                              </div>
                                              <blockquote
                                                class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                <div> Warning /
                                                  gathering debug
                                                  statistics about slow
                                                  processing can be
                                                  helpful, but there
                                                  must not be a hard
                                                  limit. Frame/poll
                                                  response processing
                                                  may need disk or
                                                  network I/O, and the
                                                  vehicle task may be
                                                  starving from punctual
                                                  high loads on higher
                                                  priority tasks (e.g.
                                                  networking) or by
                                                  needing to wait for
                                                  some semaphore --
                                                  that's outside the
                                                  application's control,
                                                  and must not lead to
                                                  termination/recreation
                                                  of the task (in case
                                                  you're heading towards
                                                  that direction).<br>
                                                  <br>
                                                  I have no idea how
                                                  much processing time
                                                  the current vehicles
                                                  actually need in their
                                                  respective worst
                                                  cases. Your draft is
                                                  probably too lax, poll
                                                  responses and frames
                                                  normally need to be
                                                  processed much faster.
                                                  I'd say 10 ms is
                                                  already too slow, but
                                                  any wait for a
                                                  queue/semaphore will
                                                  already mean at least
                                                  10 ms (FreeRTOS tick).
                                                  Probably best to begin
                                                  with just collecting
                                                  stats.<br>
                                                  <br>
                                                  Btw, to help in
                                                  narrowing down the
                                                  actual problem case,
                                                  the profiler could
                                                  collect max times per
                                                  RX message ID.<br>
                                                  <br>
                                                  Regards,<br>
                                                  Michael<br>
                                                  <br>
                                                  <br>
                                                  <div>Am 12.05.24 um
                                                    10:41 schrieb
                                                    Michael Geddes:<br>
                                                  </div>
                                                  <blockquote
                                                    type="cite">
                                                    <div dir="ltr">
                                                      <div>I have a
                                                        question for
                                                        Michael B  (or
                                                        whoever) - I
                                                        have a commit
                                                        lined up that
                                                        would add a bit
                                                        of a time check
                                                        to the poller
                                                        loop. What do we
                                                        expect the
                                                        maximum time to
                                                        execute a poller
                                                        loop command
                                                        should be? </div>
                                                      <div>This is a
                                                        rough idea (in
                                                        ms) I have..
                                                        based on nothing
                                                        much really, so
                                                        any ideas would
                                                        be appreciated:<br>
                                                      </div>
                                                      <div>    int
                                                        TardyMaxTime_ms(OvmsPoller::OvmsPollEntryType
                                                        entry_type)<br>
                                                              {<br>
                                                              switch
                                                        (entry_type)<br>
                                                                {<br>
                                                                case
                                                        OvmsPoller::OvmsPollEntryType::Poll:
                                                        return 80;<br>
                                                                case
                                                        OvmsPoller::OvmsPollEntryType::FrameRx:
                                                        return 30;<br>
                                                                case
                                                        OvmsPoller::OvmsPollEntryType::FrameTx:
                                                        return 20;<br>
                                                                case
                                                        OvmsPoller::OvmsPollEntryType::Command:
                                                        return 10;<br>
                                                                case
                                                        OvmsPoller::OvmsPollEntryType::PollState:
                                                        return 15;<br>
                                                                default:
                                                        return 80;<br>
                                                                }<br>
                                                              }<br>
                                                      </div>
                                                      <div><br>
                                                      </div>
                                                      <div>//.ichael<br>
                                                      </div>
                                                      <br>
                                                      <div
class="gmail_quote">
                                                        <div dir="ltr"
class="gmail_attr">On Mon, 6 May 2024 at 07:45, Michael Geddes <<a
href="mailto:frog@bunyip.wheelycreek.net" rel="noreferrer noreferrer"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">frog@bunyip.wheelycreek.net</a>>
                                                          wrote:<br>
                                                        </div>
                                                        <blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div
                                                          dir="auto">I
                                                          realise that I
                                                          was only using
                                                          the standard
                                                          cable to test
                                                          - which
                                                          probably is
                                                          not sufficient
                                                          - I haven't
                                                          looked closely
                                                          at how the
                                                          Leaf OBD to
                                                          Db9 cable is
                                                          different from
                                                          standard.<br>
                                                          </div>
                                                          <div
                                                          dir="auto"><br>
                                                          </div>
                                                          <div
                                                          dir="auto">Ah,
                                                          my bad out the
                                                          queue length. 
                                                           We are
                                                          definitely
                                                          queueing more
                                                          messages
                                                          though.   From
                                                          my log of when
                                                          the overflow
                                                          happened, the
                                                          poller was in
                                                          state 0 which
                                                          means OFF - ie
                                                          nothing was
                                                          being sent!!</div>
                                                          <div
                                                          dir="auto"><br>
                                                          </div>
                                                          <div
                                                          dir="auto">I'll
                                                          look at the TX
                                                          message thing
                                                          - opt in
                                                          sounds good -
                                                          though it
                                                          shouldn't be
                                                          playing that
                                                          much of a part
                                                          here as the
                                                          TXs are
                                                          infrequent in
                                                          this case (or
                                                          zero when the
                                                          leaf is off
                                                          or driving) -
                                                          On the ioniq 5
                                                          when I'm using
                                                          the HUD - I'm
                                                          polling quite
                                                          frequently -
                                                          multiple times
                                                          per second and
                                                          that seems to
                                                          be fine!.</div>
                                                          <div
                                                          dir="auto"><br>
                                                          </div>
                                                          <div
                                                          dir="auto">I
                                                          did find an
                                                          issue with the
                                                          throttling ..
                                                          but  it would
                                                          still mostly
                                                          apply the
                                                          throttling
                                                          where it
                                                          matters, so
                                                          again, it
                                                          shouldn't be
                                                          the problem
                                                          (also, we
                                                          aren't
                                                          transmitting
                                                          in the leaf
                                                          case).</div>
                                                          <div
                                                          dir="auto">
                                                          <div
                                                          dir="auto">
                                                          <div
                                                          dir="auto"><br>
                                                          </div>
                                                          <div
                                                          dir="auto">The
                                                          change I made
                                                          to the logging
                                                          of RX messages
                                                          showed how
                                                          many in a row
                                                          were
                                                          dropped... and
                                                          it was mostly
                                                          1 only in a
                                                          run - which
                                                          means even if
                                                          it is a short
                                                          time between -
                                                          that means
                                                          that the drops
                                                          are being
                                                          interleaved by
                                                          at least one
                                                          success!</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Sooo..
                                                          I'm still
                                                          wondering what
                                                          is going on. 
                                                          Some things
                                                          I'm going to
                                                          try:</div>
                                                          <div><br>
                                                          </div>
                                                          <div>* If the
                                                          number of
                                                          messages on
                                                          the Can bus
                                                          (coming in
                                                          through RX)
                                                          means that the
                                                          queue is
                                                          slowly getting
                                                          longer and not
                                                          quite catching
                                                          up, then
                                                          making the
                                                          queue longer
                                                          will help it
                                                          last longer...
                                                          but only
                                                          pushes the
                                                          problem down
                                                          the road.</div>
                                                          <div>  - Add
                                                          'current queue
                                                          length' to the
                                                          poller status 
                                                          information to
                                                          see if this is
                                                          indeed the
                                                          case?  </div>
                                                          <div>  - Add
                                                          some kind of
                                                          alert when the
                                                          queue reaches
                                                          a % full?</div>
                                                          <div>* Once
                                                          you start
                                                          overflowing
                                                          and getting
                                                          overflow log
                                                          messages, I
                                                          wonder if this
                                                          is then
                                                          contributing
                                                          to the
                                                          problem.</div>
                                                          <div>  - Push
                                                          the overflow
                                                          logging into
                                                          Poller Task
                                                          which can look
                                                          at how many
                                                          drops occurred
                                                          since last
                                                          received item.</div>
                                                          <div>* Split
                                                          up the flags
                                                          for the poller
                                                          messages into
                                                          2:<br>
                                                          </div>
                                                          <div>   -
                                                          Messages that
                                                          are/could be
                                                          happening in
                                                          the TX/RX
                                                          tasks</div>
                                                          <div>   -
                                                          Other noisy
                                                          messages that
                                                          always happen
                                                          in the poller
                                                          task.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Thoughts
                                                          on what else
                                                          we might
                                                          measure to
                                                          figure out
                                                          what is going
                                                          on?<br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <div>//.ichael</div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <br>
                                                          <div
class="gmail_quote">
                                                          <div dir="ltr"
class="gmail_attr">On Sun, 5 May 2024, 19:29 Michael Balzer via OvmsDev,
                                                          <<a
href="mailto:ovmsdev@lists.openvehicles.com" rel="noreferrer noreferrer"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ovmsdev@lists.openvehicles.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div> Michael,<br>
                                                          <br>
                                                          the queue size
                                                          isn't in
                                                          bytes, it's in
                                                          messages:<br>
                                                          <br>
                                                          <blockquote
                                                          type="cite"><font
face="monospace"> * @param uxQueueLength The maximum number of items
                                                          that the queue
                                                          can contain.<br>
                                                           *<br>
                                                           * @param
                                                          uxItemSize The
                                                          number of
                                                          bytes each
                                                          item in the
                                                          queue will
                                                          require.</font></blockquote>
                                                          <br>
                                                          Also, from the
                                                          time stamps in
                                                          Dereks log
                                                          excerpt, there
                                                          were quite
                                                          some dropped
                                                          frames in that
                                                          time window --
                                                          at least 23
                                                          frames in 40
                                                          ms, that's
                                                          bad.<br>
                                                          <br>
                                                          Queue sizes
                                                          are currently:<br>
                                                          <br>
                                                          <font
face="monospace">CONFIG_OVMS_HW_CAN_RX_QUEUE_SIZE=60<br>
CONFIG_OVMS_VEHICLE_CAN_RX_QUEUE_SIZE=60</font><br>
                                                          <br>
                                                          The new poller
                                                          now channels
                                                          all TX
                                                          callbacks
                                                          through the
                                                          task queue
                                                          additionally
                                                          to RX and
                                                          commands. So
                                                          setting the
                                                          queue size to
                                                          be larger than
                                                          the CAN RX
                                                          queue size
                                                          seems
                                                          appropriate.<br>
                                                          <br>
                                                          Nevertheless,
                                                          an overflow
                                                          with more than
                                                          60 waiting
                                                          messages still
                                                          indicates some
                                                          too long
                                                          processing
                                                          time in the
                                                          vehicle task.<br>
                                                          <br>
                                                          TX callbacks
                                                          previously
                                                          were done
                                                          directly in
                                                          the CAN
                                                          context, and
                                                          no current
                                                          vehicle
                                                          overrides the
                                                          empty default
                                                          handler, so
                                                          this imposed
                                                          almost no
                                                          additional
                                                          overhead. By
                                                          requiring a
                                                          queue entry
                                                          for each TX
                                                          callback, this
                                                          feature now
                                                          has a
                                                          potentially
                                                          high impact
                                                          for all
                                                          vehicles. If
                                                          passing these
                                                          to the task is
                                                          actually
                                                          necessary, it
                                                          needs to
                                                          become an
                                                          opt-in
                                                          feature, so
                                                          only vehicles
                                                          subscribing to
                                                          the callback
                                                          actually need
                                                          to cope with
                                                          that
                                                          additional
                                                          load &
                                                          potential
                                                          processing
                                                          delays
                                                          involved.<br>
                                                          <br>
                                                          Regards,<br>
                                                          Michael<br>
                                                          <br>
                                                          <blockquote
                                                          type="cite">
                                                          <div
class="gmail_quote">
                                                          <blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div
class="gmail_quote">
                                                          <blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div>
                                                          <div
                                                          lang="EN-AU">
                                                          <div>
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <div>
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <div>
                                                          <div>
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <blockquote
style="margin-top:5pt;margin-bottom:5pt">
                                                          <div>
                                                          <div>
                                                          <div>
                                                          <blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
                                                          <div>
                                                          <div>
                                                          <div
style="border:1pt solid rgb(204,204,204);padding:7pt"> </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                        </blockquote>
                                                      </div>
                                                    </div>
                                                  </blockquote>
                                                  <br>
                                                  <pre cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                                                </div>
_______________________________________________<br>
                                                OvmsDev mailing list<br>
                                                <a
href="mailto:OvmsDev@lists.openvehicles.com" rel="noreferrer noreferrer"
                                                  target="_blank"
                                                  moz-do-not-send="true"
class="moz-txt-link-freetext">OvmsDev@lists.openvehicles.com</a><br>
                                                <a
href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
rel="noreferrer noreferrer noreferrer" target="_blank"
                                                  moz-do-not-send="true"
class="moz-txt-link-freetext">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br>
                                              </blockquote>
                                            </div>
                                          </div>
                                        </blockquote>
                                      </div>
                                      <br>
                                      <fieldset></fieldset>
                                      <pre>_______________________________________________
OvmsDev mailing list
<a href="mailto:OvmsDev@lists.openvehicles.com"
                                      rel="noreferrer noreferrer"
                                      target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">OvmsDev@lists.openvehicles.com</a>
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
                                      rel="noreferrer noreferrer"
                                      target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                                    </blockquote>
                                    <br>
                                    <pre cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                                  </div>
_______________________________________________<br>
                                  OvmsDev mailing list<br>
                                  <a
href="mailto:OvmsDev@lists.openvehicles.com" rel="noreferrer noreferrer"
                                    target="_blank"
                                    moz-do-not-send="true"
                                    class="moz-txt-link-freetext">OvmsDev@lists.openvehicles.com</a><br>
                                  <a
href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
rel="noreferrer noreferrer noreferrer" target="_blank"
                                    moz-do-not-send="true"
                                    class="moz-txt-link-freetext">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br>
                                </blockquote>
                              </div>
                            </div>
                          </blockquote>
                        </div>
                      </blockquote>
                    </div>
                  </blockquote>
                  <br>
                  <pre cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                </div>
              </blockquote>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
  </body>
</html>