<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    Looks good, thanks.  Checks out ok.<br>
    <br>
    Sorry about the race to check in.  I wanted to do the change without
    having to worry about the base changes in esp-idf, so didn't fetch
    before the commit.<br>
    <br>
    Greg<br>
    <br>
    <br>
    <div class="moz-cite-prefix">Mark Webb-Johnson wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:5B6667D0-7944-41D0-AAA5-4FF953534708@webb-johnson.net">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      Greg, I merged your request. But it conflicted with the change I
      had just made (and pushed) a few hours earlier - to fix the same
      issue.
      <div class=""><br class="">
      </div>
      <div class="">Anyway, I cleaned things up a bit, so the init
        ordering looks better now. Please try.</div>
      <div class=""><br class="">
      </div>
      <div class="">Regards, Mark.<br class="">
        <div><br class="">
          <blockquote type="cite" class="">
            <div class="">On 24 May 2018, at 4:52 AM, Greg D. <<a
                href="mailto:gregd2350@gmail.com" class=""
                moz-do-not-send="true">gregd2350@gmail.com</a>>
              wrote:</div>
            <br class="Apple-interchange-newline">
            <div class="">
              <meta http-equiv="Content-Type" content="text/html;
                charset=UTF-8" class="">
              <div text="#000000" bgcolor="#FFFFFF" class=""> Ah, of
                course.  Depending on timing, I was calling
                xQueueReceive for a queue I hadn't created yet,
                regardless of the contents of said queue.  Listening
                just directs stuff into the queue.<br class="">
                <br class="">
                Anyway, pull request created.  Let me know if this isn't
                what you envisioned for a fix.<br class="">
                <br class="">
                Greg<br class="">
                <br class="">
                <br class="">
                <div class="moz-cite-prefix">Michael Balzer wrote:<br
                    class="">
                </div>
                <blockquote type="cite"
                  cite="mid:f46d1554-730e-77ab-243b-a2930ed75532@expeedo.de"
                  class="">
                  <meta http-equiv="Content-Type" content="text/html;
                    charset=UTF-8" class="">
                  Greg,<br class="">
                  <br class="">
                  the task is messaged to, not called by the CAN
                  framework, it runs as soon as it is created.<br
                    class="">
                  <br class="">
                  Regards,<br class="">
                  Michael<br class="">
                  <br class="">
                  <br class="">
                  <div class="moz-cite-prefix">Am 23.05.2018 um 22:16
                    schrieb Greg D.:<br class="">
                  </div>
                  <blockquote type="cite"
                    cite="mid:85b245c9-3457-327d-667e-c17db79a6d67@gmail.com"
                    class="">
                    <meta http-equiv="Content-Type" content="text/html;
                      charset=UTF-8" class="">
                    Ha, that's interesting.  I've pushed a fix to swap
                    the ordering, but am not quite convinced that this
                    is the issue.  The order of creation shouldn't
                    matter because the OBD2ECU task doesn't register
                    itself as a listener until after both are done.  So
                    why would it even be called?  <br class="">
                    <br class="">
                    Greg<br class="">
                    <br class="">
                    <br class="">
                    <div class="moz-cite-prefix">Michael Balzer wrote:<br
                        class="">
                    </div>
                    <blockquote type="cite"
                      cite="mid:3805f0c8-f5fd-0cb9-9708-5de311a6b856@expeedo.de"
                      class="">
                      <meta http-equiv="Content-Type"
                        content="text/html; charset=UTF-8" class="">
                      Greg,<br class="">
                      <br class="">
                      I just had a look. That crash is simply due to an
                      initialization order mistake in your constructor,
                      has nothing to do with the CAN bus: you create the
                      m_rxqueue _after_ starting the OBD2ECU_task, which
                      uses the m_rxqueue:<br class="">
                      <br class="">
                      <tt class="">static void OBD2ECU_task(void
                        *pvParameters)</tt><tt class=""><br class="">
                      </tt><tt class="">  {</tt><tt class=""><br
                          class="">
                      </tt><tt class="">  obd2ecu *me =
                        (obd2ecu*)pvParameters;</tt><tt class=""><br
                          class="">
                      </tt><tt class=""><br class="">
                      </tt><tt class="">  CAN_frame_t frame;</tt><tt
                        class=""><br class="">
                      </tt><tt class="">  while(1)</tt><tt class=""><br
                          class="">
                      </tt><tt class="">    {</tt><tt class=""><br
                          class="">
                      </tt><tt class="">    if (xQueueReceive(me-><b
                          class="">m_rxqueue</b>, &frame,
                        (portTickType)portMAX_DELAY)==pdTRUE)</tt><tt
                        class=""><br class="">
                      </tt><tt class="">      {</tt><tt class=""><br
                          class="">
                      </tt><tt class="">      // Only handle incoming
                        frames on our CAN bus</tt><tt class=""><br
                          class="">
                      </tt><tt class="">      if (frame.origin ==
                        me->m_can) me->IncomingFrame(&frame);</tt><tt
                        class=""><br class="">
                      </tt><tt class="">      }</tt><tt class=""><br
                          class="">
                      </tt><tt class="">    }</tt><tt class=""><br
                          class="">
                      </tt><tt class="">  }</tt><tt class=""><br
                          class="">
                      </tt><tt class=""><br class="">
                      </tt><tt class="">obd2ecu::obd2ecu(const char*
                        name, canbus* can)</tt><tt class=""><br class="">
                      </tt><tt class="">  : pcp(name)</tt><tt class=""><br
                          class="">
                      </tt><tt class="">  {</tt><tt class=""><br
                          class="">
                      </tt><tt class="">  m_can = can;</tt><tt class=""><br
                          class="">
                      </tt><tt class=""> 
                        xTaskCreatePinnedToCore(OBD2ECU_task, "OVMS
                        OBDII ECU", 6144, (void*)this, 5, &m_task,
                        1);</tt><tt class=""><br class="">
                      </tt><tt class=""><br class="">
                      </tt><b class=""><tt class="">  m_rxqueue =
                          xQueueCreate(20,sizeof(CAN_frame_t));</tt></b><tt
                        class=""><br class="">
                      </tt><tt class=""><br class="">
                      </tt><tt class="">…</tt><tt class=""><br class="">
                      </tt><br class="">
                      <br class="">
                      So the task can reach the xQueueReceive() call
                      before the queue is actually usable.<br class="">
                      <br class="">
                      Regards,<br class="">
                      Michael<br class="">
                      <br class="">
                      <br class="">
                      <div class="moz-cite-prefix">Am 23.05.2018 um
                        03:29 schrieb Greg D.:<br class="">
                      </div>
                      <blockquote type="cite"
                        cite="mid:33670a15-78c3-a1bc-95a4-07034c7193fb@gmail.com"
                        class="">
                        <meta http-equiv="Content-Type"
                          content="text/html; charset=UTF-8" class="">
                        Hi Mark,<br class="">
                        <br class="">
                        Yeah, big shrug here too.  It occurs
                        infrequently, during what was a reboot anyway,
                        and the system recovered itself, so this isn't a
                        critical issue unless it keeps happening (i.e. a
                        crash/reboot loop).  I suppose it could also
                        occur with a frame coming in from a HUD, though
                        that CAN bus is on a different interface chip so
                        if the cause is hardware dependent, it might not
                        occur.<br class="">
                        <br class="">
                        Perhaps we need to not launch the higher level
                        applications (e.g. OBD2ECU) until maybe the 5
                        second mark, when we know the system is
                        initialized and otherwise pretty stable, but
                        within that "Early" timeframe?  Just a thought,<br
                          class="">
                        <br class="">
                        Greg<br class="">
                        <br class="">
                        <br class="">
                        <div class="moz-cite-prefix">Mark Webb-Johnson
                          wrote:<br class="">
                        </div>
                        <blockquote type="cite"
                          cite="mid:B296C17D-71F0-4486-B59A-065AA23690FB@webb-johnson.net"
                          class="">
                          <meta http-equiv="Content-Type"
                            content="text/html; charset=UTF-8" class="">
                          Greg,
                          <div class=""><br class="">
                          </div>
                          <div class="">Yes,
                            the MyCan.RegisterListener(m_rxqueue) will
                            register for all messages on all CAN buses.</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">I am trying to work out a neat
                            way of adding filters to this (both hardware
                            and software). Some easy standard way of
                            filtering traffic in a standard way (like
                            the BFP filters in Unix variants do for
                            ethernet traffic). That will give us a
                            performance boost for things like odb2ecu,
                            retools, etc.</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">But you already have a "if
                            (frame.origin == me->m_can)”, so not sure
                            how this could be affecting you?</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">
                            <blockquote type="cite" class="">What
                              precisely is "early" in the context of a
                              boot crash?  That would put a bound on the
                              size and location of the timing window.</blockquote>
                          </div>
                          <div class=""><br class="">
                          </div>
                          <div class="">10 seconds, I believe. You can
                            see it with the message "I (10576)
                            housekeeping: System considered stable (RAM:
                            8b=99060-99176 32b=20)” about 10 seconds
                            into the boot.</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">Regards, Mark.<br class="">
                            <div class=""><br class="">
                              <blockquote type="cite" class="">
                                <div class="">On 23 May 2018, at 7:55
                                  AM, Greg D. <<a
                                    href="mailto:gregd2350@gmail.com"
                                    class="" moz-do-not-send="true">gregd2350@gmail.com</a>>
                                  wrote:</div>
                                <br class="Apple-interchange-newline">
                                <div class="">
                                  <meta http-equiv="Content-Type"
                                    content="text/html; charset=UTF-8"
                                    class="">
                                  <div text="#000000" bgcolor="#FFFFFF"
                                    class=""> Hi Michael,<br class="">
                                    <br class="">
                                    Correct, not a config issue.  I had
                                    assumed it was a race condition
                                    (first time I've seen it), but
                                    didn't realize it involved my code. 
                                    Yikes!<br class="">
                                    <br class="">
                                    This is the call-back into the
                                    OBD2ECU task when a CAN frame is
                                    received in the system.  Apparently
                                    everyone looking for a CAN frame
                                    gets called when a frame is received
                                    from any bus, and each process needs
                                    to filter out the ones they own. 
                                    So, somehow a CAN frame was seen to
                                    have been received before the system
                                    was fully initialized.  Perhaps
                                    something dropped into the hardware
                                    buffer after the chip was turned on,
                                    but not before the O/S had gotten
                                    everything situated for handling
                                    it.  I don't currently have anything
                                    connected to the OBDII CAN bus
                                    (CAN3) on my module, so this must
                                    have been a frame from the car
                                    itself on CAN 1.  CAN 2 is not used
                                    in my car.<br class="">
                                    <br class="">
                                    What precisely is "early" in the
                                    context of a boot crash?  That would
                                    put a bound on the size and location
                                    of the timing window.<br class="">
                                    <br class="">
                                    Greg<br class="">
                                    <br class="">
                                    <br class="">
                                    <div class="moz-cite-prefix">Michael
                                      Balzer wrote:<br class="">
                                    </div>
                                    <blockquote type="cite"
                                      cite="mid:6a8db28c-03ba-2952-d802-d720e6dd2deb@expeedo.de"
                                      class="">
                                      <meta http-equiv="Content-Type"
                                        content="text/html;
                                        charset=UTF-8" class="">
                                      Greg,<br class="">
                                      <br class="">
                                      the backtrace says the crash
                                      happened here:<br class="">
                                      <br class="">
                                      <tt class="">0x4008f6de is in
                                        vTaskEnterCritical
(/Users/mark/esp/esp-idf/components/freertos/include/freertos/portmacro.h:283).<br
                                          class="">
                                        278     * Warning: From the ISA
                                        docs: in some (unspecified)
                                        cases, the s32c1i instruction
                                        may return the<br class="">
                                        279     * *bitwise inverse* of
                                        the old mem if the mem wasn't
                                        written. This doesn't seem to
                                        happen on the<br class="">
                                        280     * ESP32 (portMUX
                                        assertions would fail).<br
                                          class="">
                                        281     */<br class="">
                                        282    static inline void
                                        uxPortCompareSet(volatile
                                        uint32_t *addr, uint32_t
                                        compare, uint32_t *set) {<br
                                          class="">
                                        283        __asm__ __volatile__
                                        (<br class="">
                                        284            "WSR        
                                        %2,SCOMPARE1 \n"<br class="">
                                        285            "S32C1I     %0,
                                        %1, 0     \n"<br class="">
                                        286            :"=r"(*set)<br
                                          class="">
                                        287            :"r"(addr),
                                        "r"(compare), "0"(*set)<br
                                          class="">
                                        0x4008eb10 is in
                                        xQueueGenericReceive
                                        (/Users/mark/esp/esp-idf/components/freertos/./queue.c:1455).<br
                                          class="">
                                        1450        statements within
                                        the function itself.  This is
                                        done in the interest<br class="">
                                        1451        of execution time
                                        efficiency. */<br class="">
                                        1452    <br class="">
                                        1453        for( ;; )<br
                                          class="">
                                        1454        {<br class="">
                                        1455           
                                        taskENTER_CRITICAL(&pxQueue->mux);<br
                                          class="">
                                        1456            {<br class="">
                                        1457                /* Is there
                                        data in the queue now?  To be
                                        running the calling task<br
                                          class="">
                                        1458                must be   
                                        the highest priority task
                                        wanting to access the queue. */<br
                                          class="">
                                        1459                if(
                                        pxQueue->uxMessagesWaiting
                                        > ( UBaseType_t ) 0 )<br
                                          class="">
                                        <b class="">0x4011f3fd is in
                                          OBD2ECU_task(void*)
(/Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/obd2ecu/src/obd2ecu.cpp:155).</b></tt><tt
                                        class=""><br class="">
                                      </tt><tt class="">150      obd2ecu
                                        *me = (obd2ecu*)pvParameters;</tt><tt
                                        class=""><br class="">
                                      </tt><tt class="">151    </tt><tt
                                        class=""><br class="">
                                      </tt><tt class="">152     
                                        CAN_frame_t frame;</tt><tt
                                        class=""><br class="">
                                      </tt><tt class="">153     
                                        while(1)</tt><tt class=""><br
                                          class="">
                                      </tt><tt class="">154        {</tt><tt
                                        class=""><br class="">
                                      </tt><b class=""><tt class="">155   
                                              if
                                          (xQueueReceive(me->m_rxqueue,
                                          &frame,
                                          (portTickType)portMAX_DELAY)==pdTRUE)</tt></b><tt
                                        class=""><br class="">
                                      </tt><tt class="">156          {</tt><tt
                                        class=""><br class="">
                                      </tt><tt class="">157          //
                                        Only handle incoming frames on
                                        our CAN bus</tt><tt class=""><br
                                          class="">
                                      </tt><tt class="">158          if
                                        (frame.origin == me->m_can)
                                        me->IncomingFrame(&frame);</tt><tt
                                        class=""><br class="">
                                      </tt><tt class="">159          }</tt><tt
                                        class=""><br class="">
                                      </tt><br class="">
                                      <br class="">
                                      So I assume that's the same bug I
                                      already reported in issue #103:<br
                                        class="">
                                      <br class="">
                                      <a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/103"
                                        moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/103</a><br
                                        class="">
                                      <br class="">
                                      …and as it now happened to you, I
                                      assume it's not configuration
                                      related. Can you verify my
                                      assumption?<br class="">
                                      <br class="">
                                      <blockquote type="cite" class="">Looks
                                        like an init race condition,
                                        i.e. m_rxqueue used in undefined
                                        state.</blockquote>
                                      <br class="">
                                      Regards,<br class="">
                                      Michael<br class="">
                                      <br class="">
                                      <br class="">
                                      <br class="">
                                      <div class="moz-cite-prefix">Am
                                        22.05.2018 um 19:24 schrieb Greg
                                        D.:<br class="">
                                      </div>
                                      <blockquote type="cite"
                                        cite="mid:45c73f38-dcea-9bac-7b5a-bedc8e08617d@gmail.com"
                                        class="">
                                        <pre class="" wrap="">Just noticed this on my production 3.1 module.  Was this an artifact of the 
upgrade from .005 to .006?

Last boot was 140983 second(s) ago
Time at boot: 2018-05-20 19:05:59 PDT
This is reset #17 since last power cycle
Detected boot reason: EarlyCrash (12/12)
Crash counters: 1 total, 1 early
Last crash: StoreProhibited exception on core 1
Registers:
PC : 0x4008f6de PS : 0x00060033 A0 : 0x8008eb13 A1 : 0x3ffd6150
A2 : 0xfefeff4e A3 : 0x0000abab A4 : 0xb33fffff A5 : 0x00000001
A6 : 0x00060020 A7 : 0x0000cdcd A8 : 0x0000abab A9 : 0x3ffd6150
A10 : 0x00000003 A11 : 0x00060023 A12 : 0x00060020 A13 : 0x3ffb4188
A14 : 0x000001e8 A15 : 0x00001800 SAR : 0x00000000 EXCCAUSE: 0x0000001d
EXCVADDR: 0xfefeff4e LBEG : 0x00000000 LEND : 0x00000000 LCOUNT : 0x00000000
Backtrace:
0x4008f6de 0x4008eb10 0x4011f3fd 0x40174f95 0x400e73be 0x400e4699 0x40085149 
0x40128bc2 0x40128df7 0x401287b4 0x400ea22d 0x400eac7a 0x400ead09 0x400ead19 
0x402207d6 0x400e9ece 0x400e9f79 0x400e45f5 0x400e4604 0x400f5c09 0x400f035d
Version: 3.1.006/ota_0/main (build idf v3.1-dev-455-gf4905cdf May 20 2018 20:08:31)
--------------------------------------------------------------------------------
Running partition: ota_0
Boot partition: ota_0
Firmware: 3.1.006/ota_0/main (build idf v3.1-dev-455-gf4905cdf May 20 2018 20:08:31)
</pre>
                                        <!--'"--><br class="">
                                        <fieldset
                                          class="mimeAttachmentHeader"></fieldset>
                                        <br class="">
                                        <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                                      </blockquote>
                                      <br class="">
                                      <pre class="moz-signature" cols="160">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
</pre>
                                      <br class="">
                                      <fieldset
                                        class="mimeAttachmentHeader"></fieldset>
                                      <br class="">
                                      <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                                    </blockquote>
                                    <br class="">
                                  </div>
_______________________________________________<br class="">
                                  OvmsDev mailing list<br class="">
                                  <a
                                    href="mailto:OvmsDev@lists.openvehicles.com"
                                    class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br
                                    class="">
                                  <a class="moz-txt-link-freetext"
                                    href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
                                    moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br
                                    class="">
                                </div>
                              </blockquote>
                            </div>
                            <br class="">
                          </div>
                          <br class="">
                          <fieldset class="mimeAttachmentHeader"></fieldset>
                          <br class="">
                          <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                        </blockquote>
                        <br class="">
                        <!--'"--><br class="">
                        <fieldset class="mimeAttachmentHeader"></fieldset>
                        <br class="">
                        <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                      </blockquote>
                      <br class="">
                      <pre class="moz-signature" cols="160">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
</pre>
                      <br class="">
                      <fieldset class="mimeAttachmentHeader"></fieldset>
                      <br class="">
                      <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                    </blockquote>
                    <br class="">
                    <!--'"--><br class="">
                    <fieldset class="mimeAttachmentHeader"></fieldset>
                    <br class="">
                    <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                  </blockquote>
                  <br class="">
                  <pre class="moz-signature" cols="160">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
</pre>
                  <br class="">
                  <fieldset class="mimeAttachmentHeader"></fieldset>
                  <br class="">
                  <pre class="" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                </blockquote>
                <br class="">
              </div>
              _______________________________________________<br
                class="">
              OvmsDev mailing list<br class="">
              <a href="mailto:OvmsDev@lists.openvehicles.com" class=""
                moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br
                class="">
              <a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br
                class="">
            </div>
          </blockquote>
        </div>
        <br class="">
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>