<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Reviving this old topic still impacting us…<div class=""><br class=""></div><div class="">Comparing the master vs for-v3.3 branches, the only thing that changed related to ppp is that in 3.3 the ppp object is dynamically created and destroyed, while in master it is statically allocated at boot. The ppp code is so simple, and I can’t see how that could be a problem. Perhaps related to position in memory, and some other memory corruption?</div><div class=""><br class=""></div><div class="">Anyway, I changed it to not destroy the ppp object when the gsm connection is lost, but merely shutdown the ppp (which is what master branch does). While it is still dynamically allocated, it is no longer as dynamic (being created just once at startup of the cellular system). I’ve never managed to reliably repeat this problem in my environment, but I think this should help.It has been running on my desktop test unit for the past four days without issue.</div><div class=""><br class=""></div><div class="">That code is committed now. I would appreciate it if others who saw this problem could try again with this latest build of the for-v3.3 branch.<br class=""><div class=""><br class=""></div><div class="">Regards, Mark.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 24 Mar 2021, at 3:53 PM, Mark Webb-Johnson <<a href="mailto:mark@webb-johnson.net" class="">mark@webb-johnson.net</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="content-isolator__container"><div class="protected-part"><div class="protected-title">Signed PGP part</div><div class="protected-content"><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Good grief, this is not so easy. Now we have:<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><br class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class=""><br class=""></font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">Core 1 register dump:</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">PC      : 0x40008044  PS      : 0x00060f30  A0      : 0x800fe2cc  A1      : 0x3ffcaa90</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">A2      : 0x3f413acc  A3      : 0x00000046  A4      : 0x00e6807e  A5      : 0x00000000</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">A6      : 0x00000000  A7      : 0x00000000  A8      : 0x00000010  A9      : 0x00e6807e</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">A10     : 0x00000078  A11     : 0x00000009  A12     : 0x3ffcaa3f  A13     : 0x00000032</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">A14     : 0x00000000  A15     : 0x3ffcaa48  SAR     : 0x00000004  EXCCAUSE: 0x0000001c</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">EXCVADDR: 0x00e6807e  LBEG    : 0x4008bdad  LEND    : 0x4008bdd1  LCOUNT  : 0x800f93f4</font></blockquote></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class=""><br class="">ELF file SHA256: 74bb0a75eeb4578b<br class=""><br class="">Backtrace: 0x40008044:0x3ffcaa90 0x400fe2c9:0x3ffcab20 0x400fe412:0x3ffcabb0 0x402937b5:0x3ffcabd0</font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class=""><br class=""></font></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><font face="Andale Mono" style="font-style: normal; font-size: 12px;" class="">0x400fe2c9 is in OvmsNetManager::DoSafePrioritiseAndIndicate() (/Users/hq.mark.johnson/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:723).<br class="">718<span class="Apple-tab-span" style="white-space: pre;">    </span>    }<br class="">719<br class="">720<span class="Apple-tab-span" style="white-space: pre;">   </span>  for (struct netif *pri = netif_list; pri != NULL; pri=pri->next)<br class="">721<span class="Apple-tab-span" style="white-space: pre;">        </span>    {<br class="">722<span class="Apple-tab-span" style="white-space: pre;">   </span>    ESP_EARLY_LOGI(TAG,"DoSafePrioritiseAndIndicate: interface %p",pri);<br class="">723<span class="Apple-tab-span" style="white-space: pre;">      </span>    ESP_EARLY_LOGI(TAG,"DoSafePrioritiseAndIndicate:   name    %s",pri->name);<br class="">724<span class="Apple-tab-span" style="white-space: pre;">      </span>    if ((pri->name[0]==search[0])&&<br class="">725<span class="Apple-tab-span" style="white-space: pre;">  </span>        (pri->name[1]==search[1]))<br class="">726<span class="Apple-tab-span" style="white-space: pre;"> </span>      {<br class="">727<span class="Apple-tab-span" style="white-space: pre;">    </span>      if (search[0] != m_previous_name[0] || search[1] != m_previous_name[1])<br class="">0x400fe412 is in SafePrioritiseAndIndicate(void*) (/Users/hq.mark.johnson/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:676).<br class="">671<span class="Apple-tab-span" style="white-space: pre;"> </span>    }<br class="">672<span class="Apple-tab-span" style="white-space: pre;">   </span>  }<br class="">673<br class="">674<span class="Apple-tab-span" style="white-space: pre;">  </span>void SafePrioritiseAndIndicate(void* ctx)<br class="">675<span class="Apple-tab-span" style="white-space: pre;"> </span>  {<br class="">676<span class="Apple-tab-span" style="white-space: pre;">  </span>  MyNetManager.DoSafePrioritiseAndIndicate();<br class="">677<span class="Apple-tab-span" style="white-space: pre;">        </span>  }<br class="">678<br class="">679<span class="Apple-tab-span" style="white-space: pre;">  </span>void OvmsNetManager::PrioritiseAndIndicate()<br class="">680<span class="Apple-tab-span" style="white-space: pre;">      </span>  {<br class="">0x402937b5 is in tcpip_thread (/Users/hq.mark.johnson/esp/esp-idf/components/lwip/lwip/src/api/tcpip.c:158).<br class="">153<span class="Apple-tab-span" style="white-space: pre;"> </span>      break;<br class="">154<span class="Apple-tab-span" style="white-space: pre;">       </span>#endif /* LWIP_TCPIP_TIMEOUT && LWIP_TIMERS */<br class="">155<br class="">156<span class="Apple-tab-span" style="white-space: pre;">    </span>    case TCPIP_MSG_CALLBACK:<br class="">157<span class="Apple-tab-span" style="white-space: pre;">    </span>      LWIP_DEBUGF(TCPIP_DEBUG, ("tcpip_thread: CALLBACK %p\n", (void *)msg));<br class="">158<span class="Apple-tab-span" style="white-space: pre;">    </span>      msg->msg.cb.function(msg->msg.cb.ctx);<br class="">159<span class="Apple-tab-span" style="white-space: pre;"> </span>      memp_free(MEMP_TCPIP_MSG_API, msg);<br class="">160<span class="Apple-tab-span" style="white-space: pre;">  </span>      break;<br class="">161<br class="">162<span class="Apple-tab-span" style="white-space: pre;">       </span>    case TCPIP_MSG_CALLBACK_STATIC:</font></blockquote><div class=""><br class=""></div><div class="">So the issue is most likely corruption of the network interface structure, not thread safe traversal.</div><div class=""><br class=""></div><div class="">I had added some ESP_EARLY_LOGI statements, so can see a little more of what is going on:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (103202) gsm-ppp: Initialising...</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (103212) events: Signal(system.modem.netmode)</span></font></div></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105902) netmanager: DoSafePrioritiseAndIndicate: start</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105902) netmanager: DoSafePrioritiseAndIndicate: connected wifi</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105912) netmanager: DoSafePrioritiseAndIndicate: interface 0x3ffed6a0</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105912) netmanager: DoSafePrioritiseAndIndicate:   name    pp</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105922) netmanager: DoSafePrioritiseAndIndicate: interface 0x3ffde854</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105932) netmanager: DoSafePrioritiseAndIndicate:   name    ap</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105932) netmanager: DoSafePrioritiseAndIndicate: interface 0x3ffde640</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105942) netmanager: DoSafePrioritiseAndIndicate:   name    st</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105952) netmanager: DoSafePrioritiseAndIndicate: end</span></font></div></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105902) gsm-ppp: StatusCallBack: None</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105902) gsm-ppp: status_cb: Connected</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (105902) gsm-ppp:    our_ipaddr  = 10.52.40.80</span></font></div></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">…</span></font></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3708442) cellular: PPP Connection disconnected</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3708442) cellular: PPP Connection disconnected</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709212) netmanager: DoSafePrioritiseAndIndicate: start</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709212) netmanager: DoSafePrioritiseAndIndicate: connected wifi</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709212) netmanager: DoSafePrioritiseAndIndicate: interface 0x3ffed6a0</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709222) netmanager: DoSafePrioritiseAndIndicate:   name    pp</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709222) netmanager: DoSafePrioritiseAndIndicate: interface 0x30323930</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709232) netmanager: DoSafePrioritiseAndIndicate:   name    f</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709242) netmanager: DoSafePrioritiseAndIndicate: interface 0x667fc000</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">I (3709252) netmanager: DoSafePrioritiseAndIndicate:   name</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-style: normal; font-size: 12px;" class="">Guru Meditation Error: Core  1 panic'ed (Interrupt wdt timeout on CPU1)</span></font></div></div></blockquote><div class=""><div class=""><br class=""></div><div class="">Doesn’t help much, apart from confirm the corruption. Took about an hour to recreate the problem.</div><div class=""><br class=""></div><div class="">I’ll keep looking.</div><div class=""><br class=""></div><div class="">Regards, Mark.</div><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 23 Mar 2021, at 4:05 PM, Mark Webb-Johnson <<a href="mailto:mark@webb-johnson.net" class="">mark@webb-johnson.net</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="content-isolator__container"><div class="protected-part"><div class="protected-title">Signed PGP part</div><div class="protected-content"><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">My attempt didn’t work (still crashes), so I’m now trying your suggestion of wrapping PrioritiseAndIndicate() in a tcpip_callback_with_block. <div class=""><br class=""></div><div class=""> 🤞🏻</div><div class=""><br class=""></div><div class="">Regards, Mark.<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 23 Mar 2021, at 3:02 PM, Michael Balzer <<a href="mailto:dexter@expeedo.de" class="">dexter@expeedo.de</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="content-isolator__container"><div class="protected-part"><div class="protected-title">Signed PGP part</div><div class="protected-content">
  
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
  
  <div class="">
    Mark,<br class="">
    <br class="">
    regarding point 2: I've had the same issue with jobs that need to
    iterate over the mongoose connection list, I introduced the
    netmanager job queue for this to delegate these to the mongoose
    context.<br class="">
    <br class="">
    I remember seeing LwIP has a similar API while browsing the source…
    yes, found it: the "tcpip_callback…" functions, e.g.:<br class="">
    <br class="">
    <font face="monospace" class="">/**<br class="">
       * Call a specific function in the thread context of<br class="">
       * tcpip_thread for easy access synchronization.<br class="">
       * A function called in that way may access lwIP core code<br class="">
       * without fearing concurrent access.<br class="">
       *<br class="">
       * @param function the function to call<br class="">
       * @param ctx parameter passed to f<br class="">
       * @param block 1 to block until the request is posted, 0 to
      non-blocking mode<br class="">
       * @return ERR_OK if the function was called, another err_t if not<br class="">
       */<br class="">
      err_t<br class="">
      tcpip_callback_with_block(tcpip_callback_fn function, void *ctx,
      u8_t block)</font><br class="">
    <br class="">
    So we probably can use this to execute PrioritiseAndIndicate() in
    the LwIP context.<br class="">
    <br class="">
    Regards,<br class="">
    Michael<br class="">
    <br class="">
    <br class="">
    <div class="moz-cite-prefix">Am 23.03.21 um 06:47 schrieb Mark
      Webb-Johnson:<br class="">
    </div>
    <blockquote type="cite" cite="mid:CB78F43A-5706-4504-A0B2-F287DC10B40C@webb-johnson.net" class="">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
      OK, some progress…
      <div class=""><br class="">
      </div>
      <div class="">
        <ol class="MailOutline">
          <li class="">I’ve added a check in the OvmsSyncHttpClient code
            to refuse to block while running as the netman (mongoose)
            task. This will now simply fail the http connection, and log
            an error. Not perfect, and not a solution to the core
            problem, but at least it avoids a known problem.<br class="">
            <br class="">
            I’m not sure of the best permanent solution to this. It
            seems that we need a callback interface to run commands
            asynchronously, and use that in mongoose event handlers.
            Adding another mongoose event loop, or using a separate
            networking socket with select(), just minimise the problem -
            they don’t solve it. The core issue here is blocking during
            a mongoose event delivery. That is going to pause all high
            level networking.<br class="">
            <br class="">
          </li>
          <li class="">I found a race condition in ovms_netmanager that
            seems nasty. The new cellular code could raise duplicate
            modem.down signals, picked up and handled in
            ovms_netmanager. As part of that it calls
            a PrioritiseAndIndicate() function that iterates over the
            network interface list (maintained by LWIP). If that network
            interface list is modified (eg; removing an interface) while
            it is being traversed, nasty crashes can happen. The ‘fix’
            I’ve done is again just a workaround to try to reduce the
            duplicate signals and hence reduce the likelyhood of the
            problem happening, but it won’t fix the core problem (that
            is in both master and for-v3.3).<br class="">
            <br class="">
            There is a netif_find function in LWIP, but (a) that
            requires an interface number that we don’t have, and (b)
            doesn’t seem to lock the list either.<br class="">
            <br class="">
            Can’t think of an elegant solution to this, other than
            modifications to lwip. We could add our own mutex and use
            that whenever we talk to lwip, but even that would miss out
            on some modifications to the network interface list, I
            guess. </li>
        </ol>
        <div class=""><br class="">
        </div>
        <div class="">These two changes are in ‘pre’ now, and I am trying them in
          my car.</div>
        <div class=""><br class="">
        </div>
        <div class="">Regards, Mark.</div>
        <div class=""><br class="">
          <blockquote type="cite" class="">
            <div class="">On 22 Mar 2021, at 6:06 PM, Michael Balzer
              <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>> wrote:</div>
            <br class="Apple-interchange-newline">
            <div class="">
              <div class="content-isolator__container">
                <div class="protected-part">
                  <div class="protected-title">Signed PGP part</div>
                  <div class="protected-content">
                    <meta http-equiv="Content-Type" content="text/html;
                      charset=UTF-8" class="">
                    <div class=""> In master, running commands via ssh
                      or server-v2 block, because these are running
                      synchronously in the mongoose context.<br class="">
                      <br class="">
                      Running commands via web doesn't block, as the
                      webcommand class starts a separate task for each
                      execution.<br class="">
                      <br class="">
                      The firmware config page does a synchronous call
                      to MyOTA.GetStatus(), so that call is executed in
                      the mongoose context. It still works in master,
                      just needs a second or two to fetch the version
                      file.<br class="">
                      <br class="">
                      Regards,<br class="">
                      Michael<br class="">
                      <br class="">
                      <br class="">
                      <div class="moz-cite-prefix">Am 22.03.21 um 10:38
                        schrieb Mark Webb-Johnson:<br class="">
                      </div>
                      <blockquote type="cite" cite="mid:1ACB6945-97E0-45E7-AA07-4EB5AC42B96F@webb-johnson.net" class="">
                        <meta http-equiv="content-type" content="text/html; charset=UTF-8" class="">
                        <div dir="ltr" class="">In master branch, at the
                          moment, if a command is run from the web shell
                          (or server v2), surely the mongoose task will
                          block as the web server / server v2 blocks
                          waiting for the command to run to completion?</div>
                        <div dir="ltr" class=""><br class="">
                        </div>
                        <div dir="ltr" class="">Doesn’t necessarily need
                          to be a networking command. Something long
                          running like the string speed tests.</div>
                        <div dir="ltr" class=""><br class="">
                        </div>
                        <div dir="ltr" class="">In v3.3 I can easily
                          detect the task wait being requested in the
                          http library (by seeing if current task id ==
                          mongoose task), and fail (which I should do
                          anyway). But I am more concerned with the
                          general case now (which I think may be wrong
                          in both master and for-v3.3).</div>
                        <div dir="ltr" class=""><br class="">
                        </div>
                        <div dir="ltr" class="">Regards, Mark</div>
                        <div dir="ltr" class=""><br class="">
                        </div>
                        <div dir="ltr" class="">
                          <blockquote type="cite" class="">On 22 Mar
                            2021, at 5:22 PM, Michael Balzer <a class="moz-txt-link-rfc2396E" href="mailto:dexter@expeedo.de" moz-do-not-send="true"><dexter@expeedo.de></a>
                            wrote:<br class="">
                            <br class="">
                          </blockquote>
                        </div>
                        <blockquote type="cite" class="">
                          <div dir="ltr" class="">
                            <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
                            I think we must avoid blocking the Mongoose
                            task, as that's the central network
                            dispatcher.<br class="">
                            <br class="">
                            Chris had implemented a workaround in one of
                            his PRs that could allow that to be done
                            temporarily by running a local Mongoose main
                            loop during a synchronous operation, but I
                            still see potential issues from that, as it
                            wasn't the standard handling as done by the
                            task, and as it may need to recurse.<br class="">
                            <br class="">
                            Maybe the old OvmsHttpClient using socket
                            I/O is the right way for synchronous network
                            operations?<br class="">
                            <br class="">
                            Regards,<br class="">
                            Michael<br class="">
                            <br class="">
                            <br class="">
                            <div class="moz-cite-prefix">Am 22.03.21 um
                              07:15 schrieb Mark Webb-Johnson:<br class="">
                            </div>
                            <blockquote type="cite" cite="mid:F88B060F-2CB7-4F77-99AD-08ACD03E95B9@webb-johnson.net" class="">
                              <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
                              Not sure how to resolve this.
                              <div class=""><br class="">
                              </div>
                              <div class="">
                                <div style="caret-color: rgb(0, 0, 0);" class="">OvmsSyncHttpClient is
                                  currently used in commands from
                                  ovms_plugins and ovms_ota.</div>
                              </div>
                              <div class=""><br class="">
                              </div>
                              <div class="">I could bring back
                                the OvmsHttpClient blocking
                                (non-mongoose) implementation, but I
                                don’t think that would address the core
                                problem here:</div>
                              <div class=""><br class="">
                              </div>
                              <blockquote style="margin: 0 0 0 40px;
                                border: none; padding: 0px;" class="">
                                <div class="">Inside a mongoose callback
                                  (inside the mongoose networking task),
                                  we are making blocking calls (and in
                                  particular calls that could block for
                                  several tens of seconds).</div>
                              </blockquote>
                              <div class=""><br class="">
                              </div>
                              <div class="">But fundamentally is it ok
                                to block the mongoose networking task
                                for extended periods during a mongoose
                                event callback?</div>
                              <div class=""><br class="">
                              </div>
                              <div class="">Mark</div>
                              <div class="">
                                <div class=""><br class="">
                                  <blockquote type="cite" class="">
                                    <div class="">On 21 Mar 2021, at
                                      9:57 PM, Michael Balzer <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>>
                                      wrote:</div>
                                    <br class="Apple-interchange-newline">
                                    <div class="">
                                      <div class="content-isolator__container">
                                        <div class="protected-part">
                                          <div class="protected-title">Signed
                                            PGP part</div>
                                          <div class="protected-content">
                                            <meta http-equiv="Content-Type" content="text/html;
                                              charset=UTF-8" class="">
                                            <div class=""> I've found
                                              opening the web UI
                                              firmware page or calling
                                              "ota status" via ssh to
                                              consistently deadlock the
                                              network on my module.<br class="">
                                              <br class="">
                                              <font class="" face="monospace">I
                                                (130531) webserver: HTTP
                                                GET /cfg/firmware<br class="">
                                                D (130531) http:
                                                OvmsSyncHttpClient:
                                                Connect to <a href="http://ovms.dexters-web.de/" class="" moz-do-not-send="true">ovms.dexters-web.de:80</a><br class="">
                                                D (130541) http:
                                                OvmsSyncHttpClient:
                                                waiting for completion</font><br class="">
                                              <br class="">
                                              After that log message,
                                              the network is dead, and
                                              the netmanager also
                                              doesn't respond:<br class="">
                                              <br class="">
                                              <font class="" face="monospace">OVMS#
                                                network list <br class="">
                                                ERROR: job failed<br class="">
                                                D (183241) netmanager:
                                                send cmd 1 from
                                                0x3ffe7054<br class="">
                                                W (193241) netmanager:
                                                ExecuteJob: cmd 1:
                                                timeout</font><br class="">
                                              <br class="">
                                              The interfaces seem to be
                                              registered and online, but
                                              nothing gets in or out:<br class="">
                                              <br class="">
                                              <font class="" face="monospace">OVMS#
                                                network status <br class="">
                                                Interface#3: pp3 (ifup=1
                                                linkup=1)<br class="">
                                                  IPv4:
                                                10.170.195.13/255.255.255.255
                                                gateway 10.64.64.64<br class="">
                                                <br class="">
                                                Interface#2: ap2 (ifup=1
                                                linkup=1)<br class="">
                                                  IPv4:
                                                192.168.4.1/255.255.255.0
                                                gateway 192.168.4.1<br class="">
                                                <br class="">
                                                Interface#1: st1 (ifup=1
                                                linkup=1)<br class="">
                                                  IPv4:
                                                192.168.2.106/255.255.255.0
                                                gateway 192.168.2.1<br class="">
                                                <br class="">
                                                DNS: 192.168.2.1<br class="">
                                                <br class="">
                                                Default Interface: st1
                                                (192.168.2.106/255.255.255.0
                                                gateway 192.168.2.1)</font><br class="">
                                              <br class="">
                                              <br class="">
                                              A couple of minutes later,
                                              server-v2 recognizes the
                                              stale connection and
                                              issues a network restart,
                                              which fails resulting in
                                              the same behaviour as
                                              shown below with finally
                                              forced reboot by loss of
                                              an important event.<br class="">
                                              <br class="">
                                              Doing "ota status" from
                                              USB works normally, so
                                              this looks like
                                              OvmsSyncHttpClient not
                                              being able to run from
                                              within a mongoose client.<br class="">
                                              <br class="">
                                              Regards,<br class="">
                                              Michael<br class="">
                                              <br class="">
                                              <br class="">
                                              <div class="moz-cite-prefix">Am
                                                18.03.21 um 08:14
                                                schrieb Mark
                                                Webb-Johnson:<br class="">
                                              </div>
                                              <blockquote type="cite" cite="mid:1FD94B8C-1060-44BF-8E3A-6D3534352100@webb-johnson.net" class="">
                                                <meta http-equiv="Content-Type" content="text/html;
                                                  charset=UTF-8" class="">
                                                Tried to repeat this,
                                                but not having much
                                                success. Here is my car
                                                module, with network
                                                still up:
                                                <div class=""><br class="">
                                                </div>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <div class="">
                                                    <div class="">OVMS#
                                                      boot status</div>
                                                    <div class="">Last
                                                      boot was 262355
                                                      second(s) ago</div>
                                                  </div>
                                                </blockquote>
                                                <div class="">
                                                  <div class=""><br class="">
                                                  </div>
                                                  <div class="">I did
                                                    manage to catch one
                                                    network related
                                                    crash after
                                                    repeatedly
                                                    disconnecting and
                                                    reconnecting the
                                                    cellular antenna.
                                                    This was:</div>
                                                  <div class=""><br class="">
                                                  </div>
                                                </div>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <div class="">
                                                    <div class=""><font class="" face="Andale
                                                        Mono"><span style="font-style:
                                                          normal;
                                                          font-size:
                                                          12px;" class="">I
                                                          (3717989)
                                                          cellular: PPP
                                                          Connection
                                                          disconnected</span></font></div>
                                                    <div class=""><font class="" face="Andale
                                                        Mono"><span style="font-style:
                                                          normal;
                                                          font-size:
                                                          12px;" class="">Guru
                                                          Meditation
                                                          Error: Core  1
                                                          panic'ed
                                                          (LoadProhibited).
                                                          Exception was
                                                          unhandled.</span></font></div>
                                                  </div>
                                                  <div class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class=""><br class="">
                                                      </span></font></div>
                                                  <div class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">0x400fe082 is in
                                                        OvmsNetManager::PrioritiseAndIndicate()
(/home/openvehicles/build/Open-Vehicle-Monitoring-System-pre/vehicle/OVMS.V3/main/ovms_netmanager.cpp:707).</span></font></div>
                                                </blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class="">
                                                    <div class=""><font class="" face="Andale
                                                        Mono"><span style="font-style:
                                                          normal;
                                                          font-size:
                                                          12px;" class="">707  
                                                                if
                                                          ((pri->name[0]==search[0])&&</span></font></div>
                                                  </blockquote>
                                                  <font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400ed360
                                                      is in
                                                      OvmsMetricString::SetValue(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >)
(/home/openvehicles/build/Open-Vehicle-Monitoring-System-pre/vehicle/OVMS.V3/main/ovms_metrics.cpp:1358).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">1357    void
                                                        OvmsMetricString::SetValue(std::string
                                                        value)</span></font></blockquote>
                                                  <font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x4008bdad
                                                      is at
                                                      ../../../../.././newlib/libc/machine/xtensa/strcmp.S:586.</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x4008bdd1
                                                      is at
                                                      ../../../../.././newlib/libc/machine/xtensa/strcmp.S:604.</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400fe886
                                                      is in
                                                      OvmsNetManager::ModemDown(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >, void*)
(/home/openvehicles/build/Open-Vehicle-Monitoring-System-pre/vehicle/OVMS.V3/main/ovms_netmanager.cpp:522).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">522        
                                                        PrioritiseAndIndicate();</span></font></blockquote>
                                                </blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400fd752
                                                      is in
                                                      std::_Function_handler<void
(std::__cxx11::basic_string<char, std::char_traits<char>,
                                                      std::allocator<char>
                                                      >, void*),
                                                      std::_Bind<std::_Mem_fn<void
(OvmsNetManager::*)(std::__cxx11::basic_string<char,
                                                      std::char_traits<char>,
std::allocator<char> >, void*)> (OvmsNetManager*,
                                                      std::_Placeholder<1>,
std::_Placeholder<2>)> >::_M_invoke(std::_Any_data
                                                      const&,
                                                      std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >&&,
                                                      void*&&)
(/home/openvehicles/build/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">600             {
                                                        return
(__object->*_M_pmf)(std::forward<_Args>(__args)...); }</span></font></blockquote>
                                                </blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400f512e
                                                      is in
                                                      std::function<void
(std::__cxx11::basic_string<char, std::char_traits<char>,
                                                      std::allocator<char>
                                                      >,
                                                      void*)>::operator()(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >, void*)
                                                      const
(/home/openvehicles/build/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">2271        
                                                         return
                                                        _M_invoker(_M_functor,
std::forward<_ArgTypes>(__args)...);</span></font></blockquote>
                                                </blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400f52f1
                                                      is in
                                                      OvmsEvents::HandleQueueSignalEvent(event_queue_t*)
(/home/openvehicles/build/Open-Vehicle-Monitoring-System-pre/vehicle/OVMS.V3/main/ovms_events.cpp:283).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">283            
                                                        m_current_callback->m_callback(m_current_event,
msg->body.signal.data);</span></font></blockquote>
                                                </blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400f53d8
                                                      is in
                                                      OvmsEvents::EventTask()
(/home/openvehicles/build/Open-Vehicle-Monitoring-System-pre/vehicle/OVMS.V3/main/ovms_events.cpp:237).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">237              
HandleQueueSignalEvent(&msg);</span></font></blockquote>
                                                </blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class=""><br class="">
                                                    </span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style:
                                                      normal; font-size:
                                                      12px;" class="">0x400f545d
                                                      is in
                                                      EventLaunchTask(void*)
(/home/openvehicles/build/Open-Vehicle-Monitoring-System-pre/vehicle/OVMS.V3/main/ovms_events.cpp:80).</span></font></blockquote>
                                                <blockquote style="margin: 0 0 0
                                                  40px; border: none;
                                                  padding: 0px;" class="">
                                                  <blockquote style="margin: 0 0 0
                                                    40px; border: none;
                                                    padding: 0px;" class=""><font class="" face="Andale Mono"><span style="font-style: normal; font-size: 12px;" class="">80      
                                                         me->EventTask();</span></font></blockquote>
                                                </blockquote>
                                                <div class="">
                                                  <div class=""><br class="">
                                                  </div>
                                                  <div class="">My
                                                    for_v3.3 branch does
                                                    include the
                                                    preliminary changes
                                                    to support the wifi
                                                    at 20MHz bandwidth,
                                                    and perhaps those
                                                    could be affecting
                                                    things. I do notice
                                                    that if I ‘power
                                                    wifi off’, then
                                                    ‘wifi mode client’,
                                                    it can connect to
                                                    the station, but not
                                                    get an IP address.
                                                    I’ve just tried to
                                                    merge in the latest
                                                    fixes to that, and
                                                    rebuilt a release. I
                                                    will continue to
                                                    test with that.</div>
                                                  <div class=""><br class="">
                                                  </div>
                                                  <div class="">Regards,
                                                    Mark.</div>
                                                  <div class=""><br class="">
                                                    <blockquote type="cite" class="">
                                                      <div class="">On
                                                        12 Mar 2021, at
                                                        10:32 PM,
                                                        Michael Balzer
                                                        <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>>
                                                        wrote:</div>
                                                      <br class="Apple-interchange-newline">
                                                      <div class="">
                                                        <div class="">
                                                          <div class="content-isolator__container">
                                                          <div class="protected-part">
                                                          <div class="protected-title">Signed
                                                          PGP part</div>
                                                          <div class="protected-content">I
                                                          just tried
                                                          switching to
                                                          for-v3.3 in my
                                                          car module
                                                          after tests on
                                                          my desk module
                                                          were OK, and
                                                          I've run into
                                                          the very same
                                                          problem with
                                                          for-v3.3. So
                                                          the issue
                                                          isn't related
                                                          to esp-idf.<br class="">
                                                          <br class="">
                                                          The network
                                                          only
                                                          occasionally
                                                          starts
                                                          normally, but
                                                          even then all
                                                          connectivity
                                                          is lost after
                                                          a couple of
                                                          minutes.<br class="">
                                                          <br class="">
                                                          The stale
                                                          connection
                                                          watchdog in
                                                          server-v2
                                                          triggers a
                                                          network
                                                          restart, but
                                                          that also
                                                          doesn't seem
                                                          to succeed:<br class="">
                                                          <br class="">
                                                          2021-03-12
                                                          14:53:01.802
                                                          CET W (981652)
ovms-server-v2: Detected stale connection (issue #241), restarting
                                                          network<br class="">
                                                          2021-03-12
                                                          14:53:01.802
                                                          CET I (981652)
                                                          esp32wifi:
                                                          Restart<br class="">
                                                          2021-03-12
                                                          14:53:01.802
                                                          CET I (981652)
                                                          esp32wifi:
                                                          Stopping WIFI
                                                          station<br class="">
                                                          2021-03-12
                                                          14:53:01.812
                                                          CET I (981662)
                                                          wifi:state:
                                                          run -> init
                                                          (0)<br class="">
                                                          2021-03-12
                                                          14:53:01.812
                                                          CET I (981662)
                                                          wifi:pm stop,
                                                          total sleep
                                                          time:
                                                          831205045 us /
                                                          975329961 us<br class="">
                                                          <br class="">
                                                          2021-03-12
                                                          14:53:01.812
                                                          CET I (981662)
wifi:new:<1,0>, old:<1,1>, ap:<1,1>, sta:<1,1>,
                                                          prof:1<br class="">
                                                          2021-03-12
                                                          14:53:01.832
                                                          CET I (981682)
                                                          wifi:flush txq<br class="">
                                                          2021-03-12
                                                          14:53:01.842
                                                          CET I (981692)
                                                          wifi:stop sw
                                                          txq<br class="">
                                                          2021-03-12
                                                          14:53:01.842
                                                          CET I (981692)
                                                          wifi:lmac stop
                                                          hw txq<br class="">
                                                          2021-03-12
                                                          14:53:01.852
                                                          CET I (981702)
                                                          esp32wifi:
                                                          Powering down
                                                          WIFI driver<br class="">
                                                          2021-03-12
                                                          14:53:01.852
                                                          CET I (981702)
                                                          wifi:Deinit
                                                          lldesc rx
                                                          mblock:16<br class="">
                                                          2021-03-12
                                                          14:53:01.862
                                                          CET I (981712)
                                                          esp32wifi:
                                                          Powering up
                                                          WIFI driver<br class="">
                                                          2021-03-12
                                                          14:53:01.862
                                                          CET I (981712)
wifi:nvs_log_init, erase log key successfully, reinit nvs log<br class="">
                                                          2021-03-12
                                                          14:53:01.882
                                                          CET I (981732)
                                                          wifi:wifi
                                                          driver task:
                                                          3ffd4d84,
                                                          prio:23,
                                                          stack:3584,
                                                          core=0<br class="">
                                                          2021-03-12
                                                          14:53:01.882
                                                          CET I (981732)
                                                          system_api:
                                                          Base MAC
                                                          address is not
                                                          set, read
                                                          default base
                                                          MAC address
                                                          from BLK0 of
                                                          EFUSE<br class="">
                                                          2021-03-12
                                                          14:53:01.882
                                                          CET I (981732)
                                                          system_api:
                                                          Base MAC
                                                          address is not
                                                          set, read
                                                          default base
                                                          MAC address
                                                          from BLK0 of
                                                          EFUSE<br class="">
                                                          2021-03-12
                                                          14:53:01.902
                                                          CET I (981752)
                                                          wifi:wifi
                                                          firmware
                                                          version:
                                                          30f9e79<br class="">
                                                          2021-03-12
                                                          14:53:01.912
                                                          CET I (981762)
                                                          wifi:config
                                                          NVS flash:
                                                          enabled<br class="">
                                                          2021-03-12
                                                          14:53:01.912
                                                          CET I (981762)
                                                          wifi:config
                                                          nano
                                                          formating:
                                                          disabled<br class="">
                                                          2021-03-12
                                                          14:53:01.912
                                                          CET I (981762)
                                                          wifi:Init data
                                                          frame dynamic
                                                          rx buffer num:
                                                          16<br class="">
                                                          2021-03-12
                                                          14:53:01.912
                                                          CET I (981762)
                                                          wifi:Init
                                                          management
                                                          frame dynamic
                                                          rx buffer num:
                                                          16<br class="">
                                                          2021-03-12
                                                          14:53:01.922
                                                          CET I (981772)
                                                          wifi:Init
                                                          management
                                                          short buffer
                                                          num: 32<br class="">
                                                          2021-03-12
                                                          14:53:01.922
                                                          CET I (981772)
                                                          wifi:Init
                                                          dynamic tx
                                                          buffer num: 16<br class="">
                                                          2021-03-12
                                                          14:53:01.922
                                                          CET I (981772)
                                                          wifi:Init
                                                          static rx
                                                          buffer size:
                                                          2212<br class="">
                                                          2021-03-12
                                                          14:53:01.922
                                                          CET I (981772)
                                                          wifi:Init
                                                          static rx
                                                          buffer num: 16<br class="">
                                                          2021-03-12
                                                          14:53:01.922
                                                          CET I (981772)
                                                          wifi:Init
                                                          dynamic rx
                                                          buffer num: 16<br class="">
                                                          2021-03-12
                                                          14:53:02.642
                                                          CET I (982492)
                                                          wifi:mode :
                                                          sta
                                                          (30:ae:a4:5f:e7:ec)
                                                          + softAP
                                                          (30:ae:a4:5f:e7:ed)<br class="">
                                                          2021-03-12
                                                          14:53:02.652
                                                          CET I (982502)
                                                          wifi:Total
                                                          power save
                                                          buffer number:
                                                          8<br class="">
                                                          2021-03-12
                                                          14:53:02.652
                                                          CET I (982502)
cellular-modem-auto: Restart<br class="">
                                                          2021-03-12
                                                          14:53:02.662
                                                          CET I (982512)
                                                          cellular:
                                                          State: Enter
                                                          PowerOffOn
                                                          state<br class="">
                                                          2021-03-12
                                                          14:53:02.662
                                                          CET I (982512)
                                                          gsm-ppp:
                                                          Shutting down
                                                          (hard)...<br class="">
                                                          2021-03-12
                                                          14:53:02.662
                                                          CET I (982512)
                                                          gsm-ppp:
                                                          StatusCallBack:
                                                          User Interrupt<br class="">
                                                          2021-03-12
                                                          14:53:02.662
                                                          CET I (982512)
                                                          gsm-ppp: PPP
                                                          connection has
                                                          been closed<br class="">
                                                          2021-03-12
                                                          14:53:02.662
                                                          CET I (982512)
                                                          gsm-ppp: PPP
                                                          is shutdown<br class="">
                                                          2021-03-12
                                                          14:53:02.662
                                                          CET I (982512)
                                                          gsm-ppp:
                                                          Shutting down
                                                          (hard)...<br class="">
                                                          2021-03-12
                                                          14:53:02.672
                                                          CET I (982522)
                                                          gsm-ppp:
                                                          StatusCallBack:
                                                          User Interrupt<br class="">
                                                          2021-03-12
                                                          14:53:02.672
                                                          CET I (982522)
                                                          gsm-ppp: PPP
                                                          connection has
                                                          been closed<br class="">
                                                          2021-03-12
                                                          14:53:02.672
                                                          CET I (982522)
                                                          gsm-ppp: PPP
                                                          is shutdown<br class="">
                                                          2021-03-12
                                                          14:53:02.672
                                                          CET I (982522)
                                                          gsm-nmea:
                                                          Shutdown
                                                          (direct)<br class="">
                                                          2021-03-12
                                                          14:53:02.672
                                                          CET I (982522)
cellular-modem-auto: Power Cycle<br class="">
                                                          2021-03-12
                                                          14:53:04.682
                                                          CET D (984532)
                                                          events:
                                                          Signal(system.wifi.down)<br class="">
                                                          2021-03-12
                                                          14:53:04.682
                                                          CET I (984532)
                                                          netmanager:
                                                          WIFI client
                                                          stop<br class="">
                                                          2021-03-12
                                                          14:53:04.682
                                                          CET E (984532)
                                                          netmanager:
                                                          Inconsistent
                                                          state: no
                                                          interface of
                                                          type 'pp'
                                                          found<br class="">
                                                          2021-03-12
                                                          14:53:04.682
                                                          CET I (984532)
                                                          netmanager:
                                                          WIFI client
                                                          down (with
                                                          MODEM up):
                                                          reconfigured
                                                          for MODEM
                                                          priority<br class="">
                                                          2021-03-12
                                                          14:53:04.692
                                                          CET D (984542)
                                                          events:
                                                          Signal(system.event)<br class="">
                                                          2021-03-12
                                                          14:53:04.692
                                                          CET D (984542)
                                                          events:
                                                          Signal(system.wifi.sta.disconnected)<br class="">
                                                          2021-03-12
                                                          14:53:04.692
                                                          CET E (984542)
                                                          netmanager:
                                                          Inconsistent
                                                          state: no
                                                          interface of
                                                          type 'pp'
                                                          found<br class="">
                                                          2021-03-12
                                                          14:53:04.692
                                                          CET I (984542)
                                                          esp32wifi: STA
                                                          disconnected
                                                          with reason 8
                                                          = ASSOC_LEAVE<br class="">
                                                          2021-03-12
                                                          14:53:04.702
                                                          CET D (984552)
                                                          events:
                                                          Signal(system.event)<br class="">
                                                          2021-03-12
                                                          14:53:04.702
                                                          CET D (984552)
                                                          events:
                                                          Signal(system.wifi.sta.stop)<br class="">
                                                          2021-03-12
                                                          14:53:04.702
                                                          CET E (984552)
                                                          netmanager:
                                                          Inconsistent
                                                          state: no
                                                          interface of
                                                          type 'pp'
                                                          found<br class="">
                                                          2021-03-12
                                                          14:53:04.712
                                                          CET D (984562)
                                                          events:
                                                          Signal(system.event)<br class="">
                                                          2021-03-12
                                                          14:53:04.712
                                                          CET D (984562)
                                                          events:
                                                          Signal(system.wifi.ap.stop)<br class="">
                                                          2021-03-12
                                                          14:53:04.712
                                                          CET E (984562)
                                                          netmanager:
                                                          Inconsistent
                                                          state: no
                                                          interface of
                                                          type 'pp'
                                                          found<br class="">
                                                          2021-03-12
                                                          14:53:04.712
                                                          CET I (984562)
                                                          netmanager:
                                                          WIFI access
                                                          point is down<br class="">
                                                          2021-03-12
                                                          14:53:04.712
                                                          CET I (984562)
                                                          esp32wifi: AP
                                                          stopped<br class="">
                                                          2021-03-12
                                                          14:53:04.722
                                                          CET D (984572)
                                                          events:
                                                          Signal(network.wifi.sta.bad)<br class="">
                                                          2021-03-12
                                                          14:53:04.722
                                                          CET D (984572)
                                                          events:
                                                          Signal(system.event)<br class="">
                                                          2021-03-12
                                                          14:53:04.722
                                                          CET D (984572)
                                                          events:
                                                          Signal(system.wifi.sta.start)<br class="">
                                                          2021-03-12
                                                          14:53:04.732
                                                          CET D (984582)
                                                          events:
                                                          Signal(system.modem.down)<br class="">
                                                          2021-03-12
                                                          14:53:04.742
                                                          CET I (984592)
                                                          netmanager:
                                                          MODEM down
                                                          (with WIFI
                                                          client down):
                                                          network
                                                          connectivity
                                                          has been lost<br class="">
                                                          2021-03-12
                                                          14:53:04.742
                                                          CET D (984592)
                                                          events:
                                                          Signal(system.modem.down)<br class="">
                                                          2021-03-12
                                                          14:53:04.752
                                                          CET D (984602)
                                                          events:
                                                          Signal(system.event)<br class="">
                                                          2021-03-12
                                                          14:53:04.752
                                                          CET D (984602)
                                                          events:
                                                          Signal(system.wifi.ap.start)<br class="">
                                                          2021-03-12
                                                          14:53:04.752
                                                          CET I (984602)
                                                          netmanager:
                                                          WIFI access
                                                          point is up<br class="">
                                                          2021-03-12
                                                          14:53:26.802
                                                          CET E
                                                          (1006652)
                                                          events:
                                                          SignalEvent:
                                                          queue overflow
                                                          (running
                                                          system.wifi.ap.start->netmanager
                                                          for 23 sec),
                                                          event
                                                          'ticker.1'
                                                          dropped<br class="">
                                                          2021-03-12
                                                          14:53:27.802
                                                          CET E
                                                          (1007652)
                                                          events:
                                                          SignalEvent:
                                                          queue overflow
                                                          (running
                                                          system.wifi.ap.start->netmanager
                                                          for 24 sec),
                                                          event
                                                          'ticker.1'
                                                          dropped<br class="">
                                                          2021-03-12
                                                          14:53:28.802
                                                          CET E
                                                          (1008652)
                                                          events:
                                                          SignalEvent:
                                                          queue overflow
                                                          (running
                                                          system.wifi.ap.start->netmanager
                                                          for 25 sec),
                                                          event
                                                          'ticker.1'
                                                          dropped<br class="">
                                                          …and so on
                                                          until<br class="">
                                                          2021-03-12
                                                          14:54:01.802
                                                          CET E
                                                          (1041652)
                                                          events:
                                                          SignalEvent:
                                                          lost important
                                                          event =>
                                                          aborting<br class="">
                                                          <br class="">
                                                          <br class="">
                                                          I need my car
                                                          now, so will
                                                          switch back to
                                                          master for
                                                          now.<br class="">
                                                          <br class="">
                                                          Mark, if
                                                          you've got
                                                          specific debug
                                                          logs I should
                                                          fetch on the
                                                          next try, tell
                                                          me.<br class="">
                                                          <br class="">
                                                          Regards,<br class="">
                                                          Michael<br class="">
                                                          <br class="">
                                                          <br class="">
                                                          Am 12.03.21 um
                                                          05:47 schrieb
                                                          Craig Leres:<br class="">
                                                          <blockquote type="cite" class="">I
                                                          just updated
                                                          to
                                                          3.2.016-68-g8e10c6b7
                                                          and still get
                                                          the network
                                                          hang
                                                          immediately
                                                          after booting
                                                          and logging
                                                          into the web
                                                          gui.<br class="">
                                                          <br class="">
                                                          But I see now
                                                          my problem is
                                                          likely that
                                                          I'm not using
                                                          the right
                                                          esp-idf (duh).
                                                          Is there a way
                                                          I can have
                                                          master build
                                                          using
                                                          ~/esp/esp-idf
                                                          and have
                                                          for-v3.3 use a
                                                          different
                                                          path?)<br class="">
                                                          <br class="">
                                                                  Craig<br class="">
_______________________________________________<br class="">
                                                          OvmsDev
                                                          mailing list<br class="">
                                                          <a href="mailto:OvmsDev@lists.openvehicles.com" class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br class="">
                                                          <a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" class="" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br class="">
                                                          </blockquote>
                                                          <br class="">
                                                          -- <br class="">
                                                          Michael Balzer
                                                          * Helkenberger
                                                          Weg 9 *
                                                          D-58256
                                                          Ennepetal<br class="">
                                                          Fon 02333 /
                                                          833 5735 *
                                                          Handy 0176 /
                                                          206 989 26<br class="">
                                                          <br class="">
                                                          <br class="">
                                                          </div>
                                                          </div>
                                                          <br class="">
                                                          <iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg==" width="200" height="10"></iframe></div>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                  </div>
                                                  <br class="">
                                                </div>
                                                <br class="">
                                                <fieldset class="mimeAttachmentHeader"></fieldset>
                                                <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                                              </blockquote>
                                              <br class="">
                                              <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                                            </div>
                                          </div>
                                        </div>
                                        <br class="">
                                        <iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg==" width="200" height="10"></iframe></div>
                                    </div>
                                  </blockquote>
                                </div>
                                <br class="">
                              </div>
                              <br class="">
                              <fieldset class="mimeAttachmentHeader"></fieldset>
                              <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                            </blockquote>
                            <br class="">
                            <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                            <span class="">_______________________________________________</span><br class="">
                            <span class="">OvmsDev mailing list</span><br class="">
                            <span class=""><a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a></span><br class="">
                            <span class=""><a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a></span><br class="">
                          </div>
                        </blockquote>
                        <br class="">
                        <fieldset class="mimeAttachmentHeader"></fieldset>
                        <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                      </blockquote>
                      <br class="">
                      <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                    </div>
                  </div>
                </div>
                <br class="">
                <iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg==" width="200" height="10"></iframe></div>
            </div>
          </blockquote>
        </div>
        <br class="">
      </div>
      <br class="">
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
    </blockquote>
    <br class="">
    <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
  </div>

</div></div><br class=""><iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" width="200" height="10" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="></iframe></div></div></blockquote></div><br class=""></div></div></div></div><br class=""><iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" width="200" height="10" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="></iframe></div></div></blockquote></div><br class=""></div></div></div></div><br class=""><iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" width="200" height="10" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="></iframe></div></div></blockquote></div><br class=""></div></div></body></html>