<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    Steve,<br>
    <br>
    writing the core dump to flash isn't possible. The only option to
    attach gdb is to…<br>
    <ol>
      <li>build with CONFIG_ESP32_PANIC_GDBSTUB enabled, then<br>
      </li>
      <li>produce the crash with "make monitor" running connected via
        USB.<br>
      </li>
    </ol>
    "make monitor" will run a (very) simple python terminal which checks
    the output for the GDBSTUB crash info and automatically invokes gdb.<br>
    <br>
    .elf files of public builds are available in the respective firmware
    folders on the servers.<br>
    <br>
    The xtensa toolkit has the full range of GCC tools, check out the
    "bin" directory.<br>
    <br>
    Regards,<br>
    Michael<br>
    <br>
    <br>
    <div class="moz-cite-prefix">Am 01.01.21 um 13:55 schrieb Steve
      Davies:<br>
    </div>
    <blockquote type="cite"
cite="mid:CABFTEGWdCJ6pT7E4hW6u4hGWmQNsSkdMFwWLMO5+KduJJQHM5Q@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div dir="ltr">Hi,
          <div><br>
          </div>
          <div>Thanks for the advice - managed to get backtraces but
            looks like I need to keep old ovms3.elfs.</div>
          <div><br>
          </div>
          <div>I didn't realise there is a full gdb - I found an option
            in 'make menuconfig" to write coredump to "flash" - will
            that work?  Is Is there enough room?</div>
          <div><br>
          </div>
          <div>If I can get a full core file I can scratch in the stack
            frames and perhaps better understand that went wrong.</div>
          <div><br>
          </div>
          <div>Steve</div>
          <div><br>
          </div>
          <div><br>
          </div>
        </div>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">On Fri, 1 Jan 2021 at 12:54,
            Michael Balzer <<a href="mailto:dexter@expeedo.de"
              moz-do-not-send="true">dexter@expeedo.de</a>> wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <div> Steve,<br>
              <br>
              decode your backtraces using the "a2l" script (see support
              folder) and the .elf file of the build the crash occured
              with. The script by default uses the latest .elf file.<br>
              <br>
              You can find the crash backtraces of previous crashes on
              the server in table "*-OVM-DebugCrash" (kept for 30 days).<br>
              <br>
              Known crash issues:
              <a
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues?q=is%3Aissue+is%3Aopen+crash"
                target="_blank" moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues?q=is%3Aissue+is%3Aopen+crash</a><br>
              <br>
              Any help with finding crash reasons is appreciated. I
              still am under the impression the PSRAM bug isn't solved
              for all edge cases, but also am certain we can find fatal
              bugs in our code as well.<br>
              <br>
              Regards,<br>
              Michael<br>
              <br>
              <br>
              <div>Am 01.01.21 um 08:47 schrieb Steve Davies:<br>
              </div>
              <blockquote type="cite">
                <div dir="ltr">Hi Ethan,
                  <div><br>
                  </div>
                  <div>Thanks for the info.  Naively doesn't seem like a
                    javascript program should be able to crash the whole
                    system.</div>
                  <div><br>
                  </div>
                  <div>I did think about rather posting to ABRP from the
                    server side - IE as data is sent to the server then
                    code on the server sends the update to ABRP.</div>
                  <div><br>
                  </div>
                  <div>But maybe I buried the lead, which really was to
                    find out how I can get a lower level view of what
                    crashed the board, and how to turn those memory
                    addresses into a better idea as to where in the code
                    the crash occurred.</div>
                  <div><br>
                  </div>
                  <div>Do you know, or Michael can you help?</div>
                  <div><br>
                  </div>
                  <div>Thanks,</div>
                  <div>Steve</div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                </div>
                <br>
                <div class="gmail_quote">
                  <div dir="ltr" class="gmail_attr">On Fri, 1 Jan 2021
                    at 03:11, Ethan Rose <<a
                      href="mailto:ethan@ethanrose.co.nz"
                      target="_blank" moz-do-not-send="true">ethan@ethanrose.co.nz</a>>
                    wrote:<br>
                  </div>
                  <blockquote class="gmail_quote" style="margin:0px 0px
                    0px 0.8ex;border-left:1px solid
                    rgb(204,204,204);padding-left:1ex">
                    <div dir="auto">I too have crashes often when
                      running the ABRP plugin on my NL ‘13. I’ve
                      resorted to using the LeafSpy/BT option instead on
                      long drives as it’s very unreliable. <br>
                      <br>
                      <div dir="ltr"><span>Regards,</span>
                        <div><span><br>
                          </span></div>
                        <div><span>Ethan Rose</span></div>
                      </div>
                      <div dir="ltr"><br>
                        <blockquote type="cite">On 1/01/2021, at 08:52,
                          Steve Davies <<a
                            href="mailto:steve@telviva.co.za"
                            target="_blank" moz-do-not-send="true">steve@telviva.co.za</a>>
                          wrote:<br>
                          <br>
                        </blockquote>
                      </div>
                      <blockquote type="cite">
                        <div dir="ltr">
                          <div dir="ltr">Well something had obviously
                            gone off the rails since eventually the OVMS
                            seemed to crash - couldn't ping it - and
                            didn't come back.
                            <div><br>
                            </div>
                            <div>I power-cycled it and it came back and
                              got a GPS lock.</div>
                            <div><br>
                            </div>
                            <div>Was there ever discussion of a watchdog
                              to reset things if it goes wonky?</div>
                            <div><br>
                            </div>
                            <div>Steve</div>
                          </div>
                          <br>
                          <div class="gmail_quote">
                            <div dir="ltr" class="gmail_attr">On Thu, 31
                              Dec 2020 at 19:37, Steve Davies <<a
                                href="mailto:steve@telviva.co.za"
                                target="_blank" moz-do-not-send="true">steve@telviva.co.za</a>>
                              wrote:<br>
                            </div>
                            <blockquote class="gmail_quote"
                              style="margin:0px 0px 0px
                              0.8ex;border-left:1px solid
                              rgb(204,204,204);padding-left:1ex">
                              <div dir="ltr">Hi,
                                <div><br>
                                </div>
                                <div><font face="monospace">I went for a
                                    scenic drive partly to test my OVMS
                                    I3 stuff and feeding data to ABRP.</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">When I got
                                    home I found that ABRP had my drive
                                    ending say 15km from home.  And when
                                    I checked OVMS it also thought I was
                                    in the same spot.</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">[6315S]
                                    v.p.altitude                        
                                        65.4m<br>
                                    [6315S] v.p.direction              
                                                 97.5°<br>
                                    [6315S] v.p.gpshdop                
                                                 1<br>
                                    [9999S] v.p.gpslock                
                                                 yes<br>
                                    [6315S] v.p.gpsmode                
                                                 AN<br>
                                    [6315S] v.p.gpsspeed                
                                                18.7052km/h<br>
                                    [6315S] v.p.latitude                
                                                -34.0808<br>
                                    [-----] v.p.location<br>
                                    [6315S] v.p.longitude              
                                                 18.4327<br>
                                    [5282S] v.p.odometer                
                                                14901km<br>
                                    [6315S] v.p.satcount                
                                                9<br>
                                    [5195S] v.p.speed                  
                                                 0km/h<br>
                                    [5195S] v.p.trip                    
                                                85km</font><br>
                                </div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">You can see
                                    that the metrics are very old (I
                                    show more digits in my version).</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">I then
                                    looked further and no metrics seemed
                                    to be updating - for instance the
                                    m.time.utc wasn't updating.</font></div>
                                <div><br>
                                </div>
                                <div><font face="monospace">I was
                                    starting to look at it when my OVMS
                                    rebooted (which seems to happen
                                    sometimes.  I've also had the
                                    logging stop - I noticed one less
                                    task so I presume that's related).</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <font face="monospace"><br>
                                </font><span
                                  style="font-family:monospace">Last
                                  boot was 127 second(s) ago</span><br>
                                <span style="font-family:monospace">Time
                                  at boot: 2020-12-31 19:10:09 SAST</span><br>
                                <span style="font-family:monospace">This
                                  is reset #8 since last power cycle</span><br>
                                <span style="font-family:monospace">Detected
                                  boot reason: Crash (12/12)</span><br>
                                <span style="font-family:monospace">Reset
                                  reason: Exception/panic (4)</span><br>
                                <span style="font-family:monospace">Crash
                                  counters: 1 total, 0 early</span><br>
                                <font face="monospace"><br>
                                </font><span
                                  style="font-family:monospace">Last
                                  crash: LoadProhibited exception on
                                  core 1</span><br>
                                <span style="font-family:monospace">Registers:</span><br>
                                <span style="font-family:monospace">PC :
                                  0x400f0161 PS : 0x00060a30 A0 :
                                  0x800f023a A1 : 0x3ffe5920</span><br>
                                <span style="font-family:monospace">A2 :
                                  0x3ffb5738 A3 : 0x00000000 A4 :
                                  0x3f8566a8 A5 : 0x00000001</span><br>
                                <span style="font-family:monospace">A6 :
                                  0x3ffe5990 A7 : 0x3f8567d0 A8 :
                                  0x800f015c A9 : 0x3ffe5910</span><br>
                                <span style="font-family:monospace">A10
                                  : 0x00000000 A11 : 0x3f414cb1 A12 :
                                  0x00000002 A13 : 0x00000001</span><br>
                                <span style="font-family:monospace">A14
                                  : 0x000000fe A15 : 0x3ffd4f90 SAR :
                                  0x00000004 EXCCAUSE: 0x0000001c</span><br>
                                <span style="font-family:monospace">EXCVADDR:
                                  0x00000000 LBEG : 0x4008bcf1 LEND :
                                  0x4008bd01 LCOUNT : 0xfffffffd</span><br>
                                <span style="font-family:monospace">Backtrace:</span><br>
                                <span style="font-family:monospace">0x400f0161
                                  0x400f0237 0x400fe87f 0x40103d4b
                                  0x40103f02 0x400fe8af 0x4013a432</span><br>
                                <div><font face="monospace">Version:
                                    3.2.015-267-g39dba375-dirty/ota_1/main
                                    (build idf
                                    v3.3.4-845-gd59ed8bba-dirty Dec 26
                                    2020 15:37:52)</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">After this
                                    crash and restart m.time.utc was
                                    empty. I rebooted and its still
                                    missing - does it only appear once
                                    NTP syncs or something?</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">Also I have
                                    no GPS position, gpsmode or anything
                                    like that (but I have a 3G session
                                    with the hologram sim and roaming on
                                    one of the local cellular
                                    operators).</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">I've made
                                    sure the GPS puck is out in the open
                                    and I guess I'll wait - does the
                                    time come from the GPS?  And any
                                    ideas why I'm not getting a GPS lock
                                    suddenly?</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">Apart from
                                    that - how do I debug crashes?  I
                                    went looking for some sort of loader
                                    memory map in the build directory to
                                    try turn the PC and backtrace
                                    addresses into something more useful
                                    but didn't pick up anything.</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace">Thanks</font></div>
                                <div><font face="monospace">Steve</font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                                <div><font face="monospace"><br>
                                  </font></div>
                              </div>
                            </blockquote>
                          </div>
                          <span>_______________________________________________</span><br>
                          <span>OvmsDev mailing list</span><br>
                          <span><a
                              href="mailto:OvmsDev@lists.openvehicles.com"
                              target="_blank" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a></span><br>
                          <span><a
                              href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
                              target="_blank" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a></span><br>
                        </div>
                      </blockquote>
                    </div>
                    _______________________________________________<br>
                    OvmsDev mailing list<br>
                    <a href="mailto:OvmsDev@lists.openvehicles.com"
                      target="_blank" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br>
                    <a
                      href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
                      rel="noreferrer" target="_blank"
                      moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br>
                  </blockquote>
                </div>
                <br>
                <fieldset></fieldset>
                <pre>_______________________________________________
OvmsDev mailing list
<a href="mailto:OvmsDev@lists.openvehicles.com" target="_blank" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" target="_blank" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
              </blockquote>
              <br>
              <pre cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
            </div>
            _______________________________________________<br>
            OvmsDev mailing list<br>
            <a href="mailto:OvmsDev@lists.openvehicles.com"
              target="_blank" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br>
            <a
              href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev"
              rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br>
          </blockquote>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
  </body>
</html>