Attempting to ssh in too soon after a reboot causes an abort() from esp_sha_read_digest_state()
[third attempt, apparently my opendkim sendmail milter was borked and needed to be restarted...] I updated to 3.3.003-518-gb6c10a59 a few days ago and noticed if I'm too impatient trying to ssh into the module after rebooting, it crashes. I suspect this is easy to reproduce. Craig =============================================================== OVMS# boot Last boot was 501 second(s) ago Time at boot: 2023-06-21 19:49:18 PDT This is reset #8 since last power cycle Detected boot reason: EarlyCrash (12/12) Reset reason: Exception/panic (4) Crash counters: 1 total, 1 early Last crash: abort() was called on core 1 Current task on core 0: IDLE0, 524 stack bytes free Current task on core 1: OVMS NetMan, 1208 stack bytes free Backtrace: 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 Version: 3.3.003-518-gb6c10a59-dirty/ota_1/main (build idf v3.3.4-849-g6e214dc33 Jun 20 2023 21:30:05) Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600 =============================================================== ice 104 % git describe 3.3.003-518-gb6c10a59 ice 105 % cat backtrace.sh #!/bin/sh PATH=/usr/local/xtensa-esp32-elf/bin:${PATH} export PATH cd ${HOME}/src/OVMS.V3 || exit 1 dn=`/bin/pwd` (set -x ; xtensa-esp32-elf-addr2line -e build/ovms3.elf $*) | sed -e s,^${dn}/,, ice 106 % !./ ./backtrace.sh 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 + xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/hwcrypto/sha.c:272 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/port/esp_sha256.c:349 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_cli.c:3894 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 main/ovms_netmanager.cpp:1198 main/ovms_netmanager.cpp:1198
Hi Craig, FYI I've received the 3 attempts on the list without any issue... :-) Le 25/06/2023 à 19:24, Craig Leres a écrit :
[third attempt, apparently my opendkim sendmail milter was borked and needed to be restarted...]
I updated to 3.3.003-518-gb6c10a59 a few days ago and noticed if I'm too impatient trying to ssh into the module after rebooting, it crashes. I suspect this is easy to reproduce.
Craig
===============================================================
OVMS# boot Last boot was 501 second(s) ago Time at boot: 2023-06-21 19:49:18 PDT This is reset #8 since last power cycle Detected boot reason: EarlyCrash (12/12) Reset reason: Exception/panic (4) Crash counters: 1 total, 1 early
Last crash: abort() was called on core 1 Current task on core 0: IDLE0, 524 stack bytes free Current task on core 1: OVMS NetMan, 1208 stack bytes free Backtrace: 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 Version: 3.3.003-518-gb6c10a59-dirty/ota_1/main (build idf v3.3.4-849-g6e214dc33 Jun 20 2023 21:30:05)
Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
===============================================================
ice 104 % git describe 3.3.003-518-gb6c10a59 ice 105 % cat backtrace.sh #!/bin/sh PATH=/usr/local/xtensa-esp32-elf/bin:${PATH} export PATH cd ${HOME}/src/OVMS.V3 || exit 1 dn=`/bin/pwd` (set -x ; xtensa-esp32-elf-addr2line -e build/ovms3.elf $*) | sed -e s,^${dn}/,, ice 106 % !./ ./backtrace.sh 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 + xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/hwcrypto/sha.c:272
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/port/esp_sha256.c:349
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_cli.c:3894
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649
/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649
components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 main/ovms_netmanager.cpp:1198 main/ovms_netmanager.cpp:1198
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Craig, I tried to repeat this, without success. Your backtrace seems to implicate sha256 hashing, in the ssl_tls call chain, but that doesn’t make much sense for ssh. Maybe memory corruption? Is anything else using ssl in your config? Regards, Mark.
On 26 Jun 2023, at 1:24 AM, Craig Leres <leres@xse.com> wrote:
[third attempt, apparently my opendkim sendmail milter was borked and needed to be restarted...]
I updated to 3.3.003-518-gb6c10a59 a few days ago and noticed if I'm too impatient trying to ssh into the module after rebooting, it crashes. I suspect this is easy to reproduce.
Craig
===============================================================
OVMS# boot Last boot was 501 second(s) ago Time at boot: 2023-06-21 19:49:18 PDT This is reset #8 since last power cycle Detected boot reason: EarlyCrash (12/12) Reset reason: Exception/panic (4) Crash counters: 1 total, 1 early
Last crash: abort() was called on core 1 Current task on core 0: IDLE0, 524 stack bytes free Current task on core 1: OVMS NetMan, 1208 stack bytes free Backtrace: 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 Version: 3.3.003-518-gb6c10a59-dirty/ota_1/main (build idf v3.3.4-849-g6e214dc33 Jun 20 2023 21:30:05)
Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
===============================================================
ice 104 % git describe 3.3.003-518-gb6c10a59 ice 105 % cat backtrace.sh #!/bin/sh PATH=/usr/local/xtensa-esp32-elf/bin:${PATH} export PATH cd ${HOME}/src/OVMS.V3 || exit 1 dn=`/bin/pwd` (set -x ; xtensa-esp32-elf-addr2line -e build/ovms3.elf $*) | sed -e s,^${dn}/,, ice 106 % !./ ./backtrace.sh 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 + xtensa-esp32-elf-addr2line -e build/ovms3.elf 0x400891b7 0x40089451 0x40268a11 0x40244aea 0x4023e892 0x40240287 0x4024f6bb 0x40240556 0x40240581 0x4012aaa0 0x4012c150 0x4012d89d 0x4012ddc3 0x4012a2aa 0x4011e2eb 0x4011e389 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/panic.c:736 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/esp32/hwcrypto/sha.c:272 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/port/esp_sha256.c:349 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_cli.c:3894 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 /home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/components/mbedtls/mbedtls/library/ssl_tls.c:7649 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 components/mongoose/mongoose/mongoose.c:1701 main/ovms_netmanager.cpp:1198 main/ovms_netmanager.cpp:1198
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
participants (3)
-
Craig Leres -
Ludovic LANGE -
Mark Webb-Johnson