Mark,

there really seems to be some issue with -285.

To help in tracking down release correlating issues I've just added another hourly developer report on my server:
https://dexters-web.de/f/firmware/developer/crashreport-overview.csv

That's basically the same source as…
https://dexters-web.de/f/firmware/developer/crashreport.csv
…just with reduced detail and more strict grouping.

Regards,
Michael


crashreport.csv:

  SELECT
    LEFT(h_data, INSTR(h_data,'/')-1) AS firmware_version,
    SUBSTRING_INDEX(SUBSTRING_INDEX(m_msg, ',', 5), ',', -1) AS car_type,
    COUNT(DISTINCT c.vehicleid) AS vehicle_count,
    COUNT(*) AS crashtype_count,
    CONVERT_TZ(MAX(h_timestamp),'+00:00','SYSTEM') AS last_time,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 12,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',13),',',-1), '') AS resetreason_name,
    SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',8),',',-1) AS crashtype,
    SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',9),',',-1) AS crashcore,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 13,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',14),',',-1), '') AS curr_event_name,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 13,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',15),',',-1), '') AS curr_event_handler,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 13,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',16),',',-1), '') AS curr_event_runtime,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 13,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',17),',',-1), '') AS wdt_tasknames,
    SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',11),',',-1) AS backtrace,
    SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',10),',',-1) AS registers,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 17,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',18),',',-1), '') AS hardware_info
  FROM ovms_historicalmessages h
  JOIN ovms_carmessages c ON ( c.vehicleid = h.vehicleid AND m_code = 'F' )
  WHERE
    h_recordtype = '*-OVM-DebugCrash' AND h_data LIKE '%build idf%'
    AND h_timestamp >= NOW() - INTERVAL 10 DAY
  GROUP BY firmware_version, car_type, resetreason_name, backtrace, curr_event_name, wdt_tasknames
  ORDER BY firmware_version, car_type, last_time


crashreport-overview.csv:

  SELECT
    SUBSTRING_INDEX(SUBSTRING_INDEX(m_msg, ',', 5), ',', -1) AS car_type,
    LEFT(h_data, INSTR(h_data,'/')-1) AS firmware_version,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 17,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',18),',',-1), '') AS hardware_info,
    IF(LENGTH(h_data) - LENGTH(REPLACE(h_data, ',', '')) >= 12,
      SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',13),',',-1), '') AS resetreason_name,
    SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',8),',',-1) AS crashtype,
    SUBSTRING_INDEX(SUBSTRING_INDEX(h_data,',',9),',',-1) AS crashcore,
    COUNT(DISTINCT c.vehicleid) AS vehicle_count,
    COUNT(*) AS crashtype_count,
    CONVERT_TZ(MAX(h_timestamp),'+00:00','SYSTEM') AS last_time
  FROM ovms_historicalmessages h
  JOIN ovms_carmessages c ON ( c.vehicleid = h.vehicleid AND m_code = 'F' )
  WHERE
    h_recordtype = '*-OVM-DebugCrash' AND h_data LIKE '%build idf%'
    AND h_timestamp >= NOW() - INTERVAL 10 DAY
  GROUP BY car_type, firmware_version, hardware_info, resetreason_name, crashtype, crashcore
  ORDER BY car_type, firmware_version, hardware_info, resetreason_name, crashtype, crashcore




Am 04.03.22 um 06:55 schrieb Mark Webb-Johnson:
Steve, Jeff,

I have noticed you running '3.3.001-285’. Can you try with the standard ‘3.3.001’ that was released a while ago? You can get it from:

http://api.openvehicles.com/firmware/ota/v3.3/main/ovms3.bin

or

http://api.openvehicles.com/firmware/ota/v3.2/edge/3.3.001.ovms3.bin

I wonder if something is broken in the latest edge?

P.S. I think that in general we may be confusing people here. The firmware/ota/<X> directory is for those version <X> mainboards. So /v3.1/, /v3.2/ or /v3.3/. The idea is that we can then have different build settings (sdkconfig) for different base hardware (in particular esp32 revision) and make improvements. At the moment v3.1 and v3.2 are symlinked together (on api.openvehicles.com) as ESP32 is the same.

Regards, Mark

On 4 Mar 2022, at 9:17 AM, Stephen Casner <casner@acm.org> wrote:

On Thu, 3 Mar 2022, Mark Webb-Johnson wrote:

Can you try to get a debug log for the cellular system?

Inline below.  This is with cellular, gsm-mux and SIM7600 set to
debug.  Any other useful tags?  Looks like it dies not long after
entering NetStart state.

                                                       -- Steve

2022-03-03 17:08:45.249 PST I (547059) cellular: Set modem driver to 'auto'
2022-03-03 17:08:45.249 PST I (547059) cellular: State: Enter PowerOffOn state
2022-03-03 17:08:45.249 PST I (547059) cellular-modem-auto: Power Cycle 2000ms
2022-03-03 17:08:47.539 PST D (549349) cellular: State timeout PowerOffOn => PoweringOn
2022-03-03 17:08:47.539 PST I (549349) cellular: State: Enter PoweringOn state
2022-03-03 17:08:47.539 PST I (549349) cellular-modem-auto: Power Cycle 4000ms
2022-03-03 17:08:51.539 PST D (553349) cellular: tx-cmd: AT
2022-03-03 17:08:51.539 PST D (553349) cellular: tx-cmd: AT
2022-03-03 17:08:51.539 PST D (553349) cellular: tx-cmd: AT
2022-03-03 17:08:51.539 PST D (553349) cellular: tx-cmd: AT
2022-03-03 17:08:52.539 PST D (554349) cellular: tx-cmd: AT
2022-03-03 17:08:53.539 PST D (555349) cellular: tx-cmd: AT
2022-03-03 17:08:54.539 PST D (556349) cellular: tx-cmd: AT
2022-03-03 17:08:55.539 PST D (557349) cellular: tx-cmd: AT
2022-03-03 17:08:56.539 PST D (558349) cellular: tx-cmd: AT
2022-03-03 17:08:57.539 PST D (559349) cellular: tx-cmd: AT
2022-03-03 17:08:58.539 PST D (560349) cellular: tx-cmd: AT
2022-03-03 17:08:59.539 PST D (561349) cellular: tx-cmd: AT
2022-03-03 17:09:00.539 PST D (562349) cellular: tx-cmd: AT
2022-03-03 17:09:01.539 PST D (563349) cellular: tx-cmd: AT
2022-03-03 17:09:02.539 PST D (564349) cellular: tx-cmd: AT
2022-03-03 17:09:02.979 PST I (564789) cellular: State: Enter Identify state
2022-03-03 17:09:03.539 PST D (565349) cellular: tx-cmd: AT+CGMM
2022-03-03 17:09:03.549 PST I (565359) cellular: Identified cellular modem: SIM7600/Experimental support for SIMCOM SIM7600
2022-03-03 17:09:03.549 PST D (565359) cellular: Remove old 'auto' modem driver
2022-03-03 17:09:03.549 PST I (565359) cellular: Set modem driver to 'SIM7600'
2022-03-03 17:09:03.549 PST I (565359) cellular: State: Enter PoweredOn state
2022-03-03 17:09:13.539 PST D (575349) cellular: tx-cmd: AT+CPIN?;+CREG=1;+CTZU=1;+CTZR=1;+CLIP=1;+CMGF=1;+CNMI=1,2,0,0,0;+CSDH=1;+CMEE=2;+CSQ;+AUTOCSQ=1,1;E0;S0=0
2022-03-03 17:09:13.569 PST D (575379) cellular: mux-rx-line #0: AT+CPIN?;+CREG=1;+CTZU=1;+CTZR=1;+CLIP=1;+CMGF=1;+CNMI=1,2,0,0,0;+CSDH=1;+CMEE=2;+CSQ;+AUTOCSQ=1,1;E0;S0=0
2022-03-03 17:09:13.569 PST D (575379) cellular: mux-rx-line #0: +CPIN: READY
2022-03-03 17:09:13.569 PST D (575379) cellular: mux-rx-line #0: +CSQ: 13,99
2022-03-03 17:09:13.579 PST I (575389) cellular: Signal Quality is: 13 (-87 dBm)
2022-03-03 17:09:15.539 PST D (577349) cellular: tx-cmd: AT+CGMR;+ICCID
2022-03-03 17:09:15.559 PST D (577369) cellular: mux-rx-line #0: +CGMR: LE20B03SIM7600M21-A
2022-03-03 17:09:15.559 PST D (577369) cellular: mux-rx-line #0: +ICCID: 8944501805175416178
2022-03-03 17:09:23.539 PST D (585349) cellular: tx-cmd: AT+CMUX=0
2022-03-03 17:09:23.549 PST I (585359) cellular: State: Enter MuxStart state
2022-03-03 17:09:23.549 PST I (585359) gsm-mux: Start MUX
2022-03-03 17:09:23.559 PST I (585369) gsm-mux: Channel #0 is open
2022-03-03 17:09:23.569 PST I (585379) gsm-mux: Channel #1 is open
2022-03-03 17:09:23.579 PST I (585389) gsm-mux: Channel #2 is open
2022-03-03 17:09:23.589 PST I (585399) gsm-mux: Channel #3 is open
2022-03-03 17:09:23.599 PST I (585409) gsm-mux: Channel #4 is open
2022-03-03 17:09:24.539 PST D (586349) cellular: State transition MuxStart => NetWait
2022-03-03 17:09:24.539 PST I (586349) cellular: State: Enter NetWait state
2022-03-03 17:09:34.539 PST D (596349) cellular: mux-tx #3: AT+CREG?;+CCLK?;+CSQ;+COPS?
2022-03-03 17:09:34.569 PST D (596379) cellular: mux-rx-line #3: +CREG: 1,5
2022-03-03 17:09:34.569 PST I (596379) cellular: Network Registration status: RegisteredRoaming
2022-03-03 17:09:34.569 PST D (596379) cellular: mux-rx-line #3: +CCLK: "22/03/03,17:09:32-32"
2022-03-03 17:09:34.569 PST D (596379) cellular: mux-rx-line #3: +CSQ: 13,99
2022-03-03 17:09:34.569 PST D (596379) cellular: mux-rx-line #3: +COPS: 0,0,"AT&T Hologram",2
2022-03-03 17:09:34.569 PST I (596379) cellular: Network Provider is: AT&T Hologram
2022-03-03 17:09:35.539 PST D (597349) cellular: State transition NetWait => NetStart
2022-03-03 17:09:35.539 PST I (597349) cellular: State: Enter NetStart state
2022-03-03 17:09:40.549 PST I (602359) housekeeping: 2022-03-03 17:09:39 PST (RAM: 8b=78500-79076 32b=12344 SPI=3439196-3442228)
2022-03-03 17:09:48.539 PST I (610349) ovms-server-v2: Send MP-0 S78.0,M,65535,127.00,done,standard,164,138,17,0,0,140,7,4,0,0,0,0,0.00,0,0,0,0,-1,0,0,0,0,0,0,0,0.00,0.00,0,0.00,0.00,1.00,0.0
2022-03-03 17:09:48.539 PST I (610349) ovms-server-v2: Send MP-0 D64,0,5,19,17,17,512,625508,0,95954,19,0,0,0,11.54,0,12.6,0,19,0,0
2022-03-03 17:09:48.549 PST I (610359) ovms-server-v2: Send MP-0 L37.346333,-122.037285,80,54,1,1,0.0,512,0,0.393,0,0,0,0,,0,0,0
2022-03-03 17:09:48.549 PST I (610359) ovms-server-v2: Send MP-0 F3.3.001-285-g601f2a70/ota_1/edge (build idf v3.3.4-848-g1ff5e24b1 Feb 20 2022 00:04:30),5YJRE11B181000033,30,0,TR1N,Revelstoke,-1,-1,OVMS WIFI BLE BT cores=2 rev=ESP32/1; MODEM SIM7600
2022-03-03 17:10:47.479 PST I (9639) command: OpenLogfile: now logging to file '/sd/crash.log'
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev


_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev


-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26