[Ovmsdev] webserver (?) crash

Tue Mar 20 13:47:44 HKT 2018

Ok, interesting.  Am I seeing an out-of-memory crash?  I can't tell...

It seems like the webserver status button is the worst offender,
probably because it's doing a lot of work and building a big payload. 
Very often hangs for a while before responding, and if so, usually
results in the job queue overflow.  If I catch it quick enough and hit
home or some other shorter task, the overflow will drain and I'll get
the new screen; otherwise a crash.  Sometimes there will be a timeout
mixed in.  I'm wondering if our little squirrel in the ESP32 just can't
pedal hard enough...

I did some tests going between wifi and modem, and it seems to be better
than yesterday.  I am still having a lot of trouble with modem framing
errors.  I get only an update or two (25-35 frames), if that, before
they start mounting, eventually relying on the 3 minute timer to reset
things.  Thought last night that perhaps it was electrical interference,
since there's a 2.4 ghz transmitter screaming right under the modem
board.  But turning off wifi altogether (mode and power) didn't change
the framing errors, so that can't be it.  Perhaps I'm just in a bad (RF)
neighborhood.  The webserver status page (when I can get it) is
reporting -95 to -97 dBm for the signal.  Seems a bit low, but I don't
have a lot of experience in that department.  I'm just using an antenna
off my portable ham radio.  Supposed to be good to over a ghz, but not
ideal, to be sure.

There are a lot of "netmanager: interface priority is pp3" messages when
running the modem.  Is that netmanager offering encouragement to the
modem, or an indication of some status that might be bad?

I've switched over to the home wifi, and will let it run for the night,
but can load another build before heading to bed in an hour or so, if
there is one.

Greg

Mark Webb-Johnson wrote:
> Seems to be ok for me. But I have 48KB RAM free (OVMS v3.0) which gives a bit more breathing room.
>
> I’ve been doing a lot of bringing up and down wifi and modem links, and switching between ap, apclient, client, and off wifi modes. It seems that the network manager is behaving better now (at least on my desk). That bug of taking down mongoose would have caused unpredictable behaviour and weird errors everywhere.
>
> I’m going to put it in my car now, and leave it running this afternoon (and for my drive home).
>
> Regards, Mark.
>
>> On 20 Mar 2018, at 12:53 PM, Greg D. <gregd2350 at gmail.com> wrote:
>>
>> Hi folks,
>>
>> Ok, so the new code seems to be working half-way decently.  But I think
>> there's a memory issue with the webserver.  After poking at home and
>> status a couple of times, my browser (chrome or Firefox) drop from a
>> nicely formatted screen to something very text-like.  Links and words,
>> no buttons.  Then another poke or two and the module crashed.  I've done
>> it a few times, and the crash usually seem to be with the Status button,
>> which, looking at the console, seems to have an auto-repeat on it.  I
>> managed to get the repeated "job queue overflow" once, followed after
>> hitting home a few times, a similar crash as below.
>>
>> This was one of the quicker crashes (browser didn't get to the text-mode
>> state), using AP mode wifi.  Browser in this case was Chrome.  I have a
>> suspicion that the issue with the text-mode stuff is a leak somewhere
>> triggered by Firefox, with its CSS fetches.
>>
>> Greg
>>
>> OVMS>
>> OVMS>
>> I (12641) housekeeping: System considered stable (free: 35212 bytes)
>> I (17331) simcom: State: Enter PoweredOn state
>> I (30481) wifi: n:1 0, o:1 1, ap:1 1, sta:1 0, prof:1
>> I (30491) wifi: station: 40:4e:36:8a:44:c0 join, AID=1, g, 20
>> I (30491) esp32wifi: AP station connected: id: 1, MAC: 40:4e:36:8a:44:c0
>> I (37121) webserver: HTTP GET /home
>> I (37371) simcom: State: Enter MuxStart state
>> I (37371) gsm-mux: Start MUX
>> I (40411) webserver: HTTP GET /status
>> I (47611) webserver: HTTP GET /shell
>> I (51031) webserver: HTTP GET /home
>> I (53891) webserver: HTTP GET /status
>> I (59031) webserver: HTTP GET /home
>> I (61581) webserver: HTTP GET /
>> I (61661) webserver: HTTP GET /home
>> I (63511) webserver: HTTP GET /home
>> I (65971) webserver: HTTP GET /status
>> OVMS> abort() was called at PC 0x400db927 on core 1
>>
>> Backtrace: 0x4008f0b8:0x3ffeb8b0 0x4008f28f:0x3ffeb8d0
>> 0x400db927:0x3ffeb8f0 0x4014880c:0x3ffeb910 0x40154899:0x3ffeb930
>> 0x40154a9f:0x3ffeb950 0x40126564:0x3ffeb980 0x401267c7:0x3ffeb9e0
>> 0x40126846:0x3ffeba00 0x4012841f:0x3ffeba50 0x400f6aff:0x3ffebb10
>> 0x400f7262:0x3ffebb40 0x400f6aff:0x3ffebb60 0x400f7c8e:0x3ffebb90
>> 0x400f7d3f:0x3ffebbc0 0x400f80eb:0x3ffebbf0 0x400f834d:0x3ffebc30
>> 0x400f4b2d:0x3ffebc80 0x400e749a:0x3ffebca0 0x400e74d9:0x3ffebcf0
>>
>> Rebooting...
>> ets Jun  8 2016 00:22:57
>>
>> rst:0xc (SW_CPU_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
>> configsip: 156795334, SPIWP:0xee
>> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
>> mode:DIO, clock div:2
>> load:0x3fff0018,len:4
>> load:0x3fff001c,len:4560
>> ho 0 tail 12 room 4
>> load:0x40078000,len:0
>> load:0x40078000,len:13176
>> entry 0x40078d38
>> I (647) cpu_start: Pro cpu up.
>> I (648) cpu_start: Starting app cpu, entry point is 0x40081300
>> I (632) cpu_start: App cpu up.
>> I (651) heap_init: Initializing. RAM available for dynamic allocation:
>> I (658) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
>> I (664) heap_init: At 3FFBBAC0 len 00024540 (145 KiB): DRAM
>> I (670) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
>> I (676) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
>> I (684) heap_init: At 40091FDC len 0000E024 (56 KiB): IRAM
>> I (689) cpu_start: Pro cpu start user code
>> I (35) ovms_main: Set default logging level for * to INFO
>> I (36) command: Initialising COMMAND (1000)
>> I (36) boot: Initialising BOOT (1100)
>> I (40) boot: Boot #4 reasons for CPU0=12 and CPU1=12
>> E (46) boot: Crash #4 detected
>> I (49) events: Initialising EVENTS (1200)
>> I (54) config: Initialising CONFIG (1400)
>> I (59) time: Initialising TIME (1500)
>> I (64) script: Initialising SCRIPTS (1600)
>> I (68) script: Using DUKTAPE javascript engine
>> I (74) metrics: Initialising METRICS (1810)
>> I (78) metrics: Expanding DUKTAPE javascript engine
>> I (88) notify: Initialising NOTIFICATIONS (1820)
>> I (89) notify: Registered notification type info
>> I (94) notify: Registered notification type error
>> I (99) notify: Registered notification type alert
>> I (105) notify: Registered notification type data
>> I (110) location: Initialising LOCATIONS (1900)
>> I (116) location: Expanding DUKTAPE javascript engine
>> I (121) vehicle: Initialising VEHICLE Factory (2000)
>> I (127) pcp: Initialising POWER (4000)
>> I (131) max7317: Initialising MAX7317 EGPIO (4200)
>> I (137) sdcard: Initialising SD CARD (4400)
>> I (142) ota: Initialising OTA (4400)
>> I (146) can: Initialising CAN (4500)
>> I (151) simcom: Initialising SIMCOM (4600)
>> I (155) test: Initialising TEST (5000)
>> I (159) ovms-module: Initialising MODULE (5100)
>> I (165) vfs: Initialising VFS (5200)
>> I (169) ovms-server: Initialising OVMS Server (6000)
>> I (174) ovms-server-v2: Initialising OVMS V2 Server (6100)
>> I (181) ovms-server-v3: Initialising OVMS V3 Server (6200)
>> I (187) obd2ecu: Initialising OBD2ECU (7000)
>> I (192) canopen: Initialising CANopen (7000)
>> I (197) esp32wifi: Initialising ESP32WIFI (8000)
>> I (202) ovms-mdns: Initialising MDNS (8100)
>> I (207) webserver: Initialising WEBSERVER (8200)
>> I (213) telnet: Initialising Telnet (8300)
>> I (217) ssh: Initialising SSH (8300)
>> I (221) re: Initialising RE Tools (8800)
>> I (226) netmanager: Initialising NETMANAGER (8999)
>> I (233) v-track: Registering Vehicle: TRACK (9000)
>> I (237) v-teslaroadster: Registering Vehicle: Tesla Roadster (9000)
>> I (243) v-teslamodels: Registering Vehicle: Tesla Model S (9000)
>> I (250) v-obdii: Registering Vehicle: OBDII (9000)
>> I (256) v-none: Registering Vehicle: NONE (9000)
>> I (261) v-demo: Registering Vehicle: DEMO (9000)
>> I (266) version: Initialising Versioning (9900)
>> I (275) cpu_start: Starting scheduler on PRO CPU.
>> I (0) cpu_start: Starting scheduler on APP CPU.
>> I (321) ovms_main: Executing on CPU core 0
>> I (321) ovms_main: Mounting CONFIG...
>> W (371) webserver: UpdateGlobalAuthFile: no password set => no auth for
>> web console
>> I (381) ovms_main: Registering default configs...
>> I (381) ovms_main: Starting HOUSEKEEPING...
>> I (381) housekeeping: Initialising HOUSEKEEPING Framework...
>> I (441) housekeeping: Executing on CPU core 1
>> I (441) housekeeping: reset_reason: cpu0=12, cpu1=12
>> I (441) housekeeping: Starting PERIPHERALS...
>> I (451) peripherals: Initialising OVMS Peripherals...
>> I (451) peripherals:   TCP/IP Adaptor
>> I (461) peripherals:   ESP32 system
>> I (461) peripherals:   SPI bus
>> I (461) peripherals:   MAX7317 I/O Expander
>> I (471) peripherals:   ESP32 CAN
>> I (471) peripherals:   ESP32 WIFI
>> I (481) peripherals:   ESP32 BLUETOOTH
>> I (481) peripherals:   ESP32 ADC
>> I (481) peripherals:   MCP2515 CAN 1/2
>> I (501) peripherals:   MCP2515 CAN 2/2
>> I (511) peripherals:   SD CARD
>> I (511) peripherals:   SIMCOM MODEM
>> I (511) gpio: GPIO[16]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup:
>> 1| Pulldown: 0| Intr:0
>> I (521) gpio: GPIO[17]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup:
>> 1| Pulldown: 0| Intr:0
>> I (531) uart: queue free spaces: 10
>> I (531) ext12v: Powering off external 12V devices
>> I (541) housekeeping: Auto init ext12v (free: 112712 bytes)
>> I (541) ext12v: Powering on external 12V devices
>> I (551) housekeeping: Auto init wifi (free: 112712 bytes)
>> I (551) wifi: wifi firmware version: ebd3e5d
>> I (561) wifi: config NVS flash: enabled
>> I (561) wifi: config nano formating: disabled
>> I (561) system_api: Base MAC address is not set, read default base MAC
>> address from BLK0 of EFUSE
>> I (571) system_api: Base MAC address is not set, read default base MAC
>> address from BLK0 of EFUSE
>> I (601) wifi: Init dynamic tx buffer num: 16
>> I (601) wifi: Init data frame dynamic rx buffer num: 16
>> I (601) wifi: Init management frame dynamic rx buffer num: 16
>> I (601) wifi: wifi driver task: 3ffe60f0, prio:23, stack:4096
>> I (611) wifi: Init static rx buffer num: 4
>> I (611) wifi: Init dynamic rx buffer num: 16
>> I (611) wifi: wifi power manager task: 0x3ffe8888 prio: 21 stack: 2560
>> I (1341) phy: phy_version: 383.0, 79a622c, Jan 30 2018, 15:38:06, 0, 0
>> I (1341) wifi: mode : sta (30:ae:a4:37:1b:64) + softAP (30:ae:a4:37:1b:65)
>> I (1351) housekeeping: Auto init modem (free: 90568 bytes)
>> I (1351) simcom: State: Enter PoweringOn state
>> I (1351) simcom: Power Cycle
>> I (1361) netmanager: WIFI access point is up
>> I (1361) webserver: Launching Web Server
>> I (1371) esp32wifi: AP started with SSID: ovms, MAC: 30:ae:a4:37:1b:65,
>> IP: 192.168.4.1
>> I (1371) telnet: Launching Telnet Server
>> I (1381) ssh: Launching SSH Server
>> I (1441) simcom: State: Enter PoweredOn state
>> I (2361) housekeeping: Auto init vehicle (free: 78820 bytes)
>> I (2361) v-teslaroadster: Tesla Roadster v1.x, v2.x and v3.0 vehicle module
>> I (2371) housekeeping: Auto init obd2ecu (free: 73088 bytes)
>> I (2571) housekeeping: Auto init server v2 (free: 64432 bytes)
>> I (2571) ovms-server-v2: OVMS Server V2 registered metric modifier is #1
>> I (2581) ovms-server-v2: Status: Starting
>> I (2581) ovms-server-v2: OVMS Server v2 running
>> I (2591) housekeeping: Auto init server v3 (free: 60120 bytes)
>> I (2591) housekeeping: Auto init done (free: 60120 bytes)
>> I (2601) housekeeping: Starting USB console...
>> I (2601) uart: queue free spaces: 30
>> I (2611) ovms-mdns: Starting MDNS
>>
>> Welcome to the Open Vehicle Monitoring System (OVMS) - Async Console
>> I (2651) version: Set version
>> I (4181) wifi: n:1 1, o:1 0, ap:1 1, sta:1 0, prof:1
>> I (4841) wifi: state: init -> auth (b0)
>> I (4841) wifi: state: auth -> auth (4a0)
>> OVMS>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.teslaclub.hk
>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.teslaclub.hk
> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev