[Ovmsdev] webserver (?) crash

Tue Mar 20 15:49:11 HKT 2018

Yes, RAM on hw3.0 is very tight now. Greg, you can save around 7 KB by not inserting an SD card.

My normal free RAM is around 22 KB after mounting an SD card, which normally still works if I take care not to stress it by additional web /
telnet sessions.

Greg, if you enable debug log level for "webserver" and "webcommand" you will get free RAM log entries on pages and command calls.

I'll try the latest changes as soon as I get home.

Regards,
Michael

Am 20.03.2018 um 07:10 schrieb Mark Webb-Johnson:
>
> I’m a little worried by 30KB RAM free. That is tight, and it doesn’t take much temporary buffers for output to cause it to run out of ram. I
> did try it just now in my car, and it seemed ok for me - but I was accessing it via AP from my iPhone.
>
> Anyway, Michael knows the webserver, so best to let him try it now.
>
> Framing errors would be indicative of a mux issue, not really RF signal. Are you sure the modem is not browning out from lack of power? Are
> you seeing any ..START.. messages in the log (which is a modem restart)? I was getting a lot of these until I switched to a powered USB hub.
> How are you powering this - 5v usb or 12v?
>
> Don’t worry about the net manager interface priority messages. Those are there because the wifi stack keeps screwing with the default
> interface, so I work around it by re-setting the default interface pretty much every time there is wifi ap/sta event. For example:
>
>  1. Start and Access Point + Station
>  2. Start a modem
>  3. Wait for everything to connect. Netmanager priority will be Station.
>  4. Shutdown access point and station. Netmanager will switch to modem.
>  5. Start a Station. Netmanager will switch to station.
>  6. (at this point, everything is fine - but network status shows a ‘dead’ AP interface left lying around unused)
>  7. Stop the station. Netmanager will switch to modem.
>  8. A second or so later, the wifi stack will shutdown the station interface, and say ‘oh look, there is an AP interface, let’s switch to
>     that’. Server v2 can’t connect any more.
>  9. Now, with my new code, Netmanager will pick up on this and switch to modem.
>
>
> If I have time, I might change the Netmanager code to only output the priority message if it actually has to change the interface.
>
> I don’t have any plans for changes in the next few hours. At work now, lunch time over, and day job beckons.
>
> Regards, Mark.
>
>> On 20 Mar 2018, at 1:47 PM, Greg D. <gregd2350 at gmail.com <mailto:gregd2350 at gmail.com>> wrote:
>>
>> Ok, interesting.  Am I seeing an out-of-memory crash?  I can't tell...
>>
>> It seems like the webserver status button is the worst offender,
>> probably because it's doing a lot of work and building a big payload.
>> Very often hangs for a while before responding, and if so, usually
>> results in the job queue overflow.  If I catch it quick enough and hit
>> home or some other shorter task, the overflow will drain and I'll get
>> the new screen; otherwise a crash.  Sometimes there will be a timeout
>> mixed in.  I'm wondering if our little squirrel in the ESP32 just can't
>> pedal hard enough...
>>
>> I did some tests going between wifi and modem, and it seems to be better
>> than yesterday.  I am still having a lot of trouble with modem framing
>> errors.  I get only an update or two (25-35 frames), if that, before
>> they start mounting, eventually relying on the 3 minute timer to reset
>> things.  Thought last night that perhaps it was electrical interference,
>> since there's a 2.4 ghz transmitter screaming right under the modem
>> board.  But turning off wifi altogether (mode and power) didn't change
>> the framing errors, so that can't be it.  Perhaps I'm just in a bad (RF)
>> neighborhood.  The webserver status page (when I can get it) is
>> reporting -95 to -97 dBm for the signal.  Seems a bit low, but I don't
>> have a lot of experience in that department.  I'm just using an antenna
>> off my portable ham radio.  Supposed to be good to over a ghz, but not
>> ideal, to be sure.
>>
>> There are a lot of "netmanager: interface priority is pp3" messages when
>> running the modem.  Is that netmanager offering encouragement to the
>> modem, or an indication of some status that might be bad?
>>
>> I've switched over to the home wifi, and will let it run for the night,
>> but can load another build before heading to bed in an hour or so, if
>> there is one.
>>
>> Greg
>>
>>
>> Mark Webb-Johnson wrote:
>>> Seems to be ok for me. But I have 48KB RAM free (OVMS v3.0) which gives a bit more breathing room.
>>>
>>> I’ve been doing a lot of bringing up and down wifi and modem links, and switching between ap, apclient, client, and off wifi modes. It seems
>>> that the network manager is behaving better now (at least on my desk). That bug of taking down mongoose would have caused unpredictable
>>> behaviour and weird errors everywhere.
>>>
>>> I’m going to put it in my car now, and leave it running this afternoon (and for my drive home).
>>>
>>> Regards, Mark.
>>>
>>>> On 20 Mar 2018, at 12:53 PM, Greg D. <gregd2350 at gmail.com <mailto:gregd2350 at gmail.com>> wrote:
>>>>
>>>> Hi folks,
>>>>
>>>> Ok, so the new code seems to be working half-way decently.  But I think
>>>> there's a memory issue with the webserver.  After poking at home and
>>>> status a couple of times, my browser (chrome or Firefox) drop from a
>>>> nicely formatted screen to something very text-like.  Links and words,
>>>> no buttons.  Then another poke or two and the module crashed.  I've done
>>>> it a few times, and the crash usually seem to be with the Status button,
>>>> which, looking at the console, seems to have an auto-repeat on it.  I
>>>> managed to get the repeated "job queue overflow" once, followed after
>>>> hitting home a few times, a similar crash as below.
>>>>
>>>> This was one of the quicker crashes (browser didn't get to the text-mode
>>>> state), using AP mode wifi.  Browser in this case was Chrome.  I have a
>>>> suspicion that the issue with the text-mode stuff is a leak somewhere
>>>> triggered by Firefox, with its CSS fetches.
>>>>
>>>> Greg
>>>>
>>>> OVMS>
>>>> OVMS>
>>>> I (12641) housekeeping: System considered stable (free: 35212 bytes)
>>>> I (17331) simcom: State: Enter PoweredOn state
>>>> I (30481) wifi: n:1 0, o:1 1, ap:1 1, sta:1 0, prof:1
>>>> I (30491) wifi: station: 40:4e:36:8a:44:c0 join, AID=1, g, 20
>>>> I (30491) esp32wifi: AP station connected: id: 1, MAC: 40:4e:36:8a:44:c0
>>>> I (37121) webserver: HTTP GET /home
>>>> I (37371) simcom: State: Enter MuxStart state
>>>> I (37371) gsm-mux: Start MUX
>>>> I (40411) webserver: HTTP GET /status
>>>> I (47611) webserver: HTTP GET /shell
>>>> I (51031) webserver: HTTP GET /home
>>>> I (53891) webserver: HTTP GET /status
>>>> I (59031) webserver: HTTP GET /home
>>>> I (61581) webserver: HTTP GET /
>>>> I (61661) webserver: HTTP GET /home
>>>> I (63511) webserver: HTTP GET /home
>>>> I (65971) webserver: HTTP GET /status
>>>> OVMS> abort() was called at PC 0x400db927 on core 1
>>>>
>>>> Backtrace: 0x4008f0b8:0x3ffeb8b0 0x4008f28f:0x3ffeb8d0
>>>> 0x400db927:0x3ffeb8f0 0x4014880c:0x3ffeb910 0x40154899:0x3ffeb930
>>>> 0x40154a9f:0x3ffeb950 0x40126564:0x3ffeb980 0x401267c7:0x3ffeb9e0
>>>> 0x40126846:0x3ffeba00 0x4012841f:0x3ffeba50 0x400f6aff:0x3ffebb10
>>>> 0x400f7262:0x3ffebb40 0x400f6aff:0x3ffebb60 0x400f7c8e:0x3ffebb90
>>>> 0x400f7d3f:0x3ffebbc0 0x400f80eb:0x3ffebbf0 0x400f834d:0x3ffebc30
>>>> 0x400f4b2d:0x3ffebc80 0x400e749a:0x3ffebca0 0x400e74d9:0x3ffebcf0
>>>>
>>>> Rebooting...
>>>> ets Jun  8 2016 00:22:57
>>>>
>>>> rst:0xc (SW_CPU_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
>>>> configsip: 156795334, SPIWP:0xee
>>>> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
>>>> mode:DIO, clock div:2
>>>> load:0x3fff0018,len:4
>>>> load:0x3fff001c,len:4560
>>>> ho 0 tail 12 room 4
>>>> load:0x40078000,len:0
>>>> load:0x40078000,len:13176
>>>> entry 0x40078d38
>>>> I (647) cpu_start: Pro cpu up.
>>>> I (648) cpu_start: Starting app cpu, entry point is 0x40081300
>>>> I (632) cpu_start: App cpu up.
>>>> I (651) heap_init: Initializing. RAM available for dynamic allocation:
>>>> I (658) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
>>>> I (664) heap_init: At 3FFBBAC0 len 00024540 (145 KiB): DRAM
>>>> I (670) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
>>>> I (676) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
>>>> I (684) heap_init: At 40091FDC len 0000E024 (56 KiB): IRAM
>>>> I (689) cpu_start: Pro cpu start user code
>>>> I (35) ovms_main: Set default logging level for * to INFO
>>>> I (36) command: Initialising COMMAND (1000)
>>>> I (36) boot: Initialising BOOT (1100)
>>>> I (40) boot: Boot #4 reasons for CPU0=12 and CPU1=12
>>>> E (46) boot: Crash #4 detected
>>>> I (49) events: Initialising EVENTS (1200)
>>>> I (54) config: Initialising CONFIG (1400)
>>>> I (59) time: Initialising TIME (1500)
>>>> I (64) script: Initialising SCRIPTS (1600)
>>>> I (68) script: Using DUKTAPE javascript engine
>>>> I (74) metrics: Initialising METRICS (1810)
>>>> I (78) metrics: Expanding DUKTAPE javascript engine
>>>> I (88) notify: Initialising NOTIFICATIONS (1820)
>>>> I (89) notify: Registered notification type info
>>>> I (94) notify: Registered notification type error
>>>> I (99) notify: Registered notification type alert
>>>> I (105) notify: Registered notification type data
>>>> I (110) location: Initialising LOCATIONS (1900)
>>>> I (116) location: Expanding DUKTAPE javascript engine
>>>> I (121) vehicle: Initialising VEHICLE Factory (2000)
>>>> I (127) pcp: Initialising POWER (4000)
>>>> I (131) max7317: Initialising MAX7317 EGPIO (4200)
>>>> I (137) sdcard: Initialising SD CARD (4400)
>>>> I (142) ota: Initialising OTA (4400)
>>>> I (146) can: Initialising CAN (4500)
>>>> I (151) simcom: Initialising SIMCOM (4600)
>>>> I (155) test: Initialising TEST (5000)
>>>> I (159) ovms-module: Initialising MODULE (5100)
>>>> I (165) vfs: Initialising VFS (5200)
>>>> I (169) ovms-server: Initialising OVMS Server (6000)
>>>> I (174) ovms-server-v2: Initialising OVMS V2 Server (6100)
>>>> I (181) ovms-server-v3: Initialising OVMS V3 Server (6200)
>>>> I (187) obd2ecu: Initialising OBD2ECU (7000)
>>>> I (192) canopen: Initialising CANopen (7000)
>>>> I (197) esp32wifi: Initialising ESP32WIFI (8000)
>>>> I (202) ovms-mdns: Initialising MDNS (8100)
>>>> I (207) webserver: Initialising WEBSERVER (8200)
>>>> I (213) telnet: Initialising Telnet (8300)
>>>> I (217) ssh: Initialising SSH (8300)
>>>> I (221) re: Initialising RE Tools (8800)
>>>> I (226) netmanager: Initialising NETMANAGER (8999)
>>>> I (233) v-track: Registering Vehicle: TRACK (9000)
>>>> I (237) v-teslaroadster: Registering Vehicle: Tesla Roadster (9000)
>>>> I (243) v-teslamodels: Registering Vehicle: Tesla Model S (9000)
>>>> I (250) v-obdii: Registering Vehicle: OBDII (9000)
>>>> I (256) v-none: Registering Vehicle: NONE (9000)
>>>> I (261) v-demo: Registering Vehicle: DEMO (9000)
>>>> I (266) version: Initialising Versioning (9900)
>>>> I (275) cpu_start: Starting scheduler on PRO CPU.
>>>> I (0) cpu_start: Starting scheduler on APP CPU.
>>>> I (321) ovms_main: Executing on CPU core 0
>>>> I (321) ovms_main: Mounting CONFIG...
>>>> W (371) webserver: UpdateGlobalAuthFile: no password set => no auth for
>>>> web console
>>>> I (381) ovms_main: Registering default configs...
>>>> I (381) ovms_main: Starting HOUSEKEEPING...
>>>> I (381) housekeeping: Initialising HOUSEKEEPING Framework...
>>>> I (441) housekeeping: Executing on CPU core 1
>>>> I (441) housekeeping: reset_reason: cpu0=12, cpu1=12
>>>> I (441) housekeeping: Starting PERIPHERALS...
>>>> I (451) peripherals: Initialising OVMS Peripherals...
>>>> I (451) peripherals:   TCP/IP Adaptor
>>>> I (461) peripherals:   ESP32 system
>>>> I (461) peripherals:   SPI bus
>>>> I (461) peripherals:   MAX7317 I/O Expander
>>>> I (471) peripherals:   ESP32 CAN
>>>> I (471) peripherals:   ESP32 WIFI
>>>> I (481) peripherals:   ESP32 BLUETOOTH
>>>> I (481) peripherals:   ESP32 ADC
>>>> I (481) peripherals:   MCP2515 CAN 1/2
>>>> I (501) peripherals:   MCP2515 CAN 2/2
>>>> I (511) peripherals:   SD CARD
>>>> I (511) peripherals:   SIMCOM MODEM
>>>> I (511) gpio: GPIO[16]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup:
>>>> 1| Pulldown: 0| Intr:0
>>>> I (521) gpio: GPIO[17]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup:
>>>> 1| Pulldown: 0| Intr:0
>>>> I (531) uart: queue free spaces: 10
>>>> I (531) ext12v: Powering off external 12V devices
>>>> I (541) housekeeping: Auto init ext12v (free: 112712 bytes)
>>>> I (541) ext12v: Powering on external 12V devices
>>>> I (551) housekeeping: Auto init wifi (free: 112712 bytes)
>>>> I (551) wifi: wifi firmware version: ebd3e5d
>>>> I (561) wifi: config NVS flash: enabled
>>>> I (561) wifi: config nano formating: disabled
>>>> I (561) system_api: Base MAC address is not set, read default base MAC
>>>> address from BLK0 of EFUSE
>>>> I (571) system_api: Base MAC address is not set, read default base MAC
>>>> address from BLK0 of EFUSE
>>>> I (601) wifi: Init dynamic tx buffer num: 16
>>>> I (601) wifi: Init data frame dynamic rx buffer num: 16
>>>> I (601) wifi: Init management frame dynamic rx buffer num: 16
>>>> I (601) wifi: wifi driver task: 3ffe60f0, prio:23, stack:4096
>>>> I (611) wifi: Init static rx buffer num: 4
>>>> I (611) wifi: Init dynamic rx buffer num: 16
>>>> I (611) wifi: wifi power manager task: 0x3ffe8888 prio: 21 stack: 2560
>>>> I (1341) phy: phy_version: 383.0, 79a622c, Jan 30 2018, 15:38:06, 0, 0
>>>> I (1341) wifi: mode : sta (30:ae:a4:37:1b:64) + softAP (30:ae:a4:37:1b:65)
>>>> I (1351) housekeeping: Auto init modem (free: 90568 bytes)
>>>> I (1351) simcom: State: Enter PoweringOn state
>>>> I (1351) simcom: Power Cycle
>>>> I (1361) netmanager: WIFI access point is up
>>>> I (1361) webserver: Launching Web Server
>>>> I (1371) esp32wifi: AP started with SSID: ovms, MAC: 30:ae:a4:37:1b:65,
>>>> IP: 192.168.4.1
>>>> I (1371) telnet: Launching Telnet Server
>>>> I (1381) ssh: Launching SSH Server
>>>> I (1441) simcom: State: Enter PoweredOn state
>>>> I (2361) housekeeping: Auto init vehicle (free: 78820 bytes)
>>>> I (2361) v-teslaroadster: Tesla Roadster v1.x, v2.x and v3.0 vehicle module
>>>> I (2371) housekeeping: Auto init obd2ecu (free: 73088 bytes)
>>>> I (2571) housekeeping: Auto init server v2 (free: 64432 bytes)
>>>> I (2571) ovms-server-v2: OVMS Server V2 registered metric modifier is #1
>>>> I (2581) ovms-server-v2: Status: Starting
>>>> I (2581) ovms-server-v2: OVMS Server v2 running
>>>> I (2591) housekeeping: Auto init server v3 (free: 60120 bytes)
>>>> I (2591) housekeeping: Auto init done (free: 60120 bytes)
>>>> I (2601) housekeeping: Starting USB console...
>>>> I (2601) uart: queue free spaces: 30
>>>> I (2611) ovms-mdns: Starting MDNS
>>>>
>>>> Welcome to the Open Vehicle Monitoring System (OVMS) - Async Console
>>>> I (2651) version: Set version
>>>> I (4181) wifi: n:1 1, o:1 0, ap:1 1, sta:1 0, prof:1
>>>> I (4841) wifi: state: init -> auth (b0)
>>>> I (4841) wifi: state: auth -> auth (4a0)
>>>> OVMS>
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.teslaclub.hk
> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20180320/29039d00/attachment.htm>