April 2018 - OvmsDev - Open Vehicles

VFS / FAT deadlock
by Mark Webb-Johnson 08 Apr '18

08 Apr '18

That timing issue on headless boot-up, with 3.1 hardware, seems to be a deadlock within Espressif vfs/fat. I can repeat the issue by: Create a script /store/events/sd.mounted/logging: log file log file /sd/ovms.log log level verbose log level info ssh Unplug USB Power up module from 12V supply Any tasks accessing SD CARD will lock-up, sometime during the boot. Accessing other VFS areas seems ok. Can re-connect USB and come in with terminal emulator, ten seconds or more after boot, to examine the stack/etc. Here is the task status: OVMS# module tasks stack Number of Tasks = 16 Stack: Now Max Total Heap 32-bit SPIRAM 3FFAFB48 1 Blk esp_timer 388 548 4096 38080 644 21756 0x400dc0e3 0x4008ebed 3FFBDD5C 2 Blk eventTask 440 1208 4608 0 0 0 0x401b3814 0x4008ebed 0x401bcc06 3FFBFB58 3 Blk OVMS Events 1204 3812 6144 77388 0 6608 0x4008526a 0x4008ebed 0x4015e6b5 0x4008533c 0x401e2e53 0x40096a8c 0x4015eb7d 0x400e585a 0x400e6ece 0x400e4450 0x40085148 0x400eaec3 0x400eb768 0x400e9d1c 0x400ea6ec 0x400ea7f8 0x4008ebed 0x400ea808 3FFC4794 4 Blk OVMS CanRx 432 432 1024 0 0 0 0x400d39fd 0x4008ebed 3FFC884C 5 Blk ipc0 392 424 1024 7776 0 0 0x4008240b 0x4008ebed 0x400814fd 3FFC8E4C 6 Blk ipc1 392 504 1024 12 0 0 0x4008240b 0x4008ebed 0x40081590 3FFCAC74 9 Rdy IDLE 368 496 1024 0 0 0 0x4008fd60 3FFCB208 10 Rdy IDLE 356 484 1024 0 0 0 0x4008fd60 0x4008e0b7 0x4008e0b7 3FFCBF9C 11 Blk Tmr Svc 392 872 3072 0 0 168816 0x40090cdf 0x40090be4 3FFCF9F4 16 Blk tiT 492 700 3072 232 0 148 0x401fc049 0x4008ebed 0x40091ef6 0x401effe8 0x401f000c 0x401e7562 0x401e7562 3FFD5CA4 17 Blk OVMS SIMCOM 1584 2400 4096 284 0 0 0x4008526a 0x4008ebed 0x4015e6b5 0x4008533c 0x40000000 0x401e2e53 0x4015eb7d 0x400e585a 0x400e6ece 0x400e4450 0x40085148 0x40216acc 0x402177dc 0x401df626 0x401df4d5 0x401dfa38 0x401dfa8b 0x401e189e 0x401e2e69 0x4015eb7d 0x400e585a 0x400e6ece 0x400e4450 0x40085148 0x401379b5 0x400e58a1 0x40137a1b 0x40137a68 3FFD7C54 18 Blk wifi 432 1824 4096 240 0 1092 0x4008ab40 0x4008ebed 3FFDA3EC 19 Blk pmT 408 568 2560 0 0 0 0x401cb6c9 0x4008ebed 0x401cb751 3FFDAA04 20 Blk OVMS Vehicle 452 452 3072 0 0 0 0x4013ce45 0x4008ebed 0x4021f6e9 3FFDEC24 21 Rdy OVMS Console 760 3048 6144 220 15488 12000 0x400e6790 0x400f2a3b 0x400f6517 0x400f6585 0x400f2a72 0x400e9830 0x40197436 0x4021f01e 0x400e4508 0x400e986c 0x400e9a68 0x400e43ac 0x400e43bb 0x400f5280 3FFDF604 22 Blk mdns 412 412 4096 0 0 0 0x4020bae6 0x4008ebed The interesting tasks are OVMS Events: $ ~/esp/xtensa-esp32-elf/bin/xtensa-esp32-elf-addr2line -pfiaC -e build/ovms3.elf 0x4008526a 0x4008ebed 0x4015e6b5 0x4008533c 0x401e2e53 0x40096a8c 0x4015eb7d 0x400e585a 0x400e6ece 0x400e4450 0x40085148 0x400eaec3 0x400eb768 0x400e9d1c 0x400ea6ec 0x400ea7f8 0x4008ebed 0x400ea808 0x4008526a: lock_acquire_generic at /Users/mark/esp/esp-idf/components/newlib/./locks.c:157 0x4008ebed: xQueueGenericReceive at /Users/mark/esp/esp-idf/components/freertos/./queue.c:2037 0x4015e6b5: esp_vfs_write at /Users/mark/esp/esp-idf/components/vfs/./vfs.c:540 0x4008533c: _lock_acquire at /Users/mark/esp/esp-idf/components/newlib/./locks.c:165 0x401e2e53: vfs_fat_fsync at /Users/mark/esp/esp-idf/components/fatfs/src/vfs_fat.c:350 0x40096a8c: _fflush_r at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdio/../../../.././newlib/libc/stdio/fflush.c:282 0x4015eb7d: fsync at /Users/mark/esp/esp-idf/components/vfs/./vfs.c:552 (discriminator 3) 0x400e585a: OvmsCommandApp::LogBuffer(LogBuffers*, char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x400e6ece: OvmsCommandApp::Log(char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x400e4450: ConsoleAsync::ConsoleLogger(char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./console_async.cpp:118 0x40085148: esp_log_write at /Users/mark/esp/esp-idf/components/log/./log.c:215 0x400eaec3: Housekeeping::Metrics(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_housekeeping.cpp:221 0x400eb768: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data() const at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:135 (inlined by) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local() const at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:170 (inlined by) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:179 (inlined by) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:544 (inlined by) void std::_Mem_fn_base<void (Housekeeping::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), true>::operator()<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*, void>(Housekeeping*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) const at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:600 (inlined by) void std::_Bind<std::_Mem_fn<void (Housekeeping::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (Housekeeping*, std::_Placeholder<1>, std::_Placeholder<2>)>::__call<void, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&, 0u, 1u, 2u>(std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&>&&, std::_Index_tuple<0u, 1u, 2u>) at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:1074 (inlined by) void std::_Bind<std::_Mem_fn<void (Housekeeping::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (Housekeeping*, std::_Placeholder<1>, std::_Placeholder<2>)>::operator()<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*, void>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:1133 (inlined by) std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*), std::_Bind<std::_Mem_fn<void (Housekeeping::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)> (Housekeeping*, std::_Placeholder<1>, std::_Placeholder<2>)> >::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, void*&&) at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:1871 0x400e9d1c: std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*) const at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional:2271 0x400ea6ec: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data() const at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:135 (inlined by) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local() const at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:170 (inlined by) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:179 (inlined by) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/basic_string.h:544 (inlined by) OvmsEvents::HandleQueueSignalEvent(event_queue_t*) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_events.cpp:152 0x400ea7f8: OvmsEvents::EventTask() at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_events.cpp:126 0x4008ebed: xQueueGenericReceive at /Users/mark/esp/esp-idf/components/freertos/./queue.c:2037 0x400ea808: std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::list<EventCallbackEntry*, std::allocator<EventCallbackEntry*> >*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::list<EventCallbackEntry*, std::allocator<EventCallbackEntry*> >*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::list<EventCallbackEntry*, std::allocator<EventCallbackEntry*> >*> > >::_M_get_insert_unique_pos(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at /Users/mark/esp/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_tree.h:1800 And OVMS SIMCOM: $ ~/esp/xtensa-esp32-elf/bin/xtensa-esp32-elf-addr2line -pfiaC -e build/ovms3.elf 0x4008526a 0x4008ebed 0x4015e6b5 0x4008533c 0x40000000 0x401e2e53 0x4015eb7d 0x400e585a 0x400e6ece 0x400e4450 0x40085148 0x40216acc 0x402177dc 0x401df626 0x401df4d5 0x401dfa38 0x401dfa8b 0x401e189e 0x401e2e69 0x4015eb7d 0x400e585a 0x400e6ece 0x400e4450 0x40085148 0x401379b5 0x400e58a1 0x40137a1b 0x40137a68 0x4008526a: lock_acquire_generic at /Users/mark/esp/esp-idf/components/newlib/./locks.c:157 0x4008ebed: xQueueGenericReceive at /Users/mark/esp/esp-idf/components/freertos/./queue.c:2037 0x4015e6b5: esp_vfs_write at /Users/mark/esp/esp-idf/components/vfs/./vfs.c:540 0x4008533c: _lock_acquire at /Users/mark/esp/esp-idf/components/newlib/./locks.c:165 0x40000000: ?? ??:0 0x401e2e53: vfs_fat_fsync at /Users/mark/esp/esp-idf/components/fatfs/src/vfs_fat.c:350 0x4015eb7d: fsync at /Users/mark/esp/esp-idf/components/vfs/./vfs.c:552 (discriminator 3) 0x400e585a: OvmsCommandApp::LogBuffer(LogBuffers*, char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x400e6ece: OvmsCommandApp::Log(char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x400e4450: ConsoleAsync::ConsoleLogger(char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./console_async.cpp:118 0x40085148: esp_log_write at /Users/mark/esp/esp-idf/components/log/./log.c:215 0x40216acc: sdmmc_write_sectors_dma at /Users/mark/esp/esp-idf/components/sdmmc/./sdmmc_cmd.c:361 (discriminator 4) 0x402177dc: sdmmc_write_sectors at /Users/mark/esp/esp-idf/components/sdmmc/./sdmmc_cmd.c:750 0x401df626: ff_sdmmc_write at /Users/mark/esp/esp-idf/components/fatfs/src/diskio_sdmmc.c:51 0x401df4d5: ff_disk_write at /Users/mark/esp/esp-idf/components/fatfs/src/diskio.c:74 0x401dfa38: sync_window at /Users/mark/esp/esp-idf/components/fatfs/src/ff.c:4730 0x401dfa8b: move_window at /Users/mark/esp/esp-idf/components/fatfs/src/ff.c:4730 0x401e189e: f_sync at /Users/mark/esp/esp-idf/components/fatfs/src/ff.c:4730 0x401e2e69: vfs_fat_fsync at /Users/mark/esp/esp-idf/components/fatfs/src/vfs_fat.c:351 0x4015eb7d: fsync at /Users/mark/esp/esp-idf/components/vfs/./vfs.c:552 (discriminator 3) 0x400e585a: OvmsCommandApp::LogBuffer(LogBuffers*, char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x400e6ece: OvmsCommandApp::Log(char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x400e4450: ConsoleAsync::ConsoleLogger(char const*, __va_list_tag) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./console_async.cpp:118 0x40085148: esp_log_write at /Users/mark/esp/esp-idf/components/log/./log.c:215 0x401379b5: simcom::Task() at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/simcom/src/simcom.cpp:88 0x400e58a1: OvmsCommandApp::HexDump(char const*, char const*, char const*, unsigned int, unsigned int) at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/./ovms_command.cpp:94 0x40137a1b: simcom::Task() at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/simcom/src/simcom.cpp:105 0x40137a68: simcom::State1Ticker1() at /Users/mark/Documents/ovms/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/simcom/src/simcom.cpp:523 The other tasks all look ok. From what I can see, the task producing the log entry is dong a fsync, which presumably locks fat then locks vfs. Then another task tries to do a log as well, and gets locked up. If I comment-out the fsync in OvmsCommandApp::LogBuffer of ovms_command.cpp (line 614), it seems to avoid the problem. I’ve partially worked around the problem by introducing a mutex lock in ovms_command.cpp, when writing to the log file. That avoids the issue in ESP IDF. I’ve also raised it to Espressif. * b38b0ef (HEAD -> master, origin/master, origin/HEAD) Mutex protect fsync function call, on logging, to workaround ESP IDF bug https://github.com/espressif/esp-idf/issues/1837 It is an ugly solution, and doesn’t seem 100% effective. I’m really not sure what is going on. Any ideas? Regards, Mark.

1 0

Getting Started with OVMS v3.1
by Mark Webb-Johnson 08 Apr '18

08 Apr '18

FYI: Email sent to all purchasers of OVMS v3.1 modules Firstly, thank you for your support of the Open Vehicles project. As the first production batch of OVMS v3.1 kits has been shipped, you should be receiving yours any day now (if not already), we thought it useful to give you a few pointers for getting started with OVMS v3.1. 1] Unboxing There is a short unboxing video on youtube. It gives an overview of what is in the kit, and what you can do to get started: https://www.youtube.com/watch?v=HpyI1q4Y89I <https://www.youtube.com/watch?v=HpyI1q4Y89I> 2] User Guide Work on the user guide has begun, but is just starting and the document is expected to evolve rapidly over the coming weeks and months. You can always find the latest version here: https://docs.google.com/document/d/16JrXR7rybp-18DrEoeh1rg6GqQT_jVBvhaXh2oE… <https://docs.google.com/document/d/16JrXR7rybp-18DrEoeh1rg6GqQT_jVBvhaXh2oE…> You can comment directly on that document, and suggest revisions. If you would like to volunteer to help extend it, that would be most appreciated - please let us know and we’ll open up more access to it for you. 3] Hologram To simplify getting started, a Hologram SIM has been included with the modem in every OVMS v3.1 kit. Hologram have also kindly provided a coupon code “OVMS” that can be used to get US$5 credit; this can be used for both existing as well as new Hologram customers. You can simply enter that coupon code at checkout (when purchasing data credits, etc). When activating your Hologram SIM, you’ll need to enter the ICCID written on the SIM itself. You can also get that electronically (without having to open up the package) from the web or terminal shells with the following command: OVMS# metric list m.net.mdm.iccid 4] Firmware Status The firmware is continually evolving on a daily basis, and the development team regularly releases updates for over-the-air download and install. In these early days of production v3.1, it is important that you update your firmware on a regular basis. Please make sure you are running the latest firmware before reporting an issue. The simplest way to update the module firmware is to connect it as a client to your wifi network (or phone hotspot), and then use the web interface Config / Firmware / Flash-from-Web; that will show you the version you are running, as well as the latest version available for update. One click on 'Flash now' and the update will be downloaded and installed. Alternatively, you can download the firmware update to a file named ovms3.bin in the root directory of a FAT formatted SD CARD and insert the card into the module's slot - the new firmware will be updated automatically and the file renamed ovms3.done to signify completion. 5] Support Please remember that this first production batch is intended for experienced OVMS v2 users and v3 developers, and note: Firmware is not complete, and should be considered beta. Functionality is approximately equivalent to basic OVMS v2 firmware, but some advanced features (such as ACC, SMS control, etc) are not available yet. Firmware is far in advance of v2 in other aspects (such as built in web server, metrics, console, tcp/ip support, diagnostics, geofencing, scripting, etc). Firmware can be updated either Over The Air (OTA), by Micro SD card, or over USB interface. Vehicle support currently includes: Kia Soul EV, Nissan Leaf, Renault Twizy, and Tesla Roadster. If you do experience a problem, or need assistance, please start by talking to other OVMS v3.1 users/developers in your area. If you still can't solve the issue, then eMail support(a)openvehicles.com <mailto:support@openvehicles.com>. Software issues can be reported directly to the project GitHub issue tracker here: https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues> Thank you for your support of the project. Open Vehicles

1 0

Problem switching to Range mode charging (TR)
by Greg D. 07 Apr '18

07 Apr '18

Hi folks, To the Roadster owners among us... I just tried switching my 2.0 Roadster from Standard to Range mode charging, and there was an unusual "pop", similar to the car or OpenEVSE contactors engaging, but louder, followed by all the fans going off and a "Power Train Service Required" error on the display. Tapping away the message, the car started charging in Range mode as normal... Odd, and rather unnerving, given that I'm heading out on a trip shortly. I'll pull the logs when I return. This was my first experience using the OVMS system to control charging. I don't want to try it again. Has anyone else tried doing this? Greg

2 8

OTA status check timeout / SSL problem?
by Michael Balzer 07 Apr '18

07 Apr '18

Mark, the server check for an OTA update now fails every time, times out after 10 seconds. I think that's because the new server currently does a redirect from http to https also on the api.openvehicles.com host. Not sure why the module doesn't fail directly on that, maybe it tries to validate the certificate which also does not match. As the openvehicles server has frequent connectivity issues from here I've added a "nocheck" option to the ota status command and use that for the standard web status page. The OTA page still checks for the update. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

3 7

LCD Car Mount Display
by Mark Webb-Johnson 07 Apr '18

07 Apr '18

Having completed the v3.x hardware, and being a complete masochist for this, I’m now working with China to see what is possible as a display (for OVMS and other projects). The idea is to have a small enclosure that includes: A 3.5” backlit LCD display. Four touch buttons (as touch-screen too hard to use in a moving vehicle) - Up, Down, Enter, Back/Cancel. A WROVER32 (probably 4MB flash to keep costs down, but perhaps 16MB is possible). This has 4MB SPIRAM. MicroSD or SD card slot (full speed 4 wire). USB for firmware flashing and console access. Bluetooth Wifi Connection for: Power (12V regulated) Data (async bus) CAN (one single CAN bus) Expansion via internal GPIO headers OVMS firmware This could be a display for OVMS, or in general be a very flexible form of in-car or general purpose hacker display. The entire project would be geared towards getting the cost down. I’m trying to get an idea of what we can get this done for. What do people think? Is this any use? Regards, Mark. 1] Easily obtainable enclosure The idea being to use this along with a standard car mount (perhaps magnetic - there are lots) 2] Other examples, for comparison

4 13

LWIP lock-up
by Mark Webb-Johnson 06 Apr '18

06 Apr '18

My car managed to repeat the issue with the task lockup I have seen occasionally. The symptom is that network connectivity fails and the async console becomes unresponsive when you issue a command to bring up/down wifi, simcom, etc. The new housekeeping 10 minute ticker worked well for this, and stopped in the middle of the night: … I (134401020) housekeeping: Local time: Sat Mar 24 20:15:45 2018 I (135001020) housekeeping: Local time: Sat Mar 24 20:25:45 2018 I (135601020) housekeeping: Local time: Sat Mar 24 20:35:45 2018 I (136201020) housekeeping: Local time: Sat Mar 24 20:45:45 2018 Looking at the logs after Sat Mar 24 20:45:45, I see: D (136543060) simcom: rx line ch=4 len=19 : +PPPD: DISCONNECTED I (136543070) simcom: PPP Connection disconnected I (136544020) simcom: Lost network connection (+PPP disconnect in NetMode) I (136544020) simcom: State: Enter NetLoss state ... I (136544030) gsm-ppp: Shutting down (hard)... I (136544040) events: Signal(system.modem.down) I (136544040) netmanager: Interface priority is st1 (x.y.z.212/255.255.248.0 gateway x.y.z.64) I (136544050) events: Signal(network.modem.down) I (136544050) netmanager: MODEM down (with WIFI client up): staying with WIFI client priority I (136544060) events: Signal(network.interface.change) OK, so ppp went down (lost network connectivity), and WIFI client was up so we kept with that. The GSM signal is still up (CREG), so we try to reconnect PPP: I (136558160) simcom: PPP Connection is ready to start I (136559020) simcom: State: Enter NetMode state I (136559020) gsm-ppp: Initialising.. ... D (136589180) simcom: rx line ch=4 len=8 : +CREG: 2 I (136589190) simcom: CREG Network Registration: Searching ... D (136589250) simcom: rx line ch=4 len=8 : +CREG: 5^[[0m ^[[0;32mI (136589250) simcom: CREG Network Registration: RegisteredRoaming ... D (136589320) simcom: rx line ch=4 len=8 : +CREG: 2^[[0m ^[[0;32mI (136589320) simcom: CREG Network Registration: Searching ... D (136589380) simcom: rx line ch=4 len=8 : +CREG: 5^[[0m ^[[0;32mI (136589390) simcom: CREG Network Registration: RegisteredRoaming ... D (136601690) simcom: rx line ch=3 len=8 : +CREG: 2^[[0m ^[[0;32mI (136601690) simcom: CREG Network Registration: Searching ... D (136601780) simcom: rx line ch=3 len=8 : +CREG: 5^[[0m ^[[0;32mI (136601780) simcom: CREG Network Registration: RegisteredRoaming ... V (136617440) gsm-ppp: tx: 7e ff 7d 23 c0 21 7d 25 7d 2c 7d 20 20 4e 6f 20 | ~.}#.!}%},} No V (136617440) gsm-ppp: tx: 6e 65 74 77 6f 72 6b 20 70 72 6f 74 6f 63 6f 6c | network protocol V (136617450) gsm-ppp: tx: 73 20 72 75 6e 6e 69 6e 67 4b 8f 7e | s runningK.~ V (136617450) simcom: tx: f9 09 ff 59 7e ff 7d 23 c0 21 7d 25 7d 2c 7d 20 | ...Y~.}#.!}%},} V (136617450) simcom: tx: 20 4e 6f 20 6e 65 74 77 6f 72 6b 20 70 72 6f 74 | No network prot V (136617460) simcom: tx: 6f 63 6f 6c 73 20 72 75 6e 6e 69 6e 67 4b 8f 7e | ocols runningK.~ V (136617460) simcom: tx: a6 f9 | .. V (136617490) simcom: rx: f9 09 ff 25 7e ff 7d 23 c0 21 7d 26 7d 2c 7d 20 | ...%~.}#.!}&},} V (136617500) simcom: rx: 7d 24 8f 7d 3d 7e fb f9 | }$.}=~.. V (136617510) gsm-mux: ProcessFrame(CHAN=2, ADDR=09, CTRL=ff, FCS=fb, LEN=24) V (136617520) gsm-mux: ChanProcessFrame(CHAN=2, ADDR=09, CTRL=ff, LEN=21, IFP=3) V (136617520) gsm-ppp: rx: 7e ff 7d 23 c0 21 7d 26 7d 2c 7d 20 7d 24 8f 7d | ~.}#.!}&},} }$.} V (136617530) gsm-ppp: rx: 3d 7e | =~ I (136617540) gsm-ppp: StatusCallBack: Connection Lost E (136617540) gsm-ppp: status_cb: Connection lost I (136617550) gsm-ppp: Shutdown (via status callback) I (136617550) events: Signal(system.modem.down) I (136617560) netmanager: Interface priority is st1 (x.y.z.212/255.255.248.0 gateway z.y.z.64) I (136617560) gsm-ppp: Attempting PPP reconnecting in 30 seconds... So, while initialising PPP we’ve been told that the cellular network is down (CREG: 2). Then, it flaps up and down like a yoyo, until the PPP connect finally fails. At this point, things go screwy. The simcom tells us the PPP is down, and then we lose the ability to transmit to the modem (or at least log those transmits): V (136618650) simcom: rx: 50 50 50 44 3a 20 44 49 53 43 4f 4e 4e 45 43 54 | PPPD: DISCONNECT V (136618660) simcom: rx: 45 44 0d 0a d4 f9 f9 11 ff 2f 0d 0a 2b 50 50 50 | ED......./..+PPP V (136618660) gsm-mux: ProcessFrame(CHAN=3, ADDR=0d, CTRL=ff, FCS=d4, LEN=29) V (136618670) gsm-mux: ChanProcessFrame(CHAN=3, ADDR=0d, CTRL=ff, LEN=26, IFP=3) D (136618680) simcom: rx line ch=3 len=19 : +PPPD: DISCONNECTED I (136618690) simcom: PPP Connection disconnected V (136618690) simcom: rx: 44 3a 20 44 49 53 43 4f 4e 4e 45 43 54 45 44 0d | D: DISCONNECTED. V (136618700) simcom: rx: 0a d9 f9 | ... V (136618710) gsm-mux: ProcessFrame(CHAN=4, ADDR=11, CTRL=ff, FCS=d9, LEN=29) V (136618710) gsm-mux: ChanProcessFrame(CHAN=4, ADDR=11, CTRL=ff, LEN=26, IFP=3) D (136618720) simcom: rx line ch=4 len=19 : +PPPD: DISCONNECTED I (136618730) simcom: PPP Connection disconnected I (136619020) simcom: Lost network connection (+PPP disconnect in NetMode) I (136619020) simcom: State: Enter NetLoss state V (136619030) simcom: tx: f9 0d ff 19 41 54 2b 43 47 41 54 54 3d 30 0d 0a | ....AT+CGATT=0.. V (136619030) simcom: tx: 14 f9 | .. I (136619030) gsm-ppp: Shutting down (hard)...^[[0m I (136619040) events: Signal(system.modem.down)^[[0m I (136619040) netmanager: Interface priority is st1 (x.y.z.212/255.255.248.0 gateway x.y.z.64) V (136632920) simcom: rx: f9 0d ff 0d 0d 0a 4f 4b 0d 0a 0f f9 | ......OK.... V (153076210) simcom: rx: f9 09 ff 1d 0d 0a 2b 43 53 51 3a 20 31 2c 39 39 | ......+CSQ: 1,99 V (153076220) simcom: rx: 0d 0a d1 f9 f9 0d ff 1d 0d 0a 2b 43 53 51 3a 20 | ..........+CSQ: V (153076220) simcom: rx: 31 2c 39 39 0d 0a 13 f9 f9 11 ff 1d 0d 0a 2b 43 | 1,99..........+C V (153076230) simcom: rx: 53 51 3a 20 31 2c 39 39 0d 0a 1e f9 | SQ: 1,99.... V (153202470) simcom: rx: f9 09 ff 1d 0d 0a 2b 43 53 51 3a 20 34 2c 39 39 | ......+CSQ: 4,99 V (153202480) simcom: rx: 0d 0a d1 f9 f9 0d ff 1d 0d 0a 2b 43 53 51 3a 20 | ..........+CSQ: V (153202490) simcom: rx: 34 2c 39 39 0d 0a 13 f9 f9 11 ff 1d 0d 0a 2b 43 | 4,99..........+C V (153202500) simcom: rx: 53 51 3a 20 34 2c 39 39 0d 0a 1e f9 | SQ: 4,99.... V (179212720) simcom: rx: f9 09 ff 1d 0d 0a 2b 43 53 51 3a 20 30 2c 39 39 | ......+CSQ: 0,99 V (179212730) simcom: rx: 0d 0a d1 f9 f9 0d ff 1d 0d 0a 2b 43 53 51 3a 20 | ..........+CSQ: V (179212730) simcom: rx: 30 2c 39 39 0d 0a 13 f9 f9 11 ff 1d 0d 0a 2b 43 | 0,99..........+C V (179212740) simcom: rx: 53 51 3a 20 30 2c 39 39 0d 0a 1e f9 | SQ: 0,99.... V (179242750) simcom: rx: f9 09 ff 1d 0d 0a 2b 43 53 51 3a 20 30 2c 39 39 | ......+CSQ: 0,99 V (179242760) simcom: rx: 0d 0a d1 f9 f9 0d ff 1d 0d 0a 2b 43 53 51 3a 20 | ..........+CSQ: V (179242770) simcom: rx: 30 2c 39 39 0d 0a 13 f9 f9 11 ff 1d 0d 0a 2b 43 | 0,99..........+C V (179242770) simcom: rx: 53 51 3a 20 30 2c 39 39 0d 0a 1e f9 | SQ: 0,99.... V (179246100) simcom: rx: f9 09 ff 19 0d 0a 2b 43 52 45 47 3a 20 32 0d 0a | ......+CREG: 2.. V (179246110) simcom: rx: d6 f9 f9 0d ff 19 0d 0a 2b 43 52 45 47 3a 20 32 | ........+CREG: 2 V (179246110) simcom: rx: 0d 0a 14 f9 f9 11 ff 19 0d 0a 2b 43 52 45 47 3a | ..........+CREG: V (179246120) simcom: rx: 20 32 0d 0a 19 f9 | 2.... V (179251580) simcom: rx: f9 09 ff 1d 0d 0a 2b 43 53 51 3a 20 34 2c 39 39 | ......+CSQ: 4,99 V (179251590) simcom: rx: 0d 0a d1 f9 f9 0d ff 1d 0d 0a 2b 43 53 51 3a 20 | ..........+CSQ: V (179251600) simcom: rx: 34 2c 39 39 0d 0a 13 f9 f9 11 ff 1d 0d 0a 2b 43 | 4,99..........+C V (179251600) simcom: rx: 53 51 3a 20 34 2c 39 39 0d 0a 1e f9 | SQ: 4,99.... I (179290100) wifi: bcn_timout,ap_probe_send_start I (179292610) wifi: ap_probe_send over, resett wifi status to disassoc I (179292610) wifi: state: run -> init (1) I (179292620) wifi: pm stop, total sleep time: 0/138543477 I (179292620) wifi: n:13 0, o:13 0, ap:255 255, sta:13 0, prof:1 The simcom itself seems to be up, and is giving us unsolicited notifications. Finally, the wifi stack falls over (bcn_timeout.ap_probe_sent). This is very nasty, and I suspect will take some time to resolve. We can probably make it more stable by introducing a check for network loss, and requiring the network to be up and stable for N seconds before even attempting a reconnection; but I don’t think that addresses the core issue (which is most likely in the lwip ppp stack). Code-wise, the simcom driver entering ‘NetLoss’ state calls m_ppp.Shutdown(true) (which issues the shutting down (hard) message). Our pppos driver issues the system.modem.down event (which causes netmanager to do it’s thing), and then calls ppapi_close(m_ppp, nocarrier). I suspect that never returns (but don’t know for sure). The other workaround is to enable watchdog timers. I’m 99% certain this sort of lockup would mess up the idle tasks and those would pickup and fire a watchdog timer. So, I’ve gone ahead and committed code to enable the watchdog timers (2 minutes), and make that available as an OTA update: OVMS# ota status Firmware: 3.1.001/ota_0/main (build idf v3.1-dev-453-g0f978bcb Mar 25 2018 14:45:59) Server Available: 3.1.001 The module in my car is now running OTA, so I can verify the OTA update process is smooth (it seems so). Regards, Mark.

3 9

OVMS v3.1 stock
by Mark Webb-Johnson 05 Apr '18

05 Apr '18

I now have stock of OVMS v3.1. I need some for HK installs, but there are some spare (both US and EU kits, as well as bare OVMS modules without modems). You guys have got priority, but I don’t want to hold on to these too long. I would rather get them out into cars. If you need one, let me know ASAP. Email mark(a)webb-johnson.net with what you need. Price is same as Fasttech - US$130 (including $20 developer discount) plus US$12 postage (EU or US). Regards, Mark

3 3

OVMS v3 Unboxing
by Mark Webb-Johnson 05 Apr '18

05 Apr '18

Enjoy. Please share among your communities. Regards, Mark https://youtu.be/HpyI1q4Y89I

2 1

Task stack optimization
by Michael Balzer 05 Apr '18

05 Apr '18

I've done some stack size watching and pushed some optimizations: - CanRxTask: 4K → 1K - COrx Task: 4K → 3K [+config] - COwrk Task: 4K → 1.5K [+config] - Vrx Task: 4K → 3K [+config] - Tmr Svc: 6K → 3K Much better now, nearly no crashes during normal operation, so most crashes now have been memory related (I had normally 2-3 crashes during a normal drive). A single crash I had when getting back home may have been due to my PC reconnecting to the webserver in parallel to my phone. There are more stacks we possibly can reduce: - SIMCOMTask normally maxes out at 2.3K (of 4) - wifi ditto - mdns normally uses 1.7K of 4 (no config for that one, defined in esp mdns_private.h) These should normally have no larger stack requirements as their operation is pretty much fixed, so that would be another 4.5 - 5K to get. As stacks need to be pinned to the internal RAM, hw3.1 also should benefit from these optimizations. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

1 1

OBDII ECU Documentation, and a request for help
by Greg D. 05 Apr '18

05 Apr '18

Hi folks In a fit of prose, if not Shakespeare, I took a pass at creating the documentation chapter for the OBD2ECU task. Comments, corrections, and general feedback is requested. Is there anything significant missing? Mark, if this looks acceptable, can it be posted to the Google Doc, or send me an invite such that I can do so? Note that I don't have any reference material on the OBDII-to-DB26 cable, other than the wiring. If we eventually have a purchase source, that should get referenced, or lacking that, perhaps pointers to where the connectors can be purchased? Placeholders are noted, but only that. For the EEs in the group, a question: I have both signal and chassis grounds connected together on the OBDII connector; should they be, or just signal ground? I think it would also be helpful to include a table of what metrics are available (supported) on what vehicles, as an aid to those who want to remap things for monitoring on an OBDII HUD or Dongle. A glance at the available documentation didn't turn up anything, so I took a first pass at the table. See the attached, which includes what I know of the Roadster's implementation (since I have one and have studied the code a bit). I would rather not try to guess at the other cars. Could the various vehicle authors contribute their columns to the table? Send me an updated spreadsheet, and I will do the merge. Thanks! Greg

5 14