March 2021 - OvmsDev - Open Vehicles

Netmanager priority issue solved
by Michael Balzer 05 Jul '23

05 Jul '23

TL;DR: you need to pull my latest esp-idf changes. I've finally found & solved the strange priority changes for our netmanager task (i.e. being raised suddenly from origin 5 to 18/22): the bug was in the esp-idf posix threads mutex implementation. I had suspected the mutex priority inheritance for a while, so added a way to retrieve the internal mutex hold count for our task list. Using this I noticed the hold count would always & only raise by 2 whenever any kind of mongoose connection was closed. That lead me to checking the thread concurrency protection for mongoose mbufs, because every mongoose connection has two mbufs associated (rx & tx), each having a posix mutex. The bug was: posix mutexes were deleted after locking (taking) them. FreeRTOS mutexes must not be deleted while being taken, that breaks the priority inheritance (more precisely the disinheritance), with the visible effect being the mutex hold count not returning to zero. As a side effect, this may also solve the strange event task starvations (hope so…). I was investigating this as I suspected some busy loop in the netmanager context. With the netman running at prio 22, that would effectively block almost all other processing including the timer service. I've found & fixed one potential busy loop trigger in the netman that would have been caused by the netman task still running while all interfaces had been lost -- not sure if that could happen, but it would explain the effects. So please watch your crash debug info & report if the issue still turns up. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

5 19

IOS app reported GSM signal strength
by Craig Leres 16 Oct '22

16 Oct '22

I've noticed for awhile but never asked the question; why does the IOS app report wifi signal strength when I'm connected to wifi as gsm signal strength? I would expect gsm to always be gsm and maybe wifi would also be shown (and identified as such) when present. Note that I run my modules in wifi client-only mode. Craig

2 2

TLS CA question
by Stephen Casner 29 Jan '22

29 Jan '22

I find that I need to enable the following option in my testing of the possible replacement of MEDTLS with WolfSSL, otherwise I get an "ASN no signer to confirm" error: WOLFSSL_ALT_CERT_CHAIN allows CA's to be presented by peer, but not part of a valid chain. Default wolfSSL behavior is to require validation of all presented peer certificates. This also allows loading intermediate CA's as trusted and ignoring no signer failures for CA's up the chain to root. The alternate certificate chain mode only requires that the peer certificate validate to a trusted CA. Is that expected for the trust arrangements we are using? A possibly related question: do we expect the server to validate clients, or only the clients to validate the server? -- Steve

4 60

Open issues blocking release 3.2.016
by Michael Balzer 21 Jun '21

21 Jun '21

As the other thread got off-topic: I'd still like to get forward regarding release 3.2.016. We identified three open issues: 1. VW e-Up sometimes stays awake for 4-5 hours. 2. Wifi AP connection is unstable with some devices. 3. More frequent location flatbed theft false alerts. My current status on these is: 1. …is probably not a bug on the OVMS but rather a real behaviour of the e-Up performing an extended 12V trickle charge. 2. …has only been reported by Steve Davies and seems (?) to have been solved by the default scan timings – Steve? 3. …seems to be a Heisenbug, i.e. hasn't had any reports since we added extended logging. Is my status correct? If so, I think these should no longer stop us from releasing 3.2.016. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

6 12

OVMS going comatose
by Stephen Casner 24 Apr '21

24 Apr '21

Since updating my in-car OVMS with a build from the master branch after my WolfSSL update I've experienced a couple of instances where connectivity from app was lost. The first time I needed a quick resolution so I just reset OVMS. It happened again during the night while charging, and this time I connected to the console to check conditions before resetting. What I found was that the async console was not responding at all. Since I have OVMS configured to drop into gdb on a crash, I should have seen something. I've turned on logging to file to see if that sheds any light. -- Steve

3 16

Coding help
by Shane ＠ Kilve Engineering 05 Apr '21

05 Apr '21

can any one help me with a bit of code to retrieve ve some data from ecu can1/6f2 00 4c 00 00 01 97 bc 00 as i cant poll this ecu it just broadcasts when its active, ive tried polling it but does not respond, i need to choose one byte and allocate it to a PID.

3 22

ESP32 CAN controller hardware bug / interrupt issue
by Michael Balzer 02 Apr '21

02 Apr '21

Second finding & commit: TL;DR: * ESP32 CAN controller is broken, delivers trashed & duplicated RX frames * we need to optimize interrupt latency Users have been reporting occasional false BMS readings on the Twizy for more than a year. The error always trashed the reading of voltage #10 and/or temperature #8, so I initially thought that must be a variant of the SPIRAM/cache bug. But the SPIRAM fix didn't help, and the bug still occured with all BMS data in internal RAM. The issue correlated with high load situations and the SIMCOM UART issue. On testing the latest changes by Marko, my car module began showing that issue regularly, so I could take a closer look. It turned out the wrong data came from the CAN bus. Checking with a CAN logger didn't show any wrong data on the bus. So I scanned our CAN driver for potential issues, found & solved some (even a serious race condition) but that didn't fix it. I then added debug dumps of all frames read, dug into the SJA1000 spec sheet and found this: The SJA1000 RX FIFO has a capacity of 64 bytes. It's supposed to work as a ring buffer, with frames wrapping around as necessary. If no space is left in the FIFO for a new frame coming in, that frame should be discarded completely, not counted, just the overflow indicator should be set & the according interrupt be generated. Reality differs. The frame causing the overflow is added to the FIFO partially. It's also counted both in the RMC register and indicated by RBS as a valid frame when retrieving the FIFO contents. The read buffer delivers the partial frame + some trashed bytes up to the nominal frame length. In the BMS frame sequence it would normally trash bytes 7 & 8 on the sixth frame of length 8 in a sequence. A standard frame needs a 3 byte header + the data bytes in the FIFO, so 6 frames would add up to 66 bytes => 64 + 2 trash. The first trash byte normally is "08", the second "84" or "2a" or sometimes "ab", possibly some internal SJA1000 data. Better yet, if you rely on the indicators of the SJA1000 when reading the FIFO, the SJA1000 will repeat the first frame within the FIFO after the invalid frame. Mostly once or twice, and sometimes 7 times or more: inv_msg: framecnt=13, invindex=6 inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 24 04 00 00 11 40 10 22 37 55 00 37 | 4..?........$....@."7U.7 inv_msg: 25 04 00 00 0a 1b 44 ff fe 4e 01 26 | 4..?........%.....D..N.& inv_msg: 54 05 00 00 37 37 37 37 37 37 37 00 | 4..?........T...7777777. inv_msg: 56 05 00 00 31 63 14 31 53 14 31 4a | 4..?........V...1c.1S.1J inv_msg: 57 05 00 00 31 43 14 31 53 15 *08 2a* | 4..?........W...1C.1S..* ^^^^^ trashed bytes … followed by 7 repetitions of the first frame: inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n If you had occasional strange false readings leading to strange effects, this may be the explanation. Note: the SJA1000 FIFO bug may need specific circumstance to occur in addition to the overflow, maybe the overflow happening on a specific byte position in the FIFO -- I haven't tried to determine that. I've tried clearing the overflow ASAP, looping on the IR indicator and looping on the RMC, the fault is always the same. The workaround I've now done is adding up the message lengths read during a FIFO run and discard all frames exceeding the 64 byte border. Not pretty, but functional. That keeps the trashed frames from being forwarded to the OVMS application level. As with the SIMCOM UART, these overflows wouldn't happen if the interrupt handler would react fast enough. It seems we've got a general interrupt latency issue. And just as with the SIMCOM I saw a reduction of overflows when allocating the CAN IRQ on core #0, another one with raising the IRQ level to 3, so that's also part of my commit. The esp-idf CAN driver apparently has the same issue even in the latest release, I'll contact the author about this or post another issue to Espressif. So our interrupt allocation is now: Task Run# Lvl ISR# Usage Source ---------------------------------------------------------------------------------------- - 0 1 0 RTC Core ETS_RTC_CORE_INTR_SOURCE esp_timer 0 1 0 RTOS esp_timer ETS_TIMER2_INTR_SOURCE ipc0 0 1 0 RTOS Scheduler Core0 ETS_FROM_CPU_INTR0_SOURCE ipc1 1 1 1 RTOS Scheduler Core1 ETS_FROM_CPU_INTR1_SOURCE main 0 1 0 RTOS Watchdog ETS_TG0_WDT_LEVEL_INTR_SOURCE OVMS Events 1 1 1 Peripherals (GPIO ISR) ETS_GPIO_INTR_SOURCE OVMS Events 1 1 1 MAX7317 (spi_nodma) ETS_SPI3_INTR_SOURCE *esp32can init 0 3 0 ESP32 CAN ETS_CAN_INTR_SOURCE** **OVMS SIMCOM 0 2 0 SIMCOM UART ETS_UART1_INTR_SOURCE* OVMS Events 1 1 1 USB console (UART) ETS_UART0_INTR_SOURCE OVMS Events 1 1 1 SD card ETS_SDIO_HOST_INTR_SOURCE ---------------------------------------------------------------------------------------- I think we need to find the source of the interrupt latency and/or find some better scheme to distribute the tasks & interrupts over the cores. I still don't know why core #1 already got an issue with just our 6 interrupts. This may be related to the SPIRAM fix adding overhead, but I've seen the FIFO overflows before applying the fix, and the CAN issue also has been present for over a year. My working hypothesis is some task/driver disabling interrupts for too often / too long, or non-IRAM ISRs getting delayed for too often / too long. And it seems this is an issue especially on core #1. Please verify & comment. Regards, Michael PS: a little side mystery: the netmanager task starts with priority 5, then gets raised to 22 (probably a mutex priority inheritance), but never gets lowered back to 5 afterwards. A FreeRTOS bug? -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

2 11

Multi frame query
by didier＠ernotte.com 02 Apr '21

02 Apr '21

Hi, I am investigating some PID with very long data. I am using the PID scanner for this.When I turn on the log level debug , I can see this as the first repy D (1456677) obd2ecu: Rcv 7ec: 8 (15 df 62 48 83 0 0 0) >From the Canbus doc, I understand that the first "1x (15)" tells me that this is the first frame of a multi frame, and the length is "5df" . The Pid scanner seems to return only "df", not "5df" data, and I can see in the log that I have around 1500 byte returned, not 220. Any explanation ? anyone see this problem as well ? The scan command is (I had to hack the code to allow timeout > 10sec) re obdii scan start 1 7e4 4883 4883 -r500-7ff -t22 -x30 Didier

5 16

Need some git school
by Stephen Casner 29 Mar '21

29 Mar '21

I haven't gotten any traction on my request for people to test the updated mongoose-wolfssl branch, so I thought it might help to rebase to the current master. I did that without complaint in my local repository, but I can't push that back to github: auge92> git push To github.com:openvehicles/Open-Vehicle-Monitoring-System-3.git ! [rejected] mongoose-wolfssl -> mongoose-wolfssl (non-fast-forward) error: failed to push some refs to 'github.com:openvehicles/Open-Vehicle-Monitoring-System-3.git' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Integrate the remote changes (e.g. hint: 'git pull ...') before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. I have all the same changes on my local repo as on github, but I observe that all the commits I made since branching (beginning with 04fafb83a3cc5563a5feb70a4bb7922457d58893) have a different commit id than what is in my local mongoose-wolfssl repo for the same changes. This is what causes the complaint above. I must have done something wrong back at the beginning of working on this branch. Any idea what I might have done? Would this be an appropriate scenario to use "git push --force" to correct the problem? -- Steve

3 20

New vehicle: Hyundai Ioniq vFL
by Michael Balzer 25 Mar '21

25 Mar '21

Everyone, by request of Henri Bachmann on the german GoingElectric forum, I've added the Hyundai Ioniq vFL (28 kWh). The implementation is based on the reverse engineering data published by the EVNotify project: https://github.com/EVNotify/EVNotiPi/blob/master/car/ioniq_bev.py Module Documentation: https://docs.openvehicles.com/en/latest/components/vehicle_hyundai_ioniqvfl… Henri has tested the module, but more tests from other Ioniq drivers are welcome. One of the PIDs needs more data/logs: the "driveMotorSpeed" didn't match the actual speed on Henri's tests. That PID isn't used by the EVNotify system so may be wrong, or could be RPM. If you can help with logs on this, enable file logging and do a test drive with level verbose on module v-hyundaivfl. Try to maintain a fixed maximum speed for 20 seconds for reference. Send me the logs and the max speed. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

2 2