November 2020 - OvmsDev - Open Vehicles

Netmanager priority issue solved
by Michael Balzer 05 Jul '23

05 Jul '23

TL;DR: you need to pull my latest esp-idf changes. I've finally found & solved the strange priority changes for our netmanager task (i.e. being raised suddenly from origin 5 to 18/22): the bug was in the esp-idf posix threads mutex implementation. I had suspected the mutex priority inheritance for a while, so added a way to retrieve the internal mutex hold count for our task list. Using this I noticed the hold count would always & only raise by 2 whenever any kind of mongoose connection was closed. That lead me to checking the thread concurrency protection for mongoose mbufs, because every mongoose connection has two mbufs associated (rx & tx), each having a posix mutex. The bug was: posix mutexes were deleted after locking (taking) them. FreeRTOS mutexes must not be deleted while being taken, that breaks the priority inheritance (more precisely the disinheritance), with the visible effect being the mutex hold count not returning to zero. As a side effect, this may also solve the strange event task starvations (hope so…). I was investigating this as I suspected some busy loop in the netmanager context. With the netman running at prio 22, that would effectively block almost all other processing including the timer service. I've found & fixed one potential busy loop trigger in the netman that would have been caused by the netman task still running while all interfaces had been lost -- not sure if that could happen, but it would explain the effects. So please watch your crash debug info & report if the issue still turns up. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

5 19

ESP32 CAN controller hardware bug / interrupt issue
by Michael Balzer 03 Apr '21

03 Apr '21

Second finding & commit: TL;DR: * ESP32 CAN controller is broken, delivers trashed & duplicated RX frames * we need to optimize interrupt latency Users have been reporting occasional false BMS readings on the Twizy for more than a year. The error always trashed the reading of voltage #10 and/or temperature #8, so I initially thought that must be a variant of the SPIRAM/cache bug. But the SPIRAM fix didn't help, and the bug still occured with all BMS data in internal RAM. The issue correlated with high load situations and the SIMCOM UART issue. On testing the latest changes by Marko, my car module began showing that issue regularly, so I could take a closer look. It turned out the wrong data came from the CAN bus. Checking with a CAN logger didn't show any wrong data on the bus. So I scanned our CAN driver for potential issues, found & solved some (even a serious race condition) but that didn't fix it. I then added debug dumps of all frames read, dug into the SJA1000 spec sheet and found this: The SJA1000 RX FIFO has a capacity of 64 bytes. It's supposed to work as a ring buffer, with frames wrapping around as necessary. If no space is left in the FIFO for a new frame coming in, that frame should be discarded completely, not counted, just the overflow indicator should be set & the according interrupt be generated. Reality differs. The frame causing the overflow is added to the FIFO partially. It's also counted both in the RMC register and indicated by RBS as a valid frame when retrieving the FIFO contents. The read buffer delivers the partial frame + some trashed bytes up to the nominal frame length. In the BMS frame sequence it would normally trash bytes 7 & 8 on the sixth frame of length 8 in a sequence. A standard frame needs a 3 byte header + the data bytes in the FIFO, so 6 frames would add up to 66 bytes => 64 + 2 trash. The first trash byte normally is "08", the second "84" or "2a" or sometimes "ab", possibly some internal SJA1000 data. Better yet, if you rely on the indicators of the SJA1000 when reading the FIFO, the SJA1000 will repeat the first frame within the FIFO after the invalid frame. Mostly once or twice, and sometimes 7 times or more: inv_msg: framecnt=13, invindex=6 inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 24 04 00 00 11 40 10 22 37 55 00 37 | 4..?........$....@."7U.7 inv_msg: 25 04 00 00 0a 1b 44 ff fe 4e 01 26 | 4..?........%.....D..N.& inv_msg: 54 05 00 00 37 37 37 37 37 37 37 00 | 4..?........T...7777777. inv_msg: 56 05 00 00 31 63 14 31 53 14 31 4a | 4..?........V...1c.1S.1J inv_msg: 57 05 00 00 31 43 14 31 53 15 *08 2a* | 4..?........W...1C.1S..* ^^^^^ trashed bytes … followed by 7 repetitions of the first frame: inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n inv_msg: 55 01 00 00 07 98 50 54 54 20 00 6e | 4..?........U.....PTT .n If you had occasional strange false readings leading to strange effects, this may be the explanation. Note: the SJA1000 FIFO bug may need specific circumstance to occur in addition to the overflow, maybe the overflow happening on a specific byte position in the FIFO -- I haven't tried to determine that. I've tried clearing the overflow ASAP, looping on the IR indicator and looping on the RMC, the fault is always the same. The workaround I've now done is adding up the message lengths read during a FIFO run and discard all frames exceeding the 64 byte border. Not pretty, but functional. That keeps the trashed frames from being forwarded to the OVMS application level. As with the SIMCOM UART, these overflows wouldn't happen if the interrupt handler would react fast enough. It seems we've got a general interrupt latency issue. And just as with the SIMCOM I saw a reduction of overflows when allocating the CAN IRQ on core #0, another one with raising the IRQ level to 3, so that's also part of my commit. The esp-idf CAN driver apparently has the same issue even in the latest release, I'll contact the author about this or post another issue to Espressif. So our interrupt allocation is now: Task Run# Lvl ISR# Usage Source ---------------------------------------------------------------------------------------- - 0 1 0 RTC Core ETS_RTC_CORE_INTR_SOURCE esp_timer 0 1 0 RTOS esp_timer ETS_TIMER2_INTR_SOURCE ipc0 0 1 0 RTOS Scheduler Core0 ETS_FROM_CPU_INTR0_SOURCE ipc1 1 1 1 RTOS Scheduler Core1 ETS_FROM_CPU_INTR1_SOURCE main 0 1 0 RTOS Watchdog ETS_TG0_WDT_LEVEL_INTR_SOURCE OVMS Events 1 1 1 Peripherals (GPIO ISR) ETS_GPIO_INTR_SOURCE OVMS Events 1 1 1 MAX7317 (spi_nodma) ETS_SPI3_INTR_SOURCE *esp32can init 0 3 0 ESP32 CAN ETS_CAN_INTR_SOURCE** **OVMS SIMCOM 0 2 0 SIMCOM UART ETS_UART1_INTR_SOURCE* OVMS Events 1 1 1 USB console (UART) ETS_UART0_INTR_SOURCE OVMS Events 1 1 1 SD card ETS_SDIO_HOST_INTR_SOURCE ---------------------------------------------------------------------------------------- I think we need to find the source of the interrupt latency and/or find some better scheme to distribute the tasks & interrupts over the cores. I still don't know why core #1 already got an issue with just our 6 interrupts. This may be related to the SPIRAM fix adding overhead, but I've seen the FIFO overflows before applying the fix, and the CAN issue also has been present for over a year. My working hypothesis is some task/driver disabling interrupts for too often / too long, or non-IRAM ISRs getting delayed for too often / too long. And it seems this is an issue especially on core #1. Please verify & comment. Regards, Michael PS: a little side mystery: the netmanager task starts with priority 5, then gets raised to 22 (probably a mutex priority inheritance), but never gets lowered back to 5 afterwards. A FreeRTOS bug? -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

2 11

Metrics: Cabin Fan
by jaunius＠gmx.com 06 Jan '21

06 Jan '21

Hi, I would like to bring back this topic. Back in 2019 we discussed on how the standard metrics for climate and cabin should look like. I remember we initially discussed of adding cabinfan as number of steps, but latter decided to go with percentage. However as it is still better for the user to show the same scale as in car, we need to relay the number of steps available. I added this metric a while to Nissan Leaf. And it works great for me. So would suggest to add it to the standard metrics as well like this. v.e.cabinfansteps [integer] Number of cabin fan speed steps. Kind Regards, Jaunius Sent using the GMX mail app

2 2

Hello from a new subscriber
by Steve Davies 01 Dec '20

01 Dec '20

Hi, I'm Stephen Davies. Mark suggested that I say hello on the list, so: "Hi!". I own a 2019 BMW I3S BEV and I'm keen to try to add support into OVMS. (I live in Cape Town in South Africa - the i3 is the one very few EVs sold here) I've done a fair amount of work reverse-engineering messages from a couple of apps that can already talk to the I3 on OBD2 port. So I have a fair list of PIDs from that. My OVMS box arrived the other day. I built an ovms-server and I've got my OVMS box talking to my server and through to the IOS OVMS app. So the next step will be to set up to be able to build the firmware from source and then to start adding the i3... Can I ask one question in conclusion: how do I get the OVMS IOS app to do a TLS connection? I told it to use port 6870 but it does not do a TLS handshake. Do OVMS devs hang out in an IRC room, Slack or some such? Thanks, Steve

2 1

DBC primer
by Michael Balzer 01 Dec '20

01 Dec '20

Everyone, I'm in contact to a guy who would like to add support for the Renault Kangoo ZE. He's no developer, but has managed to collect quite some info on the Kangoo's CAN / OBD2 communication. The Kangoo has lots of live data frames, and partially even works with the Zoe module. I told him he can define a DBC file, but saw we didn't have any documentation on this approach yet. So I've written a small primer: https://docs.openvehicles.com/en/latest/components/vehicle_dbc/docs/dbc-pri… But I haven't used DBC myself except for writing this primer now, so if someone has more experience, I'd appreciate a second look. An example on how to trigger and decode OBD2 reads would also be nice, but I think he'll manage without for now. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

3 3

ELM327 codes to
by Soko 27 Nov '20

27 Nov '20

Heya, Another thing I can't figure out at the moment. I have to poll the VWeUp and I figured out (hopefully) how to do that. But I have no clue how to translate the "usual" ELM327 Header+PID to one entry for the OvmsVehicle::poll_pid_t The only value I've understood is the polltime. On my OBD2 interfaces I use a PID of 0x22028C for SoC. But PID is here only 16 bit! Where do I put the header 7E5? What is txmoduleid and rxmoduleid? I can't believe that I'm the first one with this issue ;) Couldn't find any doc though :( So I'm very thankful for any pointing in the correct direction/doc/example. thanks Soko

7 88

OVMS on Apple Silicon Mac
by Mark Webb-Johnson 23 Nov '20

23 Nov '20

Here’s OVMS iOS App running on an Apple Silicon MacBook Air. Installs and runs like an iPad App, either in a window or full screen. Notifications come in nicely, and preference settings in Open Vehicles / Preferences menu. Works well, with zero changes. Regards, Mark

2 2

Travis CI
by Chris 18 Nov '20

18 Nov '20

Hi, Google couldn't find anything in the archives. Due to my failure to test a build before a pull request I was thinking of adding a Travis CI build file to ensure pull requests actually compile. Is there a reason this hasn't already been done? Is this something that you'd be interested in? Thanks, Chris.

2 1

Test message
by Mark Webb-Johnson 18 Nov '20

18 Nov '20

Test message to your mailing list. Please ignore. Regards, Mark.

1 1

Fwd: [openvehicles/Open-Vehicle-Monitoring-System-3] OVMSDEV mailing list - is it still functioning? (#467)
by Michael Balzer 17 Nov '20

17 Nov '20

Mark, I could not forward the mail below to you, your server responded with… <mark(a)webb-johnson.net>: host mailtp5.network-box.com[175.45.17.210] said: 554 command rejected for policy reasons (using messageid: 1d117116-28cb-11eb-af9f-7e589b37a1b9) (in reply to end of DATA command) …which is probably a local issue, because my server is on no block list. Can you have a look at the(se) issue(s)? Regards, Michael -------- Weitergeleitete Nachricht -------- Betreff: Re: [openvehicles/Open-Vehicle-Monitoring-System-3] OVMSDEV mailing list - is it still functioning? (#467) Datum: Tue, 17 Nov 2020 03:06:05 -0800 Von: elbow <notifications(a)github.com> Antwort an: openvehicles/Open-Vehicle-Monitoring-System-3 <reply+AAUU2QPIWGN6ZRPGNWBB7LV5X6HZ3EVBNHHCYREIHM(a)reply.github.com> An: openvehicles/Open-Vehicle-Monitoring-System-3 <Open-Vehicle-Monitoring-System-3(a)noreply.github.com> Kopie (CC): Michael Balzer <dexter(a)expeedo.de>, Mention <mention(a)noreply.github.com> Hi @dexterbg <https://github.com/dexterbg> @markwj <https://github.com/markwj> , The situation with the list is unchanged. I also received bounces from my mails to lists.openvehicles.com: `This is the mail system at host nbkfmailscanhq5.network-box.com. I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below. For further assistance, please send mail to postmaster. If you do so, please include this problem report. You can delete your own text from the attached returned message. |The mail system | < ovmsdev-owner(a)lists.openvehicles.com <mailto:ovmsdev-owner@lists.openvehicles.com>>: connect to 10.8.12.188[10.8.12.188]:25: Connection refused ` So the mx targets for lists.openvehicles.com are "network-box.com" machines. Apparently they can't forward to mail on? For me I just want to be added to the dev mailing list. Can it be done by hand? I'm making good progress reverse engineering I3 canbus pids. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/467…>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUU2QMN7LH2HBRPKWYU5ZDSQ…>. -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

1 0