November 2022 - OvmsDev - Open Vehicles

Warning in Duktape
by Michael Geddes 09 Aug '24

09 Aug '24

There's an unused variable 'dcc' warning in the below code. It seems that the 'Perform the callback' is not being 'performed! Is this deliberate? //.ichael void DukOvmsCommandRegisterRun(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv) { ESP_LOGD(TAG, "DukOvmsCommandRegisterRun(%s)",cmd->GetName()); auto it = MyDuktape.m_cmdmap.find(cmd); if (it == MyDuktape.m_cmdmap.end()) { ESP_LOGE(TAG, "Command '%s' cannot be found in registry",cmd->GetName ()); return; } else { DuktapeConsoleCommand* dcc = it->second; // Perform the callback } }

3 6

Netmanager priority issue solved
by Michael Balzer 05 Jul '23

05 Jul '23

TL;DR: you need to pull my latest esp-idf changes. I've finally found & solved the strange priority changes for our netmanager task (i.e. being raised suddenly from origin 5 to 18/22): the bug was in the esp-idf posix threads mutex implementation. I had suspected the mutex priority inheritance for a while, so added a way to retrieve the internal mutex hold count for our task list. Using this I noticed the hold count would always & only raise by 2 whenever any kind of mongoose connection was closed. That lead me to checking the thread concurrency protection for mongoose mbufs, because every mongoose connection has two mbufs associated (rx & tx), each having a posix mutex. The bug was: posix mutexes were deleted after locking (taking) them. FreeRTOS mutexes must not be deleted while being taken, that breaks the priority inheritance (more precisely the disinheritance), with the visible effect being the mutex hold count not returning to zero. As a side effect, this may also solve the strange event task starvations (hope so…). I was investigating this as I suspected some busy loop in the netmanager context. With the netman running at prio 22, that would effectively block almost all other processing including the timer service. I've found & fixed one potential busy loop trigger in the netman that would have been caused by the netman task still running while all interfaces had been lost -- not sure if that could happen, but it would explain the effects. So please watch your crash debug info & report if the issue still turns up. Regards, Michael -- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26

5 19

SIM7600 Registration Denied
by Mark Webb-Johnson 18 Jan '23

18 Jan '23

I’ve been fighting a problem for the past few months with registration getting denied on T-Mobile in USA using Hologram and our new 4G SIM7600G modem. Without cellular connectivity at home, it hasn’t been easy to recreate this, but I found a solution: Finally, with a good high gain antenna on my external wall facing the nearest cellular tower a couple of km away, I can get cellular (and even GPS with the little black puck) connectivity. So back to the two issues we are having: The FPLMN blacklist in the modem lists a bunch of cellular networks we are blocked from connecting to (including the ones we need). There have been reports that the connection to hologram works and is maintained, but the OVMS never connects to the v2 server. To address #1, I have extended the “cellular cmd” command to try to capture the output from the modem. There is no simple way of knowing when the command has completed, so for the moment I just look at data returned from the modem on the command channel, and stop when it hasn’t changed for a second or so. That now works like this: OVMS# cellular cmd AT+CRSM=176,28539,0,0,12 +CRSM: 144,0,"FFFFFFFFFFFFFFFFFFFFFFFF" OK OVMS# cellular cmd ATI Manufacturer: SIMCOM INCORPORATED Model: SIMCOM_SIM7600G Revision: SIM7600M21-A_V2.0.1 SVN: 01 IMEI: (redacted) +GCAP: +CGSM OK If required, we can send a relatively simple command to clear the FPLMN blacklist. It seems to work well, but I still need to test it wider (especially over v2 protocol commands, where I think we might have a recursion problem). So on to issue #2 that has been much harder to find. Finally, working with Hologram, I discovered that the problem seems to be in our handling of CREG network registration messages. I found a module with this problem, and manually issuing commands I found: AT+CREG? Registration Denied AT+CGREG? Registration Denied AT+CEREG? Registered Roaming It seems that the E-UTRANS (some 4G LTE carriers) won’t show the registration status correctly in AT+CREG. I guess they are refusing GSM (3G) connections from us (aka ‘Registration Denied’) while allowing 4G LTE. But as we only use the AT+CREG status to know whether we are connected to the network, the result is we can’t detect the E-UTRANS LTE connection and think registration has been denied. Connected to Hologram, but no data (or v2 server) connection. I did some research, and can’t find a definitive answer to how best to do this (especially as we have both 3G and 4G options, still support 2G, and a lot seems to depend on the individual cellular carrier options). So I implemented it as best I could. I re-ordered the registration statuses to be in lowest priority order, then created m_netreg_d[3] to store the registration status for each of the three types. Finally, I changed it so that m_netreg reflects the highest registration status. That way if CREG says RegistrationDenied, and CEREG says RegisteredRoaming, I treat it as RegisteredRoaming. This seems to work (on my test bench at least), and I now plan to deploy for wider testing. Code is committed and built as 3.3.002-45-g42546636 on the main firmware site. I would gratefully appreciate feedback on this. Regards, Mark.

4 21

New Metric Units
by Michael Geddes 13 Dec '22

13 Dec '22

Hi all, Some of the code I copied from Kona/Kia code had both kwh/100km and km/kwh metrics in the code as 'Other'. Adding the various power consumption Units is not particularly hard (I will have a pull-request soon) - though the conversions between them all required some thought! ... but it also made me think these two metrics that are (with the consumption units added) defined like this: m_v_trip_consumption1 = MyMetrics.InitFloat("xiq.v.trip.consumption.KWh/100km", 10, 0, kWHP100K); m_v_trip_consumption2 = MyMetrics.InitFloat("xiq.v.trip.consumption.km/kWh", 10, 0, kPkWH); These are effectively the same metric but in different units! I'm wondering if we would be better to have scripting and Duktape support for converting metrics to different unit! This might be also quite useful for those strange countries that insist on using miles as a measurement. On top of the 'metric list' and 'metric set' we could add a 'metric get' which gets a single value.. and add unit support for get/set. I've also got a pull request that improves the precision of the km<->mi conversions and factors it out. //.ichael

4 47

Automatic naming of CAN log files
by Ludovic LANGE 02 Dec '22

02 Dec '22

Hi, I'm planning to implement an "automatic naming" of the CAN log files for the VFS target. It would also take care of the file name rotation according to some criterion (file size, and/or log duration). I've created an issue to follow it here https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/748 (and some others also) but let me first share what I have in mind: The use case is the following: I want to setup OVMS as a CAN bus datalogger. It should be always-on, always and continuously logging, event after a restart. (For that, my current solution is to use event scripts, cf https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/766 ) Logging to the same file, in that case, present some challenges: * file size would grow indefinitely, which could be then make it difficult to copy / network send somewhere * file would always be opened, which could prevent any action on it (if we want to send it somewhere else, or remove it) * in case of crash / corruption the whole file could be lost * in case of reboot you'd end overwriting the file. Of course we should change the file name, but it's not easy to do it in a script. It may be possible to do this with Duktape and `|OvmsCommand.Exec|` but I fear that we may miss events if we stop / start logging on a highly loaded bus. A proposal to solve this would be to add a new mode in `canlog_vfs.cpp` that would cater for this automatic naming (inspiration for this feature coming from another product). In this mode you don't pass the path as a parameter, but instead a `log prefix` that will be used to construct the file path. Syntax could be something like : `can log start vfs-auto crtd <log prefix> [filter1] ... [filterN]` (no filename). It would automatically generate a log file name, taking into account the splits / rotations that can be configured: * if asked for by configuration - rotate file name (= increment file name) after a file size is reached * if asked for by configuration - rotate file name (= increment file name) after a certain logging duration has been reached Naming would be 'automatic' with something like : `/sd/log/<vehicleid>/<#power on>/<log prefix>/<#rotations>.<log extension>` where: * `/sd/log/` is arbitrary. Could be coming from configuration if wanted (esp. for module without sd card) * `<vehicleid>` is there to help to declutter your sdcard (if you share it with multiple modules), or to declutter your storage server (when https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/749 has been completed). We may have to make it a valid filename. * `<#power on>` is the number of time the module has powered on ; it is an always incrementing number expressed as an integer on 8 digits. Depending on the configuration, some modules will be always-on ; others will stop more frequently - the number will overflow after 99.999.999 power on, which will certainly see a change of sd card, so no fear of unwanted overwriting of files * `<log prefix>` is a way to distinguish between multiple concurrent can logs active. * `<#rotations>` is a counter of rotations for this file and this power on ; it is an integer on 8 digits ; reset to `00000001` at each poweroff and that will increment each time there is a file name rotation. some configuration items cater for the file name rotation: * `split_size_mb` : when the file has reached this size, the file is closed, the element `<#rotations>` is incremented, and a new file with the 'automatic' name is opened. * `split_duration_s` : when the file has logged for this duration, the file is closed, the element `<#rotations>` is incremented, and a new file with the 'automatic' name is opened. example of file name: `/sd/log/mycar/00000024/can1-25k/00000005.crtd` (Of course we need a way to either stop logging once the VFS is full, or to auto-remove some log files under this path) Let me know if you have any thoughts / concerns about this. Regards, Ludovic

2 5

FYI: Historical Messages on api.openvehicles.com
by Mark Webb-Johnson 27 Nov '22

27 Nov '22

FYI We had an outage last night due to issues with an excessively ovms_historicalmessages table on the api.openvehicles.com <http://api.openvehicles.com/> server. Currently, that table has more than 150 million records, and has become largely unworkable due to its size. I have temporarily moved the historical records to another table, and resumed service. Now working to find out the root cause of the issue (likely a few vehicles abusing the system). Hopefully I will be able to start restoring the old historical data later today. Regards, Mark.

1 0

Fasttech: End of an era
by Mark Webb-Johnson 23 Nov '22

23 Nov '22

Just a short note to say that, effective immediately, OVMS will no longer be available from Fasttech. They have provided incredible support for the project over the past 10+ years, and we are immensely grateful for all that they have done. The team there have been fantastic, gone above and beyond, and I highly recommend them. While a few issues have led to the decision not to sell through Fasttech any more, primarily (with COVID-19 driving international logistics costs and shipment delays up), supporting OVMS modules sold globally through Fasttech has become prohibitively expensive and simply painful for everyone concerned (us, Fasttech, and the buyer). We are still a hobbyist project, without paid employees to handle support. I would far rather that users in Europe buy through OpenEnergyMonitor, and users in North America buy through MedlockAndSons - obtaining quick simple delivery and local support. The vast majority of our users are in EU and US. We will be looking for some other way of providing for and supporting Asia Pacific customers (probably out of Hong Kong, to keep warehousing and distribution as simple as possible). I will let you know once a decision there has been made, and if anyone has any suggestions please let me know. Regards, Mark.

3 4

Kia Soul - Incorrect/Inconsistent Charging State (Kia Soul EV 30KWh)
by Mark Webb-Johnson 22 Nov '22

22 Nov '22

Anybody able to help out with this one? https://www.openvehicles.com/node/3443 Incorrect/Inconsistent Charging State (Kia Soul EV 30KWh) Thanks, Mark

1 0

Fiat500e Quirks
by William Reading 18 Nov '22

18 Nov '22

Howdy there, I recently got an OVMS set up with my Fiat 500e and was happy to see that it works, but I noticed noticed a few oddities. The first one is that it uses Valet Mode (CommandActivateValet) for preconditioning instead of CommandClimateControl: https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/blob/bd192… <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/blob/bd192…> Is there a downside to changing the code to where it uses Climate Control instead? It seems like that would fit better with the intended functionality. The other is that it seems to rapidly generate spurious charging / not charging events, which en masse seems to make the websocket connection on the web server time out. I can put together a pull request to try to fix these issues, but was wondering if there are any thoughts on these two things before getting started. There's also no pictures of the Fiat on the iOS App, but I assume that's because the last release of the iOS App predates support. Thanks, William Reading

2 3

Strange crashes when logging additional info in can logs
by Ludovic LANGE 18 Nov '22

18 Nov '22

Dear list, You may remember that I'm trying to enhance the "datalogger" side of OVMS - in different ways. One facet of this endeavour is to have (CAN) logs that also incorporate other data coming from OVMS. In particular, I'd like to have GNSS (GPS) data in the same log as my CAN data. My first try is to slightly alter `canlog.cpp` and add the ability to `LogInfo` some chosen metrics, in the same spirit of the already logged subset of events. (With CRTD format, it appears as comments) While doing this, I'm facing multiple crashes that are somehow strange: * The crashes only occur when closing the logs (`can log stop`), never when logging * The cause of the crash is not always the same (LoadProhibited, InstrFetchProhibited) * Sometimes the PC is impacted / overwritten The crashes seems to always occur in a window of 20s after the `can log stop` command (In general it's immediate, but sometime can take some more seconds). (It lead me also to discover that the feature of our version of esp-idf that resolves the memory addresses in the serial output - `make monitor` - is broken, and that it's better to use GDB for that) While I am doing my homework and try to solve this, I wanted to share it here also to see if you already encountered this and/or if my changes are not done properly / not on the right OS task / etc.... - for which I lack background and experience. (In the process I think I found a non-freed variable, but it's not causing my issue - cf https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/774 ) Here are the changes - you'll see that the crash do occur as soon as I `LogInfo` something - after the log has been closed. ------------------------------------------------------------------------ modified: vehicle/OVMS.V3/components/can/src/canlog.cpp ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── @ vehicle/OVMS.V3/components/can/src/canlog.cpp:355 @ canlog::canlog(const char* type, std::string format, canformat::canformat_serve_ using std::placeholders::_1; using std::placeholders::_2; MyEvents.RegisterEvent(IDTAG, "*", std::bind(&canlog::EventListener, this, _1, _2)); + MyMetrics.RegisterListener(IDTAG.c_str(), "*", std::bind(&canlog::MetricListener, this, _1)); int queuesize = MyConfig.GetParamValueInt("can", "log.queuesize",100); m_queue = xQueueCreate(queuesize, sizeof(CAN_log_message_t)); @ vehicle/OVMS.V3/components/can/src/canlog.cpp:365 @ canlog::canlog(const char* type, std::string format, canformat::canformat_serve_ canlog::~canlog() { MyEvents.DeregisterEvent(IDTAG); + MyMetrics.DeregisterListener(IDTAG.c_str()); if (m_task) { @ vehicle/OVMS.V3/components/can/src/canlog.cpp:441 @ void canlog::EventListener(std::string event, void* data) LogInfo(NULL, CAN_LogInfo_Event, event.c_str()); } +void canlog::MetricListener(OvmsMetric* metric) { + std::string name = metric->m_name; + ESP_LOGI(TAG,"MetricListener - enter : %s", name.c_str()); + if (startsWith(name, "v.p.gps") || (name == "v.p.latitude") || (name == "v.p.longitude") || (name == "v.p.altitude") || (name == "v.p.direction") || (name == "v.p.satcount") || (name == "m.net.wifi.sq") || (name == "m.freeram")) { + std::string metric_text = "metric: " + name; + ESP_LOGI(TAG,"MetricListener - keep : %s", metric_text.c_str()); + LogInfo(NULL, CAN_LogInfo_Event, metric_text.c_str()); + } +} + const char* canlog::GetType() { return m_type; Note: "m.net.wifi.sq" / "m.freeram" are providing debugging data, so that I can quickly receive metrics after the boot. Not what I'm interested in for the final version of course, and the list will be configurable. ------------------------------------------------------------------------ Here are some example of panic handler infos: Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled. Core 1 register dump: PC : 0x400d95f8 PS : 0x00060830 A0 : 0x8030d32c A1 : 0x3ffc4980 A2 : 0x3ffeb5a8 A3 : 0x00000000 A4 : 0xfefefefe A5 : 0x3ffb5200 A6 : 0x3f805e78 A7 : 0x000000fe A8 : 0x800d95f0 A9 : 0x3ffc4930 A10 : 0x3ffeb5a8 A11 : 0x00000000 A12 : 0x00000009 A13 : 0x73c80b46 A14 : 0x3ffc4930 A15 : 0x3f8b4164 SAR : 0x00000004 EXCCAUSE: 0x0000001c EXCVADDR: 0xfefeff2e LBEG : 0x4008abe4 LEND : 0x4008ac00 LCOUNT : 0xffffffff ELF file SHA256: eb3e5ce8f67640c8 Backtrace: 0x400d95f8:0x3ffc4980 0x4030d329:0x3ffc49e0 0x4012122d:0x3ffc4a00 0x40121296:0x3ffc4a60 0x40120e6e:0x3ffc4a80 0x400ede1f:0x3ffc4aa0 0x400eed41:0x3ffc4ae0 0x400ecdc9:0x3ffc4b00 0x4011c5b6:0x3ffc4b40 0x4011c7b6:0x3ffc4b70 0x4011c8d8:0x3ffc4be0 0x0 0x400d95f8 is exactly the address of the call to LogInfo in my changes: (gdb) list *0x400d95f8 0x400d95f8 is in canlog::MetricListener(OvmsMetric*) (vehicle/OVMS.V3/components/can/src/canlog.cpp:447). 442 std::string name = metric->m_name; 443 ESP_LOGI(TAG,"MetricListener - enter : %s", name.c_str()); 444 if (startsWith(name, "v.p.gps") || (name == "v.p.latitude") || (name == "v.p.longitude") || (name == "v.p.altitude") || (name == "v.p.direction") || (name == "v.p.satcount") || (name == "m.net.wifi.sq") || (name == "m.freeram")) { 445 std::string metric_text = "metric: " + name; 446 ESP_LOGI(TAG,"MetricListener - keep : %s", metric_text.c_str()); *447 LogInfo(NULL, CAN_LogInfo_Event, metric_text.c_str());* 448 } 449 } 450 0xfefe / 0xfefefefe seems related to some of the Heap Memory Debugging options I've activated (Cf https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/s… ) and seems to indicate a possible use-after-free issue. I'll look to debug in that direction. Another version (before Heap Memory Debugging activated) did that from time to time: ------------------------------------------------------------------------ Guru Meditation Error: Core 0 panic'ed (InstrFetchProhibited). Exception was unhandled. Core 0 register dump: PC : 0x00008100 PS : 0x00060530 A0 : 0x802f0cf0 A1 : 0x3ffd0f00 A2 : 0x3ffeb214 A3 : 0x00008100 A4 : 0x402f1df8 A5 : 0x00000000 A6 : 0x00000003 A7 : 0x3ffb556c A8 : 0x800d9540 A9 : 0x3ffd0ee0 A10 : 0x3ffeb214 A11 : 0x00000000 A12 : 0x00000009 A13 : 0x3f8b4a24 A14 : 0xffffffff A15 : 0x40404040 SAR : 0x00000018 EXCCAUSE: 0x00000014 EXCVADDR: 0x00008100 LBEG : 0x4008a848 LEND : 0x4008a864 LCOUNT : 0xffffffff ELF file SHA256: f4d0c43aa48ef759 Backtrace: 0x00008100:0x3ffd0f00 0x402f0ced:0x3ffd0fb0 0x4011c56d:0x3ffd0fd0 0x4011c5c6:0x3ffd1020 0x4011c24a:0x3ffd1040 0x4014f51d:0x3ffd1060 0x4014f953:0x3ffd1080 0x4014e6e3:0x3ffd11d0 0x4014ef16:0x3ffd1210 0x4014f001:0x3ffd1230 0x4014f119:0x3ffd1260 0x4014e889:0x3ffd1290 0x4014e9ed:0x3ffd1310 0x4014ebf9:0x3ffd13f0 ------------------------------------------------------------------------ Thanks for any hint or advices you may want to share. Regards,

2 3