I found this interesting :
https://github.com/jmagnuson/canparse <https://github.com/jmagnuson/canparse>
Our parser is loosely based on:
; With credit due to candbc-parser.y, for idea and structure ; https://github.com/Polyconseil/libcanardbc ; Copyright (C) 2007-2009 Andreas Heitmann
I have a question about this :
The dbcSignal name is defined in the DBC as the name of an OVMS metric.
I originally looked at several solutions to the issue: Change the DBC SG_ entry so that signal name is the ovms metric name. In your example, this would be: SG_ v.m.rpm : 24|16@1+ (0.125,0) [0|8031.88] "rpm" Vector__XXX Or if we want to keep this 100% pure DBC, then use: SG_ v_m_rpm : 24|16@1+ (0.125,0) [0|8031.88] "rpm” Vector__XXX and map ‘_’ to ‘.’ Add a comment to the DBC file, to provide the mapping of signal name to metric. In your example, this would be: SG_ Engine_Speed : 24|16@1+ (0.125,0) [0|8031.88] "rpm" Vector__XXX CM_ SG_ 2364539904 Engine_Speed “#ovms metric v.m.rpm"; Create a separate mapping (file or config) to map from DBC name to OVMS metrics. The solution I chose was #1. It seems the simplest and most direct. The dbc.{h,cpp} implementation of dbcSignal::SetName() already did this by trying to lookup the metric when the signal name was set. It required the signal name to be a (non standard) identifier (with periods ‘.’ in the identifier name). I have just now extended this to also support a mapping alternative (‘_’ -> ‘.’). So now you can use either ‘v.m.rpm’ or ‘v_m_rpm’ as the signal name (both will work). You should be able to try this now: SG_ v_m_rpm : 24|16@1+ (0.125,0) [0|8031.88] "rpm" Vector__XXX and you should see the speed metric (v.m.rpm) mapped correctly. The remaining problem I have is implementing the Encode/Decode functions in an efficient manner. The DBC specification says to convert everything to floating point, but that is incredibly innefficient. So, I’ve created dbcNumber that tries to abstract this out and efficiently perform mathematical operations between integers dbcNumbers without having to go through DBC floating point. Once the dbcSignal::Decode function has been implemented, the pure dbc IncomingFrameCanX handlers can be written (to call dbcSignal::DecodeMetric on each signal of the message) to update the metric appropriately. At that point, dbc based vehicles should start to work. At the moment, I have a pile of other projects I am working on (including the Retools conversion to work from DBCs, and the message chat function in iOS App). If you have time to work on the dbcSignal::Decode (and appropriate dbcNumber support), that would be wonderful. The issue is really just the bit masking and extraction for dbc signals (with two version - one for integers and a second for floats). I can handle the vehicle_dbc once the dbcSignal::Decode is working. Regards, Mark.
On 27 Jan 2019, at 2:52 AM, Olivier <ogrums@gmail.com> wrote:
Hi Mark,
I was looking for ideas to implement the code to decode and encode CAN Messages, I found this interesting :
https://github.com/jmagnuson/canparse <https://github.com/jmagnuson/canparse>
Is doing what we want to do, what do you think ?
I have a question about this :
The dbcSignal name is defined in the DBC as the name of an OVMS metric. Each dbcSignal object contains an m_metric, as a pointer to an OVMS metric, that can be set during initialisation based on a simple lookup of the metric name. That is in dbcSignal::AssignMetric(). I have done that in the dbcSignal constructor and SetName() functions, although it makes our DBC slightly non-standard conformant by allowing periods (“.”) in identifiers.
Example sample dbc file :
VERSION "A version string°"
BO_ 2364539904 EEC1 : 8 Vector__XXX SG_ Engine_Speed : 24|16@1 <mailto:24%7C16@1>+ (0.125,0) [0|8031.88] "rpm" Vector__XXX BA_ "SingleFrame" BO_ 2364539904 0; BA_ "SPN" SG_ 2364539904 Engine_Speed 190; CM_ BO_ 2364539904 "Engine Controller"; CM_ SG_ 2364539904 Engine_Speed "A description for Engine speed.";
the metric can be dynamically build like : dbc + NodeName + SignalName => dbc.eec1.engine_Speed
But is that case, I don't know how I can manage to use this metric on charts because if the dbc file is not loaded, the metric doesn't exist. And if I want to use standard metric how can we map/link signal in dbc file to standard metric ?
Or It's just I idea, with the config vehicle, we define custom metrics we need in the code, and find a way to save somewhere matrix like : custom.metric.a = byte for signal A from Message J
regards, Olivier
Le dim. 9 déc. 2018 à 15:41, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> a écrit : Olivier,
1. do you know how to config the dbc file to "compile" data from 2 Btye not contigu ?
Nope, sorry, I don’t think the DBC file can handle that. See more of my thoughts on this below.
2. The next step do after dbc load, How I can use information the value to map or link with Standard metric ?
I haven’t gotten to that bit yet, but have written (and just committed) a stub implementation, in case you (or others) have the time to help out. Otherwise, I’ll get to it after my work converting retools to use DBC.
None of the following has been implemented yet (except for stubs, and a very basic pure DBC vehicle implementation):
The dbcSignal name is defined in the DBC as the name of an OVMS metric. Each dbcSignal object contains an m_metric, as a pointer to an OVMS metric, that can be set during initialisation based on a simple lookup of the metric name. That is in dbcSignal::AssignMetric(). I have done that in the dbcSignal constructor and SetName() functions, although it makes our DBC slightly non-standard conformant by allowing periods (“.”) in identifiers.
The dbcSignal contains a Encode(dbcNumber& source, struct CAN_frame_t* msg) function to encode a particular dbcNumber source number into the specified message. We would use this if we want to transmit data based on a dbc file. I don’t think implementing this is at all urgent, as that is not something we normally do (simulate a vehicle from within the ovms module firmware). This needs to be implemented in an efficient manner, supporting signed and unsigned numbers, and little and big endian types, as well as floats; but no hurry at all for this.
The dbcSignal contains a Decode(struct CAN_frame_t& msg) function to decode a particular CAN frame, and return a dbcNumber for the specified value. This is urgent now, and high up on my list to implement. However, it is non-trivial to do this in an efficient way, dealing with signed/unsigned, big/little endian, integers/floats, etc. The DBC specification treats everything as ‘double’, which would be horrendously inefficient.
There is also a DecodeMetric() in dbcSignal that can be built on top of Decode(), to decode straight into the signal’s ovms metric.
I have implemented the stub of a component vehicle_dbc, which implements class OvmsVehicleDBC derived from OvmsVehicle. The idea is that vehicles based on DBC files will do a few special things:
Derive from OvmsVehicleDBC, rather than OvmsVehicle. Provide DBC file in the vehicle source code directory, one for each CAN bus. Embed those DBC files in the usual way (see web server for an example). Rather than calling the usual OvmsVehicle::RegisterCanBus() to register each CAN bus, instead call OvmsVehicleDBC::RegisterCanBusDBC() with the extra parameter to specify the DBC file in memory. At the start of the usual IncomingFrameCan1(), call OvmsVehicleDBC::IncomingFrameCan1 to allow the DBC system to look for and decode metrics from that incoming frame, based on the DBC file. After the OvmsVehicleDBC::IncomingFrameCan1 does its work, control passes back to the vehicle to do whatever custom work it requires (such as decode messages not supported by the DBC format, etc). IncomingFrameCan2, and IncomingFrameCan3 work in a similar way. The main thing remaining to implement is the decoding of messages into metrics (via dbcSignal).
I have also implemented a base pure DBC based vehicle type. That takes a DBC file stored on disk managed by the MyDBC system), and implement a vehicle with ZERO coding. You just configure the name of the DBC file, and everything is automatic from then onwards. For example:
# config set vehicle dbc.can1 ts # vehicle module dbc I (36084) vehicle_dbc: Pure DBC vehicle module I (36084) vehicle_dbc: Registering can bus #1 as DBC ts
Finally, we should also probably extend the DBC format to do some of the common stuff we do that the format doesn’t support. Character strings (such as VIN numbers) for example.
The next step is the dbcSignal::Decode() and dbcSignal::DecodeMetric() functions.
Regards, Mark.
On 8 Dec 2018, at 5:08 PM, Olivier <ogrums@gmail.com <mailto:ogrums@gmail.com>> wrote:
Hi Mark,
Before the update code is working to but with some error, I will fix my dbc file to add signal for all message.
I have two question : 1. do you know how to config the dbc file to "compile" data from 2 Btye not contigu ? Example : ID 00FDF040 8 0x85 0x85 0x81 0x6A 0x7A 0xD9 0x4C 0x2E // ID message contain 5 Cells Voltage Cell 1 : #Byte 0 (0x85) | #LowByte 5 (0x9) => 0x859 => 2137 * 0,0015 = 3,2055Volts Cell 2 : #Byte 1 (0x85) | #HighByte 5 (0xD) => 0x85D Cell 3 : #Byte 2 (0x81) | #LowByte 6 (0xC) => 0x81C Cell 4 : #Byte 3 (0x6A) | #HighByte 7 (0x4) => 0x6A4 Cell 5 : #Byte 4 (0x7A) | #LowByte 8 (0xE) => 0x7AE the last Byte is the ID of BMS Board (0x2?)
2. The next step do after dbc load, How I can use information the value to map or link with Standard metric ?
Thank you Olivier
Le ven. 7 déc. 2018 à 05:00, <ovmsdev-request@lists.openvehicles.com <mailto:ovmsdev-request@lists.openvehicles.com>> a écrit : Send OvmsDev mailing list submissions to ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com>
To subscribe or unsubscribe via the World Wide Web, visit http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev> or, via email, send a message with subject or body 'help' to ovmsdev-request@lists.openvehicles.com <mailto:ovmsdev-request@lists.openvehicles.com>
You can reach the person managing the list at ovmsdev-owner@lists.openvehicles.com <mailto:ovmsdev-owner@lists.openvehicles.com>
When replying, please edit your Subject line so it is more specific than "Re: Contents of OvmsDev digest..."
Today's Topics:
1. Re: OvmsDev Digest, Vol 83, Issue 3 (Mark Webb-Johnson)
----------------------------------------------------------------------
Message: 1 Date: Fri, 7 Dec 2018 09:19:41 +0800 From: Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> To: OVMS Developers <ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com>> Subject: Re: [Ovmsdev] OvmsDev Digest, Vol 83, Issue 3 Message-ID: <4F23B219-BDC5-4DA2-BAED-CC8380ABC720@webb-johnson.net <mailto:4F23B219-BDC5-4DA2-BAED-CC8380ABC720@webb-johnson.net>> Content-Type: text/plain; charset="utf-8"
Olivier,
Glad to see it is working again. I re-worked the dbc parsing code yesterday to allow sections to be in any order in the DBC (although the formal specification defines a specific order). That fixed your problem with loading the DBC?
I tried your provided Vectrix_VX1.dbc with the day before yesterday?s 1bce372b20d4a79fb633faa6b96574604d95272f code, and get a segmentation fault on parsing it. With today?s master head, no problems and the file seems to load ok. The problem is the line 'BO_ 2164185677 FrontSensorRevision: 8 FRONTSensor? - a message (BO_) without any signals (SG_). The newer code doesn?t care, but the older code couldn?t handle it.
Regarding opendbc, yes I am aware of that. A very good resource. But, most (if not all) of those DBCs there have illegal structure. I think they are only used for human (not machine) reference. Things like section terms mis-spelled, missing semi-colons, values referring to signals or messages that don?t exist, etc. It is a good resource for testing broken DBC files ;-)
Regards, Mark.
On 7 Dec 2018, at 4:37 AM, Olivier <ogrums@gmail.com <mailto:ogrums@gmail.com>> wrote:
Hello Mark,
It's strange, I have re-clone again to be sure about IDF,
$ git describe --tags --dirty v3.1-dev-2835-g151269458
And now both is working, SD read and load dbc
I have add my dbc file, and if you want to test the is this git : https://github.com/commaai/opendbc <https://github.com/commaai/opendbc> <https://github.com/commaai/opendbc <https://github.com/commaai/opendbc>>
regards, Olivier
Le jeu. 6 d?c. 2018 ? 05:16, <ovmsdev-request@lists.openvehicles.com <mailto:ovmsdev-request@lists.openvehicles.com> <mailto:ovmsdev-request@lists.openvehicles.com <mailto:ovmsdev-request@lists.openvehicles.com>>> a ?crit : Send OvmsDev mailing list submissions to ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com>>
To subscribe or unsubscribe via the World Wide Web, visit http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev> <http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>> or, via email, send a message with subject or body 'help' to ovmsdev-request@lists.openvehicles.com <mailto:ovmsdev-request@lists.openvehicles.com> <mailto:ovmsdev-request@lists.openvehicles.com <mailto:ovmsdev-request@lists.openvehicles.com>>
You can reach the person managing the list at ovmsdev-owner@lists.openvehicles.com <mailto:ovmsdev-owner@lists.openvehicles.com> <mailto:ovmsdev-owner@lists.openvehicles.com <mailto:ovmsdev-owner@lists.openvehicles.com>>
When replying, please edit your Subject line so it is more specific than "Re: Contents of OvmsDev digest..."
Today's Topics:
1. Re: Issue to store en SD Card after upgrade to idf 3.2-Dev (Pull Request 2# from M4GNV5/master) (Mark Webb-Johnson)
----------------------------------------------------------------------
Message: 1 Date: Thu, 6 Dec 2018 08:59:57 +0800 From: Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net <mailto:mark@webb-johnson.net>>> To: OVMS Developers <ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com> <mailto:ovmsdev@lists.openvehicles.com <mailto:ovmsdev@lists.openvehicles.com>>> Subject: Re: [Ovmsdev] Issue to store en SD Card after upgrade to idf 3.2-Dev (Pull Request 2# from M4GNV5/master) Message-ID: <5B58BB15-BEB4-4B9E-AAFC-E98DFA176748@webb-johnson.net <mailto:5B58BB15-BEB4-4B9E-AAFC-E98DFA176748@webb-johnson.net> <mailto:5B58BB15-BEB4-4B9E-AAFC-E98DFA176748@webb-johnson.net <mailto:5B58BB15-BEB4-4B9E-AAFC-E98DFA176748@webb-johnson.net>>> Content-Type: text/plain; charset="utf-8"
And related to this, if anyone else has any DBC files, please send them to me so that I can test them against our DBC support library (to test for robustness and completeness).
Regards, Mark
On 6 Dec 2018, at 7:54 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net> <mailto:mark@webb-johnson.net <mailto:mark@webb-johnson.net>>> wrote:
Olivier,
I think this is two different issues.
SD card issue
Are there any system-level errors shown in the logs when you try this? Please try on local console to see what is shown there.
Are you powered from USB or vehicle power?
Can you show the output of ?config list sdcard? and ?sd status??
DBC load issue
Can you send me the Vectrix.dbc file you are using, please. Most likely something in the structure of this file is causing the crash. It seems that one of the messages (BO_) cannot be parsed correctly.
What version of IDF are you using now? Just master head, right?
Regards, Mark.
On 6 Dec 2018, at 4:29 AM, Olivier <ogrums@gmail.com <mailto:ogrums@gmail.com> <mailto:ogrums@gmail.com <mailto:ogrums@gmail.com>> <mailto:ogrums@gmail.com <mailto:ogrums@gmail.com> <mailto:ogrums@gmail.com <mailto:ogrums@gmail.com>>>> wrote:
Hello,
I have some issue with SD Card on OVMS 3.1 I don't know how to fix the "bug"
I can't write or read file on SD Card, but I can make directory :
OVMS# vfs ls /sd [DIR] 05-Nov-2018 20:46 logs/ 13.5k 23-Nov-2018 19:45 Vectrix_bak.dbc 14.0k 11-Nov-2018 22:04 Vectrix.dbc 13 01-Jan-1980 00:00 FOO.TXT 13.5k 23-Nov-2018 19:45 vx1.dbc
OVMS# vfs mkdir /sd/testdir VFS directory created
OVMS# vfs ls /sd [DIR] 05-Nov-2018 20:46 logs/ 13.5k 23-Nov-2018 19:45 Vectrix_bak.dbc 14.0k 11-Nov-2018 22:04 Vectrix.dbc [DIR] 05-Dec-2018 21:14 testdir/ 13 01-Jan-1980 00:00 FOO.TXT 13.5k 23-Nov-2018 19:45 vx1.dbc
OVMS# config backup /sd/cfg.zip Creating config backup '/sd/cfg.zip'... Error: zip failed: No such file or directory
OVMS# vfs rmdir /sd/testdir VFS directory removed
And If I try do load dbc file it's crash
OVMS# dbc load test /sd/vx1.dbc
Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled. Core 1 register dump: PC : 0x40271023 PS : 0x00060730 A0 : 0x800e38f1 A1 : 0x3ffd4540 0x40271023: dbcMessage::SetID(unsigned int) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc.cpp:935
A2 : 0x00000000 A3 : 0x80feda4d A4 : 0x3ffe5360 A5 : 0x3ffe5363 A6 : 0x00000000 A7 : 0x00000000 A8 : 0x00000020 A9 : 0x3f4055b0 A10 : 0x00000001 A11 : 0x3f4061ce A12 : 0x3f81ec50 A13 : 0x00000000 A14 : 0x3ffd4524 A15 : 0x00000000 SAR : 0x00000008 EXCCAUSE: 0x0000001d EXCVADDR: 0x0000001c LBEG : 0x4009bd69 LEND : 0x4009bd9d LCOUNT : 0xffffffff
0x4009bd69: memcpy at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:291 0x4009bd9d: memcpy at /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:324
Backtrace: 0x40271023:0x3ffd4540 0x400e38ee:0x3ffd4560 0x400dfeba:0x3ffd4ea0 0x400dea49:0x3ffd4ee0 0x400dec48:0x3ffd4f80 0x400f118a:0x3ffd4fa0 0x400f129b:0x3ffd4fd0 0x400f129b:0x3ffd5000 0x400f12 d1:0x3ffd5030 0x400ecae8:0x3ffd5050 0x4010601d:0x3ffd5070 0x4010608e:0x3ffd50d0 0x400ecb1f:0x3ffd5110 0x40271193:0x3ffd5130 0x401010a9:0x3ffd5150 0x400fa5cd:0x3ffd5180 0x400fa7e1:0x3ffd51b0 0x401 00f35:0x3ffd51d0 0x40100f44:0x3ffd51f0 0x400f2489:0x3ffd5210
0x40271023: dbcMessage::SetID(unsigned int) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc.cpp:935 0x400e38ee: yyparse(void*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_parser.y:306 0x400dfeba: dbcfile::LoadFile(char const*, __sFILE*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc.cpp:935 0x400dea49: dbc::LoadFile(char const*, char const*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_app.cpp:198 0x400dec48: dbc_load(int, OvmsWriter*, OvmsCommand*, int, char const* const*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_app.cpp:62 0x400f118a: OvmsCommand::Execute(int, OvmsWriter*, int, char const* const*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73 0x400f129b: OvmsCommand::Execute(int, OvmsWriter*, int, char const* const*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73 0x400f129b: OvmsCommand::Execute(int, OvmsWriter*, int, char const* const*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73 0x400f12d1: OvmsCommandApp::Execute(int, OvmsWriter*, int, char const* const*) at ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73
I Have try my SD Card on a PC is working, I have try example code "sd_card_example_main.c" from SDK "esp-idf\examples\storage\sd_card\main", It's working to, no crash. My guest, is the hardware is OK, esp-idf is OK, but they is something in OVMS main code make this bug.
I have delete both esp-idf, and OVMS folder and clone again, compiled again, but it's the same issue.
Thank you for your help. Olivier _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> <mailto:OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com>> <mailto:OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> <mailto:OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev> <http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> <mailto:OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev> <http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>>