[Ovmsdev] Framework DBC
Olivier
ogrums at gmail.com
Sun Jan 27 02:52:23 HKT 2019
Hi Mark,
I was looking for ideas to implement the code to decode and encode CAN
Messages, I found this interesting :
https://github.com/jmagnuson/canparse
Is doing what we want to do, what do you think ?
I have a question about this :
The dbcSignal name is defined in the DBC as the name of an OVMS metric.
Each dbcSignal object contains an m_metric, as a pointer to an OVMS metric,
that can be set during initialisation based on a simple lookup of the
metric name. That is in dbcSignal::AssignMetric(). I have done that in the
dbcSignal constructor and SetName() functions, although it makes our DBC
slightly non-standard conformant by allowing periods (“.”) in identifiers.
Example sample dbc file :
VERSION "A version string°"
BO_ 2364539904 EEC1 : 8 Vector__XXX
SG_ Engine_Speed : 24|16 at 1+ (0.125,0) [0|8031.88] "rpm" Vector__XXX
BA_ "SingleFrame" BO_ 2364539904 0;
BA_ "SPN" SG_ 2364539904 Engine_Speed 190;
CM_ BO_ 2364539904 "Engine Controller";
CM_ SG_ 2364539904 Engine_Speed "A description for Engine speed.";
the metric can be dynamically build like :
dbc + NodeName + SignalName => dbc.eec1.engine_Speed
But is that case, I don't know how I can manage to use this metric on
charts because if the dbc file is not loaded, the metric doesn't exist. And
if I want to use standard metric how can we map/link signal in dbc file to
standard metric ?
Or It's just I idea, with the config vehicle, we define custom metrics we
need in the code, and find a way to save somewhere matrix like :
custom.metric.a = byte for signal A from Message J
regards,
Olivier
Le dim. 9 déc. 2018 à 15:41, Mark Webb-Johnson <mark at webb-johnson.net> a
écrit :
> Olivier,
>
> 1. do you know how to config the dbc file to "compile" data from 2 Btye
> not contigu ?
>
>
> Nope, sorry, I don’t think the DBC file can handle that. See more of my
> thoughts on this below.
>
> 2. The next step do after dbc load, How I can use information the value
> to map or link with Standard metric ?
>
>
> I haven’t gotten to that bit yet, but have written (and just committed) a
> stub implementation, in case you (or others) have the time to help out.
> Otherwise, I’ll get to it after my work converting retools to use DBC.
>
> None of the following has been implemented yet (except for stubs, and a
> very basic pure DBC vehicle implementation):
>
>
> - The dbcSignal name is defined in the DBC as the name of an OVMS
> metric. Each dbcSignal object contains an m_metric, as a pointer to an OVMS
> metric, that can be set during initialisation based on a simple lookup of
> the metric name. That is in dbcSignal::AssignMetric(). I have done that in
> the dbcSignal constructor and SetName() functions, although it makes our
> DBC slightly non-standard conformant by allowing periods (“.”) in
> identifiers.
>
> - The dbcSignal contains a Encode(dbcNumber& source, struct
> CAN_frame_t* msg) function to encode a particular dbcNumber source number
> into the specified message. We would use this if we want to transmit data
> based on a dbc file. I don’t think implementing this is at all urgent, as
> that is not something we normally do (simulate a vehicle from within the
> ovms module firmware). This needs to be implemented in an efficient manner,
> supporting signed and unsigned numbers, and little and big endian types, as
> well as floats; but no hurry at all for this.
>
> - The dbcSignal contains a Decode(struct CAN_frame_t& msg) function to
> decode a particular CAN frame, and return a dbcNumber for the specified
> value. This is urgent now, and high up on my list to implement. However, it
> is non-trivial to do this in an efficient way, dealing with
> signed/unsigned, big/little endian, integers/floats, etc. The DBC
> specification treats everything as ‘double’, which would be horrendously
> inefficient.
>
> - There is also a DecodeMetric() in dbcSignal that can be built on top
> of Decode(), to decode straight into the signal’s ovms metric.
>
> - I have implemented the stub of a component vehicle_dbc, which
> implements class OvmsVehicleDBC derived from OvmsVehicle. The idea is that
> vehicles based on DBC files will do a few special things:
>
> - Derive from OvmsVehicleDBC, rather than OvmsVehicle.
> - Provide DBC file in the vehicle source code directory, one for
> each CAN bus.
> - Embed those DBC files in the usual way (see web server for an
> example).
> - Rather than calling the usual OvmsVehicle::RegisterCanBus() to
> register each CAN bus, instead call OvmsVehicleDBC::RegisterCanBusDBC()
> with the extra parameter to specify the DBC file in memory.
> - At the start of the usual IncomingFrameCan1(), call
> OvmsVehicleDBC::IncomingFrameCan1 to allow the DBC system to look for and
> decode metrics from that incoming frame, based on the DBC file.
> - After the OvmsVehicleDBC::IncomingFrameCan1 does its work,
> control passes back to the vehicle to do whatever custom work it requires
> (such as decode messages not supported by the DBC format, etc).
> - IncomingFrameCan2, and IncomingFrameCan3 work in a similar way.
> - The main thing remaining to implement is the decoding of messages
> into metrics (via dbcSignal).
>
> - I have also implemented a base pure DBC based vehicle type. That
> takes a DBC file stored on disk managed by the MyDBC system), and implement
> a vehicle with ZERO coding. You just configure the name of the DBC file,
> and everything is automatic from then onwards. For example:
>
> # config set vehicle dbc.can1 ts
> # vehicle module dbc
> I (36084) vehicle_dbc: Pure DBC vehicle module
> I (36084) vehicle_dbc: Registering can bus #1 as DBC ts
>
> - Finally, we should also probably extend the DBC format to do some of
> the common stuff we do that the format doesn’t support. Character strings
> (such as VIN numbers) for example.
>
>
> The next step is the dbcSignal::Decode() and dbcSignal::DecodeMetric()
> functions.
>
> Regards, Mark.
>
> On 8 Dec 2018, at 5:08 PM, Olivier <ogrums at gmail.com> wrote:
>
> Hi Mark,
>
> Before the update code is working to but with some error, I will fix my
> dbc file to add signal for all message.
>
> I have two question :
> 1. do you know how to config the dbc file to "compile" data from 2 Btye
> not contigu ?
> Example : ID 00FDF040 8 0x85 0x85 0x81 0x6A 0x7A 0xD9 0x4C 0x2E // ID
> message contain 5 Cells Voltage
> Cell 1 : #Byte 0 (0x85) | #LowByte 5 (0x9) => 0x859 => 2137 * 0,0015 =
> 3,2055Volts
> Cell 2 : #Byte 1 (0x85) | #HighByte 5 (0xD) => 0x85D
> Cell 3 : #Byte 2 (0x81) | #LowByte 6 (0xC) => 0x81C
> Cell 4 : #Byte 3 (0x6A) | #HighByte 7 (0x4) => 0x6A4
> Cell 5 : #Byte 4 (0x7A) | #LowByte 8 (0xE) => 0x7AE
> the last Byte is the ID of BMS Board (0x2?)
>
> 2. The next step do after dbc load, How I can use information the value
> to map or link with Standard metric ?
>
> Thank you
> Olivier
>
> Le ven. 7 déc. 2018 à 05:00, <ovmsdev-request at lists.openvehicles.com> a
> écrit :
>
>> Send OvmsDev mailing list submissions to
>> ovmsdev at lists.openvehicles.com
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>> or, via email, send a message with subject or body 'help' to
>> ovmsdev-request at lists.openvehicles.com
>>
>> You can reach the person managing the list at
>> ovmsdev-owner at lists.openvehicles.com
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of OvmsDev digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: OvmsDev Digest, Vol 83, Issue 3 (Mark Webb-Johnson)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 7 Dec 2018 09:19:41 +0800
>> From: Mark Webb-Johnson <mark at webb-johnson.net>
>> To: OVMS Developers <ovmsdev at lists.openvehicles.com>
>> Subject: Re: [Ovmsdev] OvmsDev Digest, Vol 83, Issue 3
>> Message-ID: <4F23B219-BDC5-4DA2-BAED-CC8380ABC720 at webb-johnson.net>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Olivier,
>>
>> Glad to see it is working again. I re-worked the dbc parsing code
>> yesterday to allow sections to be in any order in the DBC (although the
>> formal specification defines a specific order). That fixed your problem
>> with loading the DBC?
>>
>> I tried your provided Vectrix_VX1.dbc with the day before yesterday?s
>> 1bce372b20d4a79fb633faa6b96574604d95272f code, and get a segmentation fault
>> on parsing it. With today?s master head, no problems and the file seems to
>> load ok. The problem is the line 'BO_ 2164185677 FrontSensorRevision: 8
>> FRONTSensor? - a message (BO_) without any signals (SG_). The newer code
>> doesn?t care, but the older code couldn?t handle it.
>>
>> Regarding opendbc, yes I am aware of that. A very good resource. But,
>> most (if not all) of those DBCs there have illegal structure. I think they
>> are only used for human (not machine) reference. Things like section terms
>> mis-spelled, missing semi-colons, values referring to signals or messages
>> that don?t exist, etc. It is a good resource for testing broken DBC files
>> ;-)
>>
>> Regards, Mark.
>>
>> > On 7 Dec 2018, at 4:37 AM, Olivier <ogrums at gmail.com> wrote:
>> >
>> > Hello Mark,
>> >
>> > It's strange, I have re-clone again to be sure about IDF,
>> >
>> > $ git describe --tags --dirty
>> > v3.1-dev-2835-g151269458
>> >
>> > And now both is working, SD read and load dbc
>> >
>> > I have add my dbc file, and if you want to test the is this git :
>> https://github.com/commaai/opendbc <https://github.com/commaai/opendbc>
>> >
>> > regards,
>> > Olivier
>> >
>> > Le jeu. 6 d?c. 2018 ? 05:16, <ovmsdev-request at lists.openvehicles.com
>> <mailto:ovmsdev-request at lists.openvehicles.com>> a ?crit :
>> > Send OvmsDev mailing list submissions to
>> > ovmsdev at lists.openvehicles.com <mailto:
>> ovmsdev at lists.openvehicles.com>
>> >
>> > To subscribe or unsubscribe via the World Wide Web, visit
>> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev <
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>> > or, via email, send a message with subject or body 'help' to
>> > ovmsdev-request at lists.openvehicles.com <mailto:
>> ovmsdev-request at lists.openvehicles.com>
>> >
>> > You can reach the person managing the list at
>> > ovmsdev-owner at lists.openvehicles.com <mailto:
>> ovmsdev-owner at lists.openvehicles.com>
>> >
>> > When replying, please edit your Subject line so it is more specific
>> > than "Re: Contents of OvmsDev digest..."
>> >
>> >
>> > Today's Topics:
>> >
>> > 1. Re: Issue to store en SD Card after upgrade to idf 3.2-Dev
>> > (Pull Request 2# from M4GNV5/master) (Mark Webb-Johnson)
>> >
>> >
>> > ----------------------------------------------------------------------
>> >
>> > Message: 1
>> > Date: Thu, 6 Dec 2018 08:59:57 +0800
>> > From: Mark Webb-Johnson <mark at webb-johnson.net <mailto:
>> mark at webb-johnson.net>>
>> > To: OVMS Developers <ovmsdev at lists.openvehicles.com <mailto:
>> ovmsdev at lists.openvehicles.com>>
>> > Subject: Re: [Ovmsdev] Issue to store en SD Card after upgrade to idf
>> > 3.2-Dev (Pull Request 2# from M4GNV5/master)
>> > Message-ID: <5B58BB15-BEB4-4B9E-AAFC-E98DFA176748 at webb-johnson.net
>> <mailto:5B58BB15-BEB4-4B9E-AAFC-E98DFA176748 at webb-johnson.net>>
>> > Content-Type: text/plain; charset="utf-8"
>> >
>> > And related to this, if anyone else has any DBC files, please send them
>> to me so that I can test them against our DBC support library (to test for
>> robustness and completeness).
>> >
>> > Regards, Mark
>> >
>> > > On 6 Dec 2018, at 7:54 AM, Mark Webb-Johnson <mark at webb-johnson.net
>> <mailto:mark at webb-johnson.net>> wrote:
>> > >
>> > > Olivier,
>> > >
>> > > I think this is two different issues.
>> > >
>> > > SD card issue
>> > >
>> > > Are there any system-level errors shown in the logs when you try
>> this? Please try on local console to see what is shown there.
>> > >
>> > > Are you powered from USB or vehicle power?
>> > >
>> > > Can you show the output of ?config list sdcard? and ?sd status??
>> > >
>> > > DBC load issue
>> > >
>> > > Can you send me the Vectrix.dbc file you are using, please. Most
>> likely something in the structure of this file is causing the crash. It
>> seems that one of the messages (BO_) cannot be parsed correctly.
>> > >
>> > > What version of IDF are you using now? Just master head, right?
>> > >
>> > > Regards, Mark.
>> > >
>> > >> On 6 Dec 2018, at 4:29 AM, Olivier <ogrums at gmail.com <mailto:
>> ogrums at gmail.com> <mailto:ogrums at gmail.com <mailto:ogrums at gmail.com>>>
>> wrote:
>> > >>
>> > >> Hello,
>> > >>
>> > >> I have some issue with SD Card on OVMS 3.1 I don't know how to fix
>> the "bug"
>> > >>
>> > >> I can't write or read file on SD Card, but I can make directory :
>> > >>
>> > >> OVMS# vfs ls /sd
>> > >> [DIR] 05-Nov-2018 20:46 logs/
>> > >> 13.5k 23-Nov-2018 19:45 Vectrix_bak.dbc
>> > >> 14.0k 11-Nov-2018 22:04 Vectrix.dbc
>> > >> 13 01-Jan-1980 00:00 FOO.TXT
>> > >> 13.5k 23-Nov-2018 19:45 vx1.dbc
>> > >>
>> > >> OVMS# vfs mkdir /sd/testdir
>> > >> VFS directory created
>> > >>
>> > >> OVMS# vfs ls /sd
>> > >> [DIR] 05-Nov-2018 20:46 logs/
>> > >> 13.5k 23-Nov-2018 19:45 Vectrix_bak.dbc
>> > >> 14.0k 11-Nov-2018 22:04 Vectrix.dbc
>> > >> [DIR] 05-Dec-2018 21:14 testdir/
>> > >> 13 01-Jan-1980 00:00 FOO.TXT
>> > >> 13.5k 23-Nov-2018 19:45 vx1.dbc
>> > >>
>> > >> OVMS# config backup /sd/cfg.zip
>> > >> Creating config backup '/sd/cfg.zip'...
>> > >> Error: zip failed: No such file or directory
>> > >>
>> > >> OVMS# vfs rmdir /sd/testdir
>> > >> VFS directory removed
>> > >>
>> > >> And If I try do load dbc file it's crash
>> > >>
>> > >> OVMS# dbc load test /sd/vx1.dbc
>> > >>
>> > >> Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception
>> was unhandled.
>> > >> Core 1 register dump:
>> > >> PC : 0x40271023 PS : 0x00060730 A0 : 0x800e38f1
>> A1 : 0x3ffd4540
>> > >> 0x40271023: dbcMessage::SetID(unsigned int) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc.cpp:935
>> > >>
>> > >> A2 : 0x00000000 A3 : 0x80feda4d A4 : 0x3ffe5360
>> A5 : 0x3ffe5363
>> > >> A6 : 0x00000000 A7 : 0x00000000 A8 : 0x00000020
>> A9 : 0x3f4055b0
>> > >> A10 : 0x00000001 A11 : 0x3f4061ce A12 : 0x3f81ec50
>> A13 : 0x00000000
>> > >> A14 : 0x3ffd4524 A15 : 0x00000000 SAR : 0x00000008
>> EXCCAUSE: 0x0000001d
>> > >> EXCVADDR: 0x0000001c LBEG : 0x4009bd69 LEND : 0x4009bd9d
>> LCOUNT : 0xffffffff
>> > >>
>> > >> 0x4009bd69: memcpy at
>> /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:291
>> > >> 0x4009bd9d: memcpy at
>> /home/jeroen/esp8266/esp32/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/machine/xtensa/../../../../.././newlib/libc/machine/xtensa/memcpy.S:324
>> > >>
>> > >> Backtrace: 0x40271023:0x3ffd4540 0x400e38ee:0x3ffd4560
>> 0x400dfeba:0x3ffd4ea0 0x400dea49:0x3ffd4ee0 0x400dec48:0x3ffd4f80
>> 0x400f118a:0x3ffd4fa0 0x400f129b:0x3ffd4fd0 0x400f129b:0x3ffd5000 0x400f12
>> > >> d1:0x3ffd5030 0x400ecae8:0x3ffd5050 0x4010601d:0x3ffd5070
>> 0x4010608e:0x3ffd50d0 0x400ecb1f:0x3ffd5110 0x40271193:0x3ffd5130
>> 0x401010a9:0x3ffd5150 0x400fa5cd:0x3ffd5180 0x400fa7e1:0x3ffd51b0 0x401
>> > >> 00f35:0x3ffd51d0 0x40100f44:0x3ffd51f0 0x400f2489:0x3ffd5210
>> > >>
>> > >> 0x40271023: dbcMessage::SetID(unsigned int) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc.cpp:935
>> > >> 0x400e38ee: yyparse(void*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_parser.y:306
>> > >> 0x400dfeba: dbcfile::LoadFile(char const*, __sFILE*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc.cpp:935
>> > >> 0x400dea49: dbc::LoadFile(char const*, char const*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_app.cpp:198
>> > >> 0x400dec48: dbc_load(int, OvmsWriter*, OvmsCommand*, int, char
>> const* const*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_app.cpp:62
>> > >> 0x400f118a: OvmsCommand::Execute(int, OvmsWriter*, int, char const*
>> const*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73
>> > >> 0x400f129b: OvmsCommand::Execute(int, OvmsWriter*, int, char const*
>> const*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73
>> > >> 0x400f129b: OvmsCommand::Execute(int, OvmsWriter*, int, char const*
>> const*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73
>> > >> 0x400f12d1: OvmsCommandApp::Execute(int, OvmsWriter*, int, char
>> const* const*) at
>> ?????/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_command.cpp:73
>> > >>
>> > >> I Have try my SD Card on a PC is working, I have try example code
>> "sd_card_example_main.c" from SDK "esp-idf\examples\storage\sd_card\main",
>> It's working to, no crash.
>> > >> My guest, is the hardware is OK, esp-idf is OK, but they is
>> something in OVMS main code make this bug.
>> > >>
>> > >> I have delete both esp-idf, and OVMS folder and clone again,
>> compiled again, but it's the same issue.
>> > >>
>> > >> Thank you for your help.
>> > >> Olivier
>> > >> _______________________________________________
>> > >> OvmsDev mailing list
>> > >> OvmsDev at lists.openvehicles.com <mailto:
>> OvmsDev at lists.openvehicles.com> <mailto:OvmsDev at lists.openvehicles.com
>> <mailto:OvmsDev at lists.openvehicles.com>>
>> > >> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>> > >
>> > > _______________________________________________
>> > > OvmsDev mailing list
>> > > OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com
>> >
>> > > http://lists.openvehicles.com/mailman/listinfo/ovmsdev <
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>> >
>> > -------------- next part --------------
>> > An HTML attachment was scrubbed...
>> > URL: <
>> http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20181206/8ceb27f0/attachment-0001.html
>> <
>> http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20181206/8ceb27f0/attachment-0001.html
>> >>
>> >
>> > ------------------------------
>> >
>> > Subject: Digest Footer
>> >
>> > _______________________________________________
>> > OvmsDev mailing list
>> > OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev <
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>> >
>> >
>> > ------------------------------
>> >
>> > End of OvmsDev Digest, Vol 83, Issue 3
>> > **************************************
>> > <Vectrix_VX1.dbc>_______________________________________________
>> > OvmsDev mailing list
>> > OvmsDev at lists.openvehicles.com
>> > http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20181207/34756cd8/attachment-0001.html
>> >
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>> ------------------------------
>>
>> End of OvmsDev Digest, Vol 83, Issue 5
>> **************************************
>>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20190126/41ee4783/attachment-0001.html>
More information about the OvmsDev
mailing list