Craig, The component.mk controls the compilation. It does: yacc -o $(COMPONENT_PATH)/yacclex/dbc_parser.cpp -d $(COMPONENT_PATH)/src/dbc_parser.y Expectation here is that the ‘-o’ produces dbc_parser.cpp and dbs_parser.hpp lex -o $(COMPONENT_PATH)/yacclex/dbc_tokeniser.c $(COMPONENT_PATH)/src/dbc_tokeniser.l This #include the dbc_parser.hpp and produced dbc_tokeniser.c Then dbc_tokeniser.c and dbc_parser.cpp are compiled as normal We use ‘yacclex’ as a temporary path, to keep these auto-generated code files separate. The behaviour of bison is that ‘-o’ outputs both .hpp and .cpp files (from the ‘-o’ directive). If think byacc may behave differently. We need both the header and code files, and they need to be named .hpp and .cpp as they need to produce c++ code. The dbc_tokeniser is in C, as Lex only really supports C and we can “extern C” the stuff we need when we access it (there is not much). Can you try running those commands manually, and see what comes out? Or try bison? Regards, Mark

On 25 Nov 2018, at 5:33 AM, Craig Leres <leres@xse.com> wrote:

On 11/22/18 8:38 PM, Mark Webb-Johnson wrote:

...

Could you guys check, and let me know if you see any issues? In particular, on windows. If no issues, I’ll proceed and finish up the grammar and implementation.

I'm unable to compile this under FreeBSD.

I think the first problem is really a FreeBSD bug. When !defined(__FreeBSD__) the generated code unconditionally defines __dead2 in flex.skl just before including stdio.h which includes sys/cdefs.h. But the toolchain version of sys/cdefs.h also defines __dead2. Here's the problem report I filed:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233481

After solving that my build fails with:

CC /home/ice/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/dbc/yacclex/dbc_tokeniser.o /home/ice/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_tokeniser.l:38:26: fatal error: dbc_parser.hpp: No such file or directory compilation terminated. gmake[1]: *** [/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/make/component_wrapper.mk:286: yacclex/dbc_tokeniser.o] Error 1

I suspect the makefile is assuming flex/lex will generate dbc_parser.hpp; mine generates dbc_parser.cpp. If i put in a symlink for dbc_parser.hppI can get incrementally further:

CC /home/ice/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/dbc/yacclex/dbc_tokeniser.o In file included from /home/ice/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_tokeniser.l:38:0: /home/ice/u0/leres/src/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/dbc/src/dbc_parser.y:43:18: fatal error: string: No such file or directory compilation terminated. gmake[1]: *** [/home/ice/u0/leres/esp/openvehicles-xtensa-esp32-elf/make/component_wrapper.mk:286: yacclex/dbc_tokeniser.o] Error 1 gmake: *** [/home/ice/u0/leres/esp/esp-idf/make/project.mk:468: component-dbc-build] Error 2

That looks c++ code in dbc_parser.y that's being compiled with the C compiler?

#include <string>

If I change it to string.h then I have problems with dbc.h:

#include <string> #include <map> #include <list>

And in this case it looks like dbc.h is included by both c++ and C sources.

Craig

Pulled it. A few minor fixes that I hope work for you: HOSTTYPE is not universally used, so I re-worked that (defaulting to yacc) Bison doesn’t seem to support YY_NO_UNPUT, but instead uses '%option nounput’, so I used both Also added some documentation in the readme file. Hopefully this is all ok now. I am proceeding with the actual implementation. Regards, Mark.

On 26 Nov 2018, at 2:53 AM, Craig Leres <leres@xse.com> wrote:

On 11/25/18 4:49 AM, Mark Webb-Johnson wrote:

...

Or try bison?

I already had bison installed so I changed the dbc component.mk file to use it on FreeBSD. Here's the pull request:

https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/176/fi...

As noted I think it would make more sense for YACC (and probably LEX) to be defined at a higher level. I couldn't figure out where CC and friends are defined so took the easy way out.

While I was making changes I added the YY_NO_INPUT and YY_NO_UNPUT defines to dbc_tokeniser.l.

Craig

The first implementation of this has been committed. It builds for me on Mac and Linux, but I had to do a ‘make clean’ on one of my boxes to make it compile properly (it was complaining about dbc_tokeniser.c missing - but that was old and replaced with dbc_tokeniser.cpp, so I guess an issue with a pre-compiled header somewhere). I’ve since tried updating the component.mk to workaround this (so it hopefully won’t happen again). The overall result looks quite good. The dbc_parser.y file is clean, and we now support the important VERSION, NS_, BS_, BU_, VAL_TABLE_, BO_, SG_, VAL_, and CM_ sections. There are others in the DBC standard, but not commonly used and of little use to us; we can simply add if required. Code size is roughly 25KB of flash (including the dbcapp for command handling). Using LEX/YACC after 30+ years was ‘interesting’. The DBC grammar is not too complex, and tokenisation rather trivial, so not too hard to port. The next step in parsing DBC can simply be derived from the current token, so YACC’s LR parser is probably overkill for this, but anyway it provides a good solid foundation for reading DBC files, and supporting extensions to the grammar over time. This is all in 3.1.011-60-g88dc859, which will go out tonight to ’test’ release level ota. At the moment, you can: dbc load - parse and load a DBC file into memory dbc unload - unload a DBC file from memory dbc list - show a summary of loaded DBC files dbc show - export a memory DBC file to DBC format The last of these ‘dbc show’ will change later to be a summary of the messages/signals, and the ‘dbc show’ functionality moved to a new ‘dbc save’ command. OVMS# dbc load TS /store/dbc/teslamodels_can3.dbc Loaded DBC TS ok OVMS# dbc list TS: 3 message(s), 42 signal(s), 92% coverage The 92% coverage figure is interesting. That is the number of signal bits decoded vs the total number of bits defined in messages. If all bus messages are in the DBC, it should give an indication of how much of the bus information has been decoded. To recap, here is the overall master plan: Support DBC files - loading and saving. These DBC files become the single place to document the broadcast message behaviour of each CAN bus for each vehicle type. Migrate RE tools to be fundamentally based on DBC files - support DBC multiplexors (rather than RE tools unique keys), show DBC signal values, and allow for maintenance of the DBC (and, for example, auto-creation of messages based on what is seen on the bus). This will all be real-time. Provide a web interface for RE tools and DBC file maintenance. Again, real-time (either from bus or simulated crtd captures). Provide vehicle module support for DBC files, loading, and acting on CAN bus messages to produce metrics without code. Implement something similar for OBDII-like polling architectures. So, now we are at step #1. Pretty much done. The only issue I have is that DBC likes to use floating point numbers for everything (such as minimum/maximum values, offsets and scale factors, etc). That is horrendously inefficient for step #4 (and not at all optimal for #2). So, it would be good to have an opaque type for ’number’ that automatically switches between integer and floating point maths for this. Onwards… Regards, Mark.