<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Should not be. I don’t intend to change the bits that just deal with frames (such as vehicles).<div class=""><br class=""></div><div class="">The CAN_frame_t is embedded in CAN_log_message_t, so it is trivial anyway.</div><div class=""><br class=""></div><div class="">Regards, Mark.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 18 Jun 2019, at 5:03 AM, Greg D. <<a href="mailto:gregd2350@gmail.com" class="">gregd2350@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div text="#000000" bgcolor="#FFFFFF" class="">
Hi Mark,<br class="">
<br class="">
I presume no changes to CAN_frame_t that would be visible to other
applications, e.g. OBD2ECU, right?<br class="">
<br class="">
Also, any chance that the changes to MCP2515 might help diagnose the
cause of our bus hangs?<br class="">
<br class="">
Greg<br class="">
<br class="">
<br class="">
<div class="moz-cite-prefix">Mark Webb-Johnson wrote:<br class="">
</div>
<blockquote type="cite" cite="mid:182AA6E6-0E33-41E4-B4BE-B378048B47BA@webb-johnson.net" class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div class=""><br class="">
</div>
I’ve completed most of the work of this restructuring, and the new
arrangement looks good. The new work uses CAN_log_message_t
(rather than CAN_frame_t) for most things other than basic frame
tx/rx, and that is based on Michael’s can log CAN_LogMsg_t (almost
unchanged, apart from timestamp). I’ve added a canfilter class to
be able to do software filtering (not just in logging, but also in
general). Then candump* has moved to canformat* to virtualise all
the different formats (CRTD, PCAP, etc) for can dumps (with a
modular registration of formats so we can add more formats easily,
without change to core components).
<div class=""><br class="">
</div>
<div class="">The last change I have to make is to canlog. I want
it to use the canfilter and canformat systems so all it is doing
is logging (and not involved in filtering, or formatting - both
of which are now general purpose facilities). It seems that the
‘re serve’ framework can move into this, and perhaps we can log
to things other than the file system.</div>
<div class=""><br class="">
</div>
<div class="">The changes were quite extensive:</div>
<div class=""><br class="">
</div>
<div class="">
<ul class="MailOutline">
<li class="">Everything in the can component</li>
<li class="">retools component</li>
<li class="">mcp2515 component</li>
<li class="">esp32can component</li>
<li class="">dbc component</li>
</ul>
<div class=""><br class="">
</div>
<div class="">I should be done with the changes in the next day or two,
and then will do some testing in my car. Hopefully should be
able to merge my commits later this week.</div>
<div class=""><br class="">
</div>
<div class="">Regards, Mark.</div>
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On 12 Jun 2019, at 9:00 PM, Mark Webb-Johnson
<<a href="mailto:mark@webb-johnson.net" class="" moz-do-not-send="true">mark@webb-johnson.net</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode:
space; line-break: after-white-space;" class="">I also
see the filter system in canlog could be very useful as
a general filtering system. I suggest to move that into
the central CAN class, and allow such software filters
to be attached to listeners, callbacks, and loggers.
<div class=""><br class="">
</div>
<div class="">By refactoring this, I think we can get
some very useful functionality.</div>
<div class=""><br class="">
</div>
<div class="">@Michael, one question, for you: What is
the purpose of RegisterCallback (and the callbacks vs
listeners, in general). Listeners get their frames
delivered via a queue (so must have a task running).
Callbacks are simple function callbacks running
synchronously with the CAN task. From what I can see,
the only user of callbacks is vehicle_twizy where
CanResponder is set as the callback on CAN1. Why is
that necessary, given that the standard vehicle object
has IncomingFrameCan1() to receive frames. Is this
required for performance (running synchronously with
the CAN receive task), or some other reason why it
can’t run from the vehicle task on queue receive of
the frame? In other words, why can’t we just call
CanResponder() from within IncomingFrameCan1()?</div>
<div class=""><br class="">
</div>
<div class="">Regards, Mark.<br class="">
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On 11 Jun 2019, at 8:49 PM, Mark
Webb-Johnson <<a href="mailto:mark@webb-johnson.net" class="" moz-do-not-send="true">mark@webb-johnson.net</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div style="word-wrap: break-word;
-webkit-nbsp-mode: space; line-break:
after-white-space;" class="">It has become a
little duplicated:
<div class=""><br class="">
</div>
<blockquote style="margin: 0 0 0 40px; border:
none; padding: 0px;" class="">
<div class="">
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">struct
CAN_frame_t</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""> {</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""> canbus*
origin; //
Origin of the frame</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">
CAN_FIR_t FIR;
// Frame information record</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">
uint32_t MsgID;
// Message ID</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""> union</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""> {</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">
uint8_t u8[8];
// Payload byte access</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">
uint32_t u32[2];
// Payload u32 access (Att: little
endian!)</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">
uint64_t u64;
// Payload u64 access (Att: little
endian!)</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""> }
data;</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""><br class="">
</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class="">
esp_err_t Write(canbus* bus=NULL,
TickType_t maxqueuewait=0); // bus:
NULL=origin</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""> };</span></font></div>
</div>
<div class=""><font class="" face="Andale
Mono"><span style="font-style: normal;
font-size: 14px;" class=""><br class="">
</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-size: 14px;" class="">typedef struct<br class="">
{<br class="">
uint32_t timestamp;<br class="">
canbus* bus;<br class="">
CAN_LogEntry_t type;<br class="">
union<br class="">
{<br class="">
CAN_frame_t frame;<br class="">
CAN_status_t status;<br class="">
char* text;<br class="">
};<br class="">
} CAN_LogMsg_t;</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-size: 14px;" class=""><br class="">
</span></font></div>
<div class=""><font class="" face="Andale
Mono"><span style="font-size: 14px;" class="">
<div class="">typedef struct</div>
<div class=""> {</div>
<div class=""> CAN_frame_t last;</div>
<div class=""> uint32_t rxcount;</div>
<div class=""> struct
__attribute__((__packed__))</div>
<div class=""> {</div>
<div class=""> struct {</div>
<div class=""> uint8_t Ignore:1;
// 0x01</div>
<div class=""> uint8_t Changed:1;
// 0x02</div>
<div class=""> uint8_t
Discovered:1; // 0x04</div>
<div class=""> uint8_t :1;
// 0x08</div>
<div class=""> uint8_t :1;
// 0x10</div>
<div class=""> uint8_t :1;
// 0x20</div>
<div class=""> uint8_t :1;
// 0x40</div>
<div class=""> uint8_t :1;
// 0x80</div>
<div class=""> } b;</div>
<div class=""> uint8_t dc;
// Data bytes changed</div>
<div class=""> uint8_t dd;
// Data bytes discovered</div>
<div class=""> uint8_t spare;</div>
<div class=""> } attr;</div>
<div class=""> } re_record_t;</div>
</span></font></div>
</blockquote>
<div class="">
<div class=""><br class="">
</div>
<div class="">CAN_frame_t has origin for the
can bus the message arrived on, while
CAN_LogMsg_t has frame.origin and bus
(presumably because bus is needed for
status and text messages).</div>
<div class=""><br class="">
</div>
<div class="">Then we have candump and
canlog virtual interfaces (canlog supports
’trace’ and ‘crtd’, while candump supports
‘crtd’ and ‘pap’).</div>
<div class=""><br class="">
</div>
</div>
<div class="">The users of these higher level
structures are RETOOLS and CANLOG. I suggest
the following changes:</div>
<div class=""><br class="">
</div>
<div class="">
<ul class="MailOutline">
<li class="">Have a single format
conversion virtual class, and then
initial implementations for pcap, crtd
and trace. That can virtualise the
conversion of a CAN message to be
logged/transmitted/received to/from the
external format. This can include a
factory for registration of types (by
textual name), as well as support
functions for command registration
(including whether they have support for
reading, writing, or both). This just
provides format conversion and does not
provide any file handling or
transmission functions. Canlog and
Candump get merged to form this.<br class="">
<br class="">
</li>
<li class="">Perhaps base this on
CAN_LogMsg_t, but I suggest a more
generic name (perhaps CAN_message_t, to
differentiate that this is a can message
rather than a simple frame). It is
CAN_frame_t + additional support for
stats and general text messages.<br class="">
<br class="">
</li>
<li class="">Tidy up the CAN_LogMsg_t bus
vs frame.origin issue.<br class="">
</li>
</ul>
</div>
<div class=""><br class="">
</div>
<div class="">Simply put, separate the
formatting into a standalone virtual
implementations, then leave retools and
canlog to handle the actual RE stuff or file
logging, or whatever else is required with
the data that gets converted.</div>
<div class=""><br class="">
</div>
<div class="">Does that make sense?</div>
<div class=""><br class="">
</div>
<div class="">Regards, Mark.</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On 11 Jun 2019, at 3:01
PM, Michael Balzer <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Mark,<br class="">
<br class="">
when implementing the can log
framework I introduced the
CAN_LogMsg_t specifically to capture
the original timestamp of the frame,
so it wouldn't get lost if<br class="">
the logging task gets behind, and to
transport other log data types than
just frames to the loggers.<br class="">
<br class="">
I would also expect the RE tools
stream to produce event info &
statistics entries like can log
does, as that is very helpful in
analysing. Maybe rebasing<br class="">
candump on can log would be better?<br class="">
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<br class="">
Am 11.06.19 um 08:37 schrieb Mark
Webb-Johnson:<br class="">
<blockquote type="cite" class="">This
really should be unified, and
probably not too hard to do. Crazy
to have two so similar frameworks.
I think we can just have one
virtual class for converting CAN
frames to/from other formats, and
then implementations for CRTD,
PCAP, etc.<br class="">
<br class="">
I do remember that last time I
looked at this, canlog used
CAN_LogMsg_t, and candump used
CAN_frame_t (with CAN_LogMsg_t
having some other fields for
logging statistics, etc), and that
made it non-trivial. Perhaps
canlog should be built on top of
candump?<br class="">
<br class="">
I will have a look at it.<br class="">
<br class="">
Regards, Mark.<br class="">
<br class="">
<blockquote type="cite" class="">On
6 Jun 2019, at 4:06 PM, Michael
Balzer <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>>
wrote:<br class="">
<br class="">
Steve,<br class="">
<br class="">
Mark quoted from the RE tools
candump_crtd class, which has
its own crtd formatter.<br class="">
<br class="">
For the canlog framework I
currently get the timestamp from
esp_log_timestamp(). Should be
easy to change that to
gettimeofday().<br class="">
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<br class="">
Am 05.06.19 um 18:39 schrieb
Stephen Casner:<br class="">
<blockquote type="cite" class="">Mark,<br class="">
<br class="">
If by "unix julian time" you
mean a timestamp like
1559751698 (the<br class="">
time as I write this), than
that is not what Greg and I
have observed<br class="">
in the CRTD logs. Here are
the first lines in our files:<br class="">
<br class="">
151766.574 CXX Info Type:crtd;
Path:'/sd/gps.crtd';
Filter:1:100-100;
Vehicle:TR2N;<br class="">
<br class="">
383603.293 CXX Info Type:crtd;
Path:'/sd/location.crtd';
Filter:1:100-100;
Vehicle:TR1N;<br class="">
<br class="">
Yet OVMS does have the current
time from NTP:<br class="">
<br class="">
OVMS# time<br class="">
Time Zone:<br class="">
UTC Time: 2019-06-05
16:35:45 UTC<br class="">
Local Time: 2019-06-05
16:35:45 GMT<br class="">
Provider: ntp<br class="">
<br class="">
PROVIDER STRATUM
UPDATE TIME<br class="">
*ntp 1
57 Wed Jun 5 16:35:44
2019<br class="">
<br class="">
Who calls candump_crtd::get()
with the timeval that goes
into the<br class="">
sprintf() you mention?<br class="">
<br class="">
-- Steve<br class="">
<br class="">
On Wed, 5 Jun 2019, Mark
Webb-Johnson wrote:<br class="">
<br class="">
<blockquote type="cite" class="">The CRTD timestamps
should be unix julian time
(not uptime). Produced by
this:<br class="">
<br class="">
sprintf(m_buf,"%ld.%06ld
%sR%s %0*X",<br class="">
time->tv_sec,
time->tv_usec,<br class="">
busnumber,<br class="">
(frame->FIR.B.FF ==
CAN_frame_std) ? "11":"29",<br class="">
(frame->FIR.B.FF ==
CAN_frame_std) ? 3 : 8,<br class="">
frame->MsgID);<br class="">
<br class="">
Once module gets time from
the vehicle (or gps), it
should show the correct
timestamps.<br class="">
<br class="">
Regards, Mark.<br class="">
</blockquote>
_______________________________________________<br class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br class="">
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" class="" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br class="">
</blockquote>
-- <br class="">
Michael Balzer * Helkenberger
Weg 9 * D-58256 Ennepetal<br class="">
Fon 02333 / 833 5735 * Handy
0176 / 206 989 26<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br class="">
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" class="" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br class="">
</blockquote>
_______________________________________________<br class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br class="">
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" class="" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br class="">
</blockquote>
<br class="">
-- <br class="">
Michael Balzer * Helkenberger Weg 9
* D-58256 Ennepetal<br class="">
Fon 02333 / 833 5735 * Handy 0176 /
206 989 26<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br class="">
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" class="" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
_______________________________________________<br class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class="" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br class="">
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
<br class="">
<fieldset class="mimeAttachmentHeader"></fieldset>
<br class="">
<pre wrap="" class="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br class="">
</div>
_______________________________________________<br class="">OvmsDev mailing list<br class=""><a href="mailto:OvmsDev@lists.openvehicles.com" class="">OvmsDev@lists.openvehicles.com</a><br class="">http://lists.openvehicles.com/mailman/listinfo/ovmsdev<br class=""></div></blockquote></div><br class=""></div></body></html>