<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi Mark,<br>
<br>
Ok, this is interesting. Good idea to expand the status field; that
should at least give us space to note things of potential interest.
<br>
<br>
Regarding the RXB0 overflow status, I intentionally left that bit
out of the error status because it's not really an error. It just
notes that a frame was put into RXB1. If RXB1 overflows, then
there's an error, and we should increment the Rx overflow counter
and log the warning. No objection to including RXB0, since it can
be something of interest (sort of a warning that we got half way to
a data loss overflow), but that was my logic on setting the mask the
way it was.<br>
<br>
But it's interesting that in your test you did actually lose data
(rx overflow=3), but no data lost warning was issued. That's odd.
Also, the number of frames sent and received don't match up. We
appeared to have received more than was sent (24884-116 <
24777). But, Tx Interrupts = Rx pkts. Coincidence?<br>
<br>
Is there a way to compare sent and received data, to see what was
lost, for example, were they 3 random single frames, or two in a
row? Might give an idea of the size of the distraction that's
causing the overflow.<br>
<br>
Do we have any timing data from Tom's vehicle? Just wondering what
the inter-frame gaps are, and such. Since it locks up so quickly, a
Wireshark trace would be wonderful to pick through, or perhaps even
replay.<br>
<br>
Greg<br>
<br>
<br>
<div class="moz-cite-prefix">Mark Webb-Johnson wrote:<br>
</div>
<blockquote type="cite"
cite="mid:B8081404-D1E3-4652-9ECB-773512EB9485@webb-johnson.net">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
I’m trying to recreate this with my three-can-buses-connected DB9
plugged in. Transmitting on CAN1 should make it appear on CAN2 and
CAN3.
<div class=""><br class="">
</div>
<div class="">Here is what I see:</div>
<div class=""><br class="">
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;" class="">
<div class="">
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# can can1 start
active 1000000</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Can bus can1 started
in mode active at speed 1000000bps</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# can can2 start
active 1000000</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Can bus can2 started
in mode active at speed 1000000bps</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# test cantx can1
25000</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Testing 25000 frames
on can1</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Transmitted 25000
frames in 6.466209s = 258us/frame</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class=""><br class="">
</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# can can1 status</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">CAN: can1</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Mode: Active</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Speed: 1000000</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Interrupts:
24771</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx pkt:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx err:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx ovrflw:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx pkt:
24880</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx delays:
24703</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx err:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx ovrflw:
109</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Err flags: 0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# can can2 status</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class=""><br class="">
</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">CAN: can2</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Mode: Active</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Speed: 1000000</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Interrupts:
19084</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx pkt:
24770</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx err:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx ovrflw:
1</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx pkt:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx delays:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx err:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx ovrflw:
0</span></font></div>
<div class=""><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Err flags: 0x2040</span></font></div>
</div>
</blockquote>
<div class="">
<div><br class="">
</div>
<div>Note the err flags 0x2040 on CAN2, but the bus remains up
and working fine.</div>
<div><br class="">
</div>
<div>Repeating the test gives us:</div>
<div><br class="">
</div>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;" class="">
<div class="">
<div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# test cantx
can1 25000</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Testing 25000 frames
on can1</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Transmitted 25000
frames in 6.479670s = 259us/frame</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class=""><br class="">
</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# can can1
status</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">CAN: can1</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Mode: Active</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Speed: 1000000</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Interrupts:
49546</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx pkt:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx err:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx ovrflw:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx pkt:
49771</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx delays:
49417</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx err:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx ovrflw:
207</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Err flags: 0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class=""><br class="">
</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# can can2
status</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">CAN: can2</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Mode: Active</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Speed: 1000000</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Interrupts:
38288</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx pkt:
49545</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx err:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Rx ovrflw:
3</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx pkt:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx delays:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx err:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Tx ovrflw:
0</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">Err flags: 0x2040</span></font></div>
</div>
</div>
</blockquote>
<div class="">
<div><br class="">
</div>
<div>Looking at the mcp2515 code, it seems that Err flags are
only stored "if (intstat & 0b10100000)”. That is "ERRF
0x80 = message tx/rx error” or "ERRIF 0x20 = overflow / error
state change”. It is also set to "(intstat & 0b10100000)
<< 8 | errflag”, so doesn’t show all the error statuses.
It is hard to rely on that for other errors/status on lock-up.
Given that error_flags is a uint32_t, I think we can store
more in it to allow for better diagnosis.</div>
<div><br class="">
</div>
<div>I changed the mcp2515 driver to always set error_flags, on
each interrupt handled (except spurious interrupts with no
flags found), as follows:</div>
<div><br class="">
</div>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;" class="">
<div>
<div><font class="" face="Andale Mono"><span style="font-size:
14px;" class="">SSSSSSSSFFFFFFFF***EB01TLLLLLLLL</span></font></div>
<div>
<ul class="MailOutline">
<li class=""><span style="font-family: "Andale
Mono"; font-size: 14px;" class="">SSSSSSSS =
intstat</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">FFFFFFFF = errflag</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">B = RXB0 or RXB1
overflow flags cleared</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">0 = RXB0 overflowed</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">1 = RXB1 overflowed</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">T = TX buffer has
become available</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">E = Error/WakeUp
flags were cleared</span></li>
<li class=""><span style="font-size: 14px; font-family:
"Andale Mono";" class="">LLLLLLLL = intflag</span></li>
</ul>
</div>
</div>
</blockquote>
<div class="">
<div><br class="">
</div>
<div>I did find a problem on line 300: if (intstat &
0b10100000). It think that should be 0b11100000 (to also
pickup the RXB0 overflow), and removed
the m_status.rxbuf_overflow++ from RXB0 overflow (as it is not
really an overflow - as RXB1 got the data).</div>
<div><br class="">
</div>
<div>With those changes made, I get:</div>
<div><br class="">
</div>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;" class="">
<div>
<div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class="">OVMS# test cantx
can1 25000<br class="">
Testing 25000 frames on can1<br class="">
Transmitted 25000 frames in 6.389849s = 255us/frame</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class=""><br class="">
OVMS# can can1 status<br class="">
CAN: can1<br class="">
Mode: Active<br class="">
Speed: 1000000<br class="">
Interrupts: 24777<br class="">
Rx pkt: 0<br class="">
Rx err: 0<br class="">
Rx ovrflw: 0<br class="">
Tx pkt: 24884<br class="">
Tx delays: 24739<br class="">
Tx err: 0<br class="">
Tx ovrflw: 116<br class="">
Err flags: 0x00000000</span></font></div>
<div><font class="" face="Andale Mono"><span
style="font-size: 14px;" class=""><br class="">
OVMS# can can2 status<br class="">
CAN: can2<br class="">
Mode: Active<br class="">
Speed: 1000000<br class="">
Interrupts: 18935<br class="">
Rx pkt: 24777<br class="">
Rx err: 0<br class="">
Rx ovrflw: 0<br class="">
Tx pkt: 0<br class="">
Tx delays: 0<br class="">
Tx err: 0<br class="">
Tx ovrflw: 0<br class="">
Err flags: 0x01000001</span></font></div>
</div>
</div>
</blockquote>
<div class="">
<div><br class="">
</div>
<div>I don’t think I’ve fixed anything (apart from that minor
issue with RXB0 overflow diagnostics), but hopefully the new
error_flags display should help finding out what is causing
this lockup. Hopefully I haven’t broken anything.</div>
<div><br class="">
</div>
<div>Regards, Mark.</div>
<div><br class="">
</div>
<div class="">
<blockquote type="cite" class="">
<div class="">On 7 Jul 2018, at 10:42 AM, Tom Parker <<a
href="mailto:tom@carrott.org" class=""
moz-do-not-send="true">tom@carrott.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">On 07/07/18 00:05, Mark Webb-Johnson wrote:<br
class="">
<blockquote type="cite" class="">Err flags 0x2040. The
0x20 part is the error interrupt. The 0x40 part is
"RX0OVR: Receive Buffer 0 Overflow Flag bit”.<br
class="">
<br class="">
Where the number on ‘can can2 status’ moving at all?
Or completely stuck?<br class="">
</blockquote>
<br class="">
None of the can can2 status numbers change when the can
bus is broken. After power cycling it they move.<br
class="">
<br class="">
<blockquote type="cite" class="">Seems different than
the fault Greg and I are seeing. This one likely to be
interrupt flag, or buffer overflow, not being cleared
correctly. I’m guessing the overflow because that just
doesn’t seem correct in mcp2515::RxCallback(). I’ll
focus on that and have a look.<br class="">
</blockquote>
<br class="">
I just checked the car again and it stopped with Rx
ovrflw number only 1281, half what it got to last time.<br
class="">
<br class="">
_______________________________________________<br
class="">
OvmsDev mailing list<br class="">
<a href="mailto:OvmsDev@lists.openvehicles.com" class=""
moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a><br
class="">
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br
class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
</body>
</html>