<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Mark, List,<br>
<br>
I just checked in some changes that have improved my GPS streaming
stability on all my latest test drives to 100%.<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92">https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92</a><br>
<br>
I still don't know why or how the modem rx buffer full condition
could lead to real crashes, but eliminating those also eliminated
the crashes.<br>
<br>
A major part in the buffer filling up was played by the massive
delays needed on net_msg_start() to make sure the modem is ready
without polling for it's status prompts. That indirectly also played
a role in the other problem source, being my network functions
assuming the modem is basically "ready" if net_msg_sendpending is 0.<br>
<br>
But it is not ready if it still processes some command -- like the
GPS NMEA request. I now insert another hard coded delay to make sure
the modem is ready.<br>
<br>
These hard delays don't really feel good and they do add up. Any net
msg now already needs at least 1.5 seconds just for the
net_msg_start() part, so this also disturbs the ticker timing.<br>
<br>
I think we could eliminate most hard delays by introducing some sort
of modem status following. A simple inline modem reply waiting loop
might already solve most problems. This could basically be a loop of
net_poll() calls with exit condition on checking the buffer for "OK"
/ "ERROR" / other prompts -- maybe with an additional timeout.<br>
<br>
Problem could be this net_poll() loop would run outside the main
loop, so it would omit all "idle" and "ticker" callbacks until the
response comes in (or timeout occurs).<br>
<br>
The benefit would be a much tighter and faster coupling of the
modem, and being able to react according to the modem status
responses for certain commands.<br>
<br>
Have there been any thoughts on this before? Shall I follow this
path a bit more or isn't it worth the effort?<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">Am 01.05.2013 15:05, schrieb Mark
Webb-Johnson:<br>
</div>
<blockquote
cite="mid:9F9F889F-030D-4051-8DA2-714ABA9CB20C@webb-johnson.net"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
Michael,
<div><br>
</div>
<div>No hardware flow control in OVMS circuit. But, we are only
9600baud and interrupt-driven - I think it should be ok.</div>
<div><br>
</div>
<div>My concern is RAM - say we have two channels open, we're
going to need 2 more line buffers.</div>
<div><br>
</div>
<div>Here is the latest 'map' file:</div>
<div><br>
</div>
<div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div> <font face="Andale Mono"> Section
Type Address Location Size(Bytes)</font></div>
<div><font face="Andale Mono"> ---------
--------- --------- --------- ---------</font></div>
<div><font face="Andale Mono"> TX_CRYPTO
udata 0x000100 data 0x000100<br>
RX_CRYPTO udata 0x000200 data
0x000100<br>
PM_CRYPTO udata 0x000300 data
0x000100<br>
.stack udata 0x000c00 data
0x000100<br>
.udata_crypt_hmac.o udata 0x000400 data
0x0000d8<br>
NETMSG_SP udata 0x000500 data
0x0000c8<br>
NETBUF_SP udata 0x000600 data
0x0000c8<br>
NETBUF udata 0x000700 data
0x0000c8<br>
.udata_UARTIntC.o udata 0x000800 data
0x0000c7<br>
LOGGING udata 0x000060 data
0x000088<br>
SFR_UNBANKED1 udata 0x000f80 data
0x000080<br>
.idata_ovms.o idata 0x000900 data
0x000070<br>
vehicle_overlay_data udata 0x000970 data
0x00006e<br>
.idata_net_sms.o idata 0x000a00 data
0x000063<br>
SFR_BANKED12 udata 0x000f00 data
0x000060<br>
SFR_BANKED11 udata 0x000e20 data
0x000060<br>
.idata_net_msg.o idata 0x000a63 data
0x000045<br>
.udata_crypt_md5.o udata 0x000aa8 data
0x000040<br>
.idata_crypt_md5.o idata 0x000b00 data
0x000040<br>
.idata_net.o idata 0x0005c8 data
0x000035<br>
.udata_ovms.o udata 0x0006c8 data
0x000030<br>
.idata_vehicle.o idata 0x0007c8 data
0x00002b<br>
.udata_net_msg.o udata 0x0004d8 data
0x000026<br>
.udata_params.o udata 0x0009de data
0x000020<br>
.idata_diag.o idata 0x0008c7 data
0x00001b<br>
SFR_UNBANKED0 udata 0x000f60 data
0x000018<br>
.idata_vehicle_twizy.o idata 0x0000e8 data
0x000014<br>
MATH_DATA udata 0x000020 data
0x000014<br>
.udata_vehicle.o udata 0x000ae8 data
0x000012<br>
SFR_BANKED2 udata 0x000d80 data
0x00000c<br>
SFR_BANKED1 udata 0x000d70 data
0x00000c<br>
SFR_BANKED0 udata 0x000d60 data
0x00000c<br>
.udata_c018i.o udata 0x0007f3 data
0x00000a<br>
.udata_crypt_base64.o udata 0x0006f8 data
0x000008<br>
SFR_BANKED6 udata 0x000de0 data
0x000008<br>
.udata_net_sms.o udata 0x0008e2 data
0x000007<br>
.idata_led.o idata 0x000afa data
0x000005<br>
SFR_BANKED7 udata 0x000df0 data
0x000004<br>
SFR_BANKED3 udata 0x000d90 data
0x000004<br>
.udata_net.o udata 0x0005fd data
0x000003<br>
.idata_stokpr.o idata 0x0009fe data
0x000002<br>
SFR_BANKED4 udata 0x000dd4 data
0x000002<br>
SEED_DATA idata 0x0004fe data
0x000002<br>
.idata_logging.o idata 0x0007fd data
0x000001<br>
SFR_BANKED10 udata 0x000dfc data
0x000001<br>
.udata_led.o udata 0x000aff data
0x000001<br>
SFR_BANKED9 udata 0x000dfa data
0x000001<br>
SFR_BANKED8 udata 0x000df8 data
0x000001<br>
SFR_BANKED5 udata 0x000dd8 data
0x000001<br>
DELAYDAT1 udata 0x000034 data
0x000001</font></div>
</blockquote>
</div>
<div><br>
</div>
<div>Stack definitely seems separately accounted for. The
TX_CRYPTO and RX_CRYPTO are the rc4 contexts. The PM_CRYPTO is
for Paranoid Mode (a lot of RAM for that, considering how few
use it).</div>
<div><br>
</div>
<div>The crypt_hmac needs 64 bytes for iPad, 64 bytes for oPad,
then 70 bytes for a MD5_CTX. Looking at the code, a lot could be
saved. It looks like the iPad and oPad could be merged (slightly
slower code, but saves a lot). Then, the pad and context could
be on the stack (134 bytes of 256 bytes) - scary but a big win.</div>
<div><br>
</div>
<div>The crypt_md5 PADDING (64 bytes) seems to be able to be moved
to rom with little impact.</div>
<div><br>
</div>
<div>From there, the returns seem to diminish.</div>
<div><br>
</div>
<div>Regards, Mark.</div>
<div><br>
</div>
<div>
<div>
<div>On 1 May, 2013, at 6:18 PM, Michael Balzer <<a
moz-do-not-send="true" href="mailto:dexter@expeedo.de">dexter@expeedo.de</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Mark,<br>
<br>
I also see the current modem communication layer as a
problem source. The read buffer only takes 128 bytes,
which will not be sufficient for many situations -- e.g. a
GPS response now needs already two NMEA sentences adding
up to about 108 bytes -- come in another modem answer
before the buffer gets handled, and bang we go...<br>
<br>
The multiplex mode is quite interesting, didn't know this
before -- although a really fully asynchronous modem comm
handling would not need that, I think. But it would need
more buffer RAM...<br>
<br>
Hm... the SIM908 manual says: "In Multiplex mode, it is
recommended to use the hardware flow control."<br>
<br>
As far as I understand the circuit schemes the current
hardware does not connect RTS/CTS (pins 66+67) -- right?<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 30.04.2013 03:28, schrieb
Mark Webb-Johnson:<br>
</div>
<blockquote
cite="mid:605E7C89-F120-47B1-967E-BEB2D8AD7942@webb-johnson.net"
type="cite">Michael,
<div><br>
</div>
<div>Looking at the design for 3G modules, I noticed
something recommended for those modules that seems to
be supported for the SIM900 base we use (both SIM900
OVMS v1 and SIM908 OVMS v2). CMUX mode.</div>
<div><br>
</div>
<blockquote class="webkit-indent-blockquote"
style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div><a moz-do-not-send="true"
href="http://www.m2m-platforms.com/seminars/2007seminar_material/Telit_CMUX_2_M2M_Platforms_seminar_2007.pdf">http://www.m2m-platforms.com/seminars/2007seminar_material/Telit_CMUX_2_M2M_Platforms_seminar_2007.pdf</a></div>
</blockquote>
<div><br>
</div>
<div>This is GSM 7.10 TE-MS multiplexor protocol.</div>
<div><br>
</div>
<div>The main problem we have is co-ordinating all the
different comms channels of OVMS onto a single serial
link to the modem. Incoming SMSes, status indicators,
outgoing GPRS TCP data, etc, all run over each other.</div>
<div><br>
</div>
<div>The CMUX protocol appears to allow us to have 3 (or
4?) virtual async links, over a single physical async
port. We could use one for SMS, another for TCP, and a
third for GPS polling. The 3G modules even allow the
GPS NMEA data to be streamed back over one of the
virtual ports (but I don't think that is available in
the SIM908 module we use - hard to tell as the CMUX
documents focus on SIM900 not SIM908 [the SIM908
embeds SIM900 and adds GPS]).</div>
<div><br>
</div>
<div>It would mean some major restructuring of the comms
layer (NET), and we would have to build a small
library to encode/decode GSM 7.10, but may be a much
more stable approach in the long-term.</div>
<div><br>
</div>
<div>SIMCOM seem to implement a subset of the full GSM
7.10 functionality. Here's the manual on it for
SIM900:</div>
<div><br>
</div>
<blockquote class="webkit-indent-blockquote"
style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div><a moz-do-not-send="true"
href="http://www.mt-system.ru/sites/default/files/sim900_multiplexer_user_manual_application_note_v1.3.pdf">http://www.mt-system.ru/sites/default/files/sim900_multiplexer_user_manual_application_note_v1.3.pdf</a></div>
<div><br>
</div>
<div><span><Mail Attachment.png></span></div>
</blockquote>
<div><br>
</div>
<div>Regards, Mark.</div>
<div><br>
<div>
<div>On 29 Apr, 2013, at 11:48 PM, Michael Balzer
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Am
28.04.2013 17:06, schrieb Michael Balzer:<br>
<blockquote cite="mid:517D3AED.40206@expeedo.de"
type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
I just checked in my fix for the USSD reply
handler. My tests with streaming enabled did
not produce any more crashes, so I think that
was the problem.<br>
</blockquote>
<br>
I was wrong, it's still crashing. I have to dig
deeper it seems.<br>
<br>
Has anyone observed crashes with streaming mode
on other vehicles, or is this a Twizy specific
bug now?<br>
<br>
Thanks,<br>
Michael<br>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
</pre>
</div>
<span><dexter.vcf></span>_______________________________________________<br>
OvmsDev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a><br>
<a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OvmsDev mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
</pre>
</div>
<span><dexter.vcf></span>_______________________________________________<br>
OvmsDev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a><br>
<a class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a>
<a class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
</pre>
</body>
</html>