<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Michael,<div><br></div><div>I reviewed the patch. It seems fine.</div><div><br></div><div>When I originally wrote the net.{c,h} framework, I thought about doing some sort of send-expect (chat) control. It was just hard to implement with limited ram, while keeping the state based system. A lot of the lower state transitions are event based (most of the initialisation works that way), but once we get into the main online state it stays there. Is it possible to introduce some more READY states to handle this? Maybe even a transition system based on a modem result (i.e.; wait until you get string "OK" then go to this state, or time-out after N seconds)?</div><div><br></div><div>Last point is that some of the delays are because there is no response for some modem commands, and if you send too fast that causes a loss of data.</div><div><br></div><div>Regards, Mark.</div><div><br><div><div>On 8 May, 2013, at 2:52 AM, Michael Balzer <<a href="mailto:dexter@expeedo.de">dexter@expeedo.de</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
Mark, List,<br>
<br>
I just checked in some changes that have improved my GPS streaming
stability on all my latest test drives to 100%.<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92">https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92</a><br>
<br>
I still don't know why or how the modem rx buffer full condition
could lead to real crashes, but eliminating those also eliminated
the crashes.<br>
<br>
A major part in the buffer filling up was played by the massive
delays needed on net_msg_start() to make sure the modem is ready
without polling for it's status prompts. That indirectly also played
a role in the other problem source, being my network functions
assuming the modem is basically "ready" if net_msg_sendpending is 0.<br>
<br>
But it is not ready if it still processes some command -- like the
GPS NMEA request. I now insert another hard coded delay to make sure
the modem is ready.<br>
<br>
These hard delays don't really feel good and they do add up. Any net
msg now already needs at least 1.5 seconds just for the
net_msg_start() part, so this also disturbs the ticker timing.<br>
<br>
I think we could eliminate most hard delays by introducing some sort
of modem status following. A simple inline modem reply waiting loop
might already solve most problems. This could basically be a loop of
net_poll() calls with exit condition on checking the buffer for "OK"
/ "ERROR" / other prompts -- maybe with an additional timeout.<br>
<br>
Problem could be this net_poll() loop would run outside the main
loop, so it would omit all "idle" and "ticker" callbacks until the
response comes in (or timeout occurs).<br>
<br>
The benefit would be a much tighter and faster coupling of the
modem, and being able to react according to the modem status
responses for certain commands.<br>
<br>
Have there been any thoughts on this before? Shall I follow this
path a bit more or isn't it worth the effort?<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">Am 01.05.2013 15:05, schrieb Mark
Webb-Johnson:<br>
</div>
<blockquote cite="mid:9F9F889F-030D-4051-8DA2-714ABA9CB20C@webb-johnson.net" type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
Michael,
<div><br>
</div>
<div>No hardware flow control in OVMS circuit. But, we are only
9600baud and interrupt-driven - I think it should be ok.</div>
<div><br>
</div>
<div>My concern is RAM - say we have two channels open, we're
going to need 2 more line buffers.</div>
<div><br>
</div>
<div>Here is the latest 'map' file:</div>
<div><br>
</div>
<div>
<blockquote style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div> <font face="Andale Mono"> Section
Type Address Location Size(Bytes)</font></div>
<div><font face="Andale Mono"> ---------
--------- --------- --------- ---------</font></div>
<div><font face="Andale Mono"> TX_CRYPTO
udata 0x000100 data 0x000100<br>
RX_CRYPTO udata 0x000200 data
0x000100<br>
PM_CRYPTO udata 0x000300 data
0x000100<br>
.stack udata 0x000c00 data
0x000100<br>
.udata_crypt_hmac.o udata 0x000400 data
0x0000d8<br>
NETMSG_SP udata 0x000500 data
0x0000c8<br>
NETBUF_SP udata 0x000600 data
0x0000c8<br>
NETBUF udata 0x000700 data
0x0000c8<br>
.udata_UARTIntC.o udata 0x000800 data
0x0000c7<br>
LOGGING udata 0x000060 data
0x000088<br>
SFR_UNBANKED1 udata 0x000f80 data
0x000080<br>
.idata_ovms.o idata 0x000900 data
0x000070<br>
vehicle_overlay_data udata 0x000970 data
0x00006e<br>
.idata_net_sms.o idata 0x000a00 data
0x000063<br>
SFR_BANKED12 udata 0x000f00 data
0x000060<br>
SFR_BANKED11 udata 0x000e20 data
0x000060<br>
.idata_net_msg.o idata 0x000a63 data
0x000045<br>
.udata_crypt_md5.o udata 0x000aa8 data
0x000040<br>
.idata_crypt_md5.o idata 0x000b00 data
0x000040<br>
.idata_net.o idata 0x0005c8 data
0x000035<br>
.udata_ovms.o udata 0x0006c8 data
0x000030<br>
.idata_vehicle.o idata 0x0007c8 data
0x00002b<br>
.udata_net_msg.o udata 0x0004d8 data
0x000026<br>
.udata_params.o udata 0x0009de data
0x000020<br>
.idata_diag.o idata 0x0008c7 data
0x00001b<br>
SFR_UNBANKED0 udata 0x000f60 data
0x000018<br>
.idata_vehicle_twizy.o idata 0x0000e8 data
0x000014<br>
MATH_DATA udata 0x000020 data
0x000014<br>
.udata_vehicle.o udata 0x000ae8 data
0x000012<br>
SFR_BANKED2 udata 0x000d80 data
0x00000c<br>
SFR_BANKED1 udata 0x000d70 data
0x00000c<br>
SFR_BANKED0 udata 0x000d60 data
0x00000c<br>
.udata_c018i.o udata 0x0007f3 data
0x00000a<br>
.udata_crypt_base64.o udata 0x0006f8 data
0x000008<br>
SFR_BANKED6 udata 0x000de0 data
0x000008<br>
.udata_net_sms.o udata 0x0008e2 data
0x000007<br>
.idata_led.o idata 0x000afa data
0x000005<br>
SFR_BANKED7 udata 0x000df0 data
0x000004<br>
SFR_BANKED3 udata 0x000d90 data
0x000004<br>
.udata_net.o udata 0x0005fd data
0x000003<br>
.idata_stokpr.o idata 0x0009fe data
0x000002<br>
SFR_BANKED4 udata 0x000dd4 data
0x000002<br>
SEED_DATA idata 0x0004fe data
0x000002<br>
.idata_logging.o idata 0x0007fd data
0x000001<br>
SFR_BANKED10 udata 0x000dfc data
0x000001<br>
.udata_led.o udata 0x000aff data
0x000001<br>
SFR_BANKED9 udata 0x000dfa data
0x000001<br>
SFR_BANKED8 udata 0x000df8 data
0x000001<br>
SFR_BANKED5 udata 0x000dd8 data
0x000001<br>
DELAYDAT1 udata 0x000034 data
0x000001</font></div>
</blockquote>
</div>
<div><br>
</div>
<div>Stack definitely seems separately accounted for. The
TX_CRYPTO and RX_CRYPTO are the rc4 contexts. The PM_CRYPTO is
for Paranoid Mode (a lot of RAM for that, considering how few
use it).</div>
<div><br>
</div>
<div>The crypt_hmac needs 64 bytes for iPad, 64 bytes for oPad,
then 70 bytes for a MD5_CTX. Looking at the code, a lot could be
saved. It looks like the iPad and oPad could be merged (slightly
slower code, but saves a lot). Then, the pad and context could
be on the stack (134 bytes of 256 bytes) - scary but a big win.</div>
<div><br>
</div>
<div>The crypt_md5 PADDING (64 bytes) seems to be able to be moved
to rom with little impact.</div>
<div><br>
</div>
<div>From there, the returns seem to diminish.</div>
<div><br>
</div>
<div>Regards, Mark.</div>
<div><br>
</div>
<div>
<div>
<div>On 1 May, 2013, at 6:18 PM, Michael Balzer <<a moz-do-not-send="true" href="mailto:dexter@expeedo.de">dexter@expeedo.de</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Mark,<br>
<br>
I also see the current modem communication layer as a
problem source. The read buffer only takes 128 bytes,
which will not be sufficient for many situations -- e.g. a
GPS response now needs already two NMEA sentences adding
up to about 108 bytes -- come in another modem answer
before the buffer gets handled, and bang we go...<br>
<br>
The multiplex mode is quite interesting, didn't know this
before -- although a really fully asynchronous modem comm
handling would not need that, I think. But it would need
more buffer RAM...<br>
<br>
Hm... the SIM908 manual says: "In Multiplex mode, it is
recommended to use the hardware flow control."<br>
<br>
As far as I understand the circuit schemes the current
hardware does not connect RTS/CTS (pins 66+67) -- right?<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 30.04.2013 03:28, schrieb
Mark Webb-Johnson:<br>
</div>
<blockquote cite="mid:605E7C89-F120-47B1-967E-BEB2D8AD7942@webb-johnson.net" type="cite">Michael,
<div><br>
</div>
<div>Looking at the design for 3G modules, I noticed
something recommended for those modules that seems to
be supported for the SIM900 base we use (both SIM900
OVMS v1 and SIM908 OVMS v2). CMUX mode.</div>
<div><br>
</div>
<blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div><a moz-do-not-send="true" href="http://www.m2m-platforms.com/seminars/2007seminar_material/Telit_CMUX_2_M2M_Platforms_seminar_2007.pdf">http://www.m2m-platforms.com/seminars/2007seminar_material/Telit_CMUX_2_M2M_Platforms_seminar_2007.pdf</a></div>
</blockquote>
<div><br>
</div>
<div>This is GSM 7.10 TE-MS multiplexor protocol.</div>
<div><br>
</div>
<div>The main problem we have is co-ordinating all the
different comms channels of OVMS onto a single serial
link to the modem. Incoming SMSes, status indicators,
outgoing GPRS TCP data, etc, all run over each other.</div>
<div><br>
</div>
<div>The CMUX protocol appears to allow us to have 3 (or
4?) virtual async links, over a single physical async
port. We could use one for SMS, another for TCP, and a
third for GPS polling. The 3G modules even allow the
GPS NMEA data to be streamed back over one of the
virtual ports (but I don't think that is available in
the SIM908 module we use - hard to tell as the CMUX
documents focus on SIM900 not SIM908 [the SIM908
embeds SIM900 and adds GPS]).</div>
<div><br>
</div>
<div>It would mean some major restructuring of the comms
layer (NET), and we would have to build a small
library to encode/decode GSM 7.10, but may be a much
more stable approach in the long-term.</div>
<div><br>
</div>
<div>SIMCOM seem to implement a subset of the full GSM
7.10 functionality. Here's the manual on it for
SIM900:</div>
<div><br>
</div>
<blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding:
0px;">
<div><a moz-do-not-send="true" href="http://www.mt-system.ru/sites/default/files/sim900_multiplexer_user_manual_application_note_v1.3.pdf">http://www.mt-system.ru/sites/default/files/sim900_multiplexer_user_manual_application_note_v1.3.pdf</a></div>
<div><br>
</div>
<div><span><Mail Attachment.png></span></div>
</blockquote>
<div><br>
</div>
<div>Regards, Mark.</div>
<div><br>
<div>
<div>On 29 Apr, 2013, at 11:48 PM, Michael Balzer
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Am
28.04.2013 17:06, schrieb Michael Balzer:<br>
<blockquote cite="mid:517D3AED.40206@expeedo.de" type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
I just checked in my fix for the USSD reply
handler. My tests with streaming enabled did
not produce any more crashes, so I think that
was the problem.<br>
</blockquote>
<br>
I was wrong, it's still crashing. I have to dig
deeper it seems.<br>
<br>
Has anyone observed crashes with streaming mode
on other vehicles, or is this a Twizy specific
bug now?<br>
<br>
Thanks,<br>
Michael<br>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
</pre>
</div>
<span><dexter.vcf></span>_______________________________________________<br>
OvmsDev mailing list<br>
<a moz-do-not-send="true" href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a><br>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OvmsDev mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
</pre>
</div>
<span><dexter.vcf></span>_______________________________________________<br>
OvmsDev mailing list<br>
<a moz-do-not-send="true" href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a><br>
<a class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a>
<a class="moz-txt-link-freetext" href="http://lists.teslaclub.hk/mailman/listinfo/ovmsdev">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
</pre>
</div>
<span><dexter.vcf></span>_______________________________________________<br>OvmsDev mailing list<br><a href="mailto:OvmsDev@lists.teslaclub.hk">OvmsDev@lists.teslaclub.hk</a><br>http://lists.teslaclub.hk/mailman/listinfo/ovmsdev<br></blockquote></div><br></div></body></html>