[Ovmsdev] Crashes with streaming mode

Michael Balzer dexter at expeedo.de
Wed May 8 02:52:09 HKT 2013


Mark, List,

I just checked in some changes that have improved my GPS streaming 
stability on all my latest test drives to 100%.

https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92

I still don't know why or how the modem rx buffer full condition could 
lead to real crashes, but eliminating those also eliminated the crashes.

A major part in the buffer filling up was played by the massive delays 
needed on net_msg_start() to make sure the modem is ready without 
polling for it's status prompts. That indirectly also played a role in 
the other problem source, being my network functions assuming the modem 
is basically "ready" if net_msg_sendpending is 0.

But it is not ready if it still processes some command -- like the GPS 
NMEA request. I now insert another hard coded delay to make sure the 
modem is ready.

These hard delays don't really feel good and they do add up. Any net msg 
now already needs at least 1.5 seconds just for the net_msg_start() 
part, so this also disturbs the ticker timing.

I think we could eliminate most hard delays by introducing some sort of 
modem status following. A simple inline modem reply waiting loop might 
already solve most problems. This could basically be a loop of 
net_poll() calls with exit condition on checking the buffer for "OK" / 
"ERROR" / other prompts -- maybe with an additional timeout.

Problem could be this net_poll() loop would run outside the main loop, 
so it would omit all "idle" and "ticker" callbacks until the response 
comes in (or timeout occurs).

The benefit would be a much tighter and faster coupling of the modem, 
and being able to react according to the modem status responses for 
certain commands.

Have there been any thoughts on this before? Shall I follow this path a 
bit more or isn't it worth the effort?

Regards,
Michael



Am 01.05.2013 15:05, schrieb Mark Webb-Johnson:
> Michael,
>
> No hardware flow control in OVMS circuit. But, we are only 9600baud 
> and interrupt-driven - I think it should be ok.
>
> My concern is RAM - say we have two channels open, we're going to need 
> 2 more line buffers.
>
> Here is the latest 'map' file:
>
>               Section Type    Address   Location Size(Bytes)
>                     ---------  ---------  ---------  ---------  ---------
>                     TX_CRYPTO  udata   0x000100       data   0x000100
>                     RX_CRYPTO      udata   0x000200       data   0x000100
>                     PM_CRYPTO      udata   0x000300       data   0x000100
>                        .stack      udata   0x000c00       data   0x000100
>           .udata_crypt_hmac.o      udata   0x000400       data   0x0000d8
>                     NETMSG_SP      udata   0x000500       data   0x0000c8
>                     NETBUF_SP      udata   0x000600       data   0x0000c8
>                        NETBUF      udata   0x000700       data   0x0000c8
>             .udata_UARTIntC.o      udata   0x000800       data   0x0000c7
>                       LOGGING      udata   0x000060       data   0x000088
>                 SFR_UNBANKED1      udata   0x000f80       data   0x000080
>                 .idata_ovms.o      idata   0x000900       data   0x000070
>          vehicle_overlay_data      udata   0x000970       data   0x00006e
>              .idata_net_sms.o      idata   0x000a00       data   0x000063
>                  SFR_BANKED12      udata   0x000f00       data   0x000060
>                  SFR_BANKED11      udata   0x000e20       data   0x000060
>              .idata_net_msg.o      idata   0x000a63       data   0x000045
>            .udata_crypt_md5.o      udata   0x000aa8       data   0x000040
>            .idata_crypt_md5.o      idata   0x000b00       data   0x000040
>                  .idata_net.o      idata   0x0005c8       data   0x000035
>                 .udata_ovms.o      udata   0x0006c8       data   0x000030
>              .idata_vehicle.o      idata   0x0007c8       data   0x00002b
>              .udata_net_msg.o      udata   0x0004d8       data   0x000026
>               .udata_params.o      udata   0x0009de       data   0x000020
>                 .idata_diag.o      idata   0x0008c7       data   0x00001b
>                 SFR_UNBANKED0      udata   0x000f60       data   0x000018
>        .idata_vehicle_twizy.o      idata   0x0000e8       data   0x000014
>                     MATH_DATA      udata   0x000020       data   0x000014
>              .udata_vehicle.o      udata   0x000ae8       data   0x000012
>                   SFR_BANKED2      udata   0x000d80       data   0x00000c
>                   SFR_BANKED1      udata   0x000d70       data   0x00000c
>                   SFR_BANKED0      udata   0x000d60       data   0x00000c
>                .udata_c018i.o      udata   0x0007f3       data   0x00000a
>         .udata_crypt_base64.o      udata   0x0006f8       data   0x000008
>                   SFR_BANKED6      udata   0x000de0       data   0x000008
>              .udata_net_sms.o      udata   0x0008e2       data   0x000007
>                  .idata_led.o      idata   0x000afa       data   0x000005
>                   SFR_BANKED7      udata   0x000df0       data   0x000004
>                   SFR_BANKED3      udata   0x000d90       data   0x000004
>                  .udata_net.o      udata   0x0005fd       data   0x000003
>               .idata_stokpr.o      idata   0x0009fe       data   0x000002
>                   SFR_BANKED4      udata   0x000dd4       data   0x000002
>                     SEED_DATA      idata   0x0004fe       data   0x000002
>              .idata_logging.o      idata   0x0007fd       data   0x000001
>                  SFR_BANKED10      udata   0x000dfc       data   0x000001
>                  .udata_led.o      udata   0x000aff       data   0x000001
>                   SFR_BANKED9      udata   0x000dfa       data   0x000001
>                   SFR_BANKED8      udata   0x000df8       data   0x000001
>                   SFR_BANKED5      udata   0x000dd8       data   0x000001
>                     DELAYDAT1      udata   0x000034       data   0x000001
>
>
> Stack definitely seems separately accounted for. The TX_CRYPTO and 
> RX_CRYPTO are the rc4 contexts. The PM_CRYPTO is for Paranoid Mode (a 
> lot of RAM for that, considering how few use it).
>
> The crypt_hmac needs 64 bytes for iPad, 64 bytes for oPad, then 70 
> bytes for a MD5_CTX. Looking at the code, a lot could be saved. It 
> looks like the iPad and oPad could be merged (slightly slower code, 
> but saves a lot). Then, the pad and context could be on the stack (134 
> bytes of 256 bytes) - scary but a big win.
>
> The crypt_md5 PADDING (64 bytes) seems to be able to be moved to rom 
> with little impact.
>
> From there, the returns seem to diminish.
>
> Regards, Mark.
>
> On 1 May, 2013, at 6:18 PM, Michael Balzer <dexter at expeedo.de 
> <mailto:dexter at expeedo.de>> wrote:
>
>> Mark,
>>
>> I also see the current modem communication layer as a problem source. 
>> The read buffer only takes 128 bytes, which will not be sufficient 
>> for many situations -- e.g. a GPS response now needs already two NMEA 
>> sentences adding up to about 108 bytes -- come in another modem 
>> answer before the buffer gets handled, and bang we go...
>>
>> The multiplex mode is quite interesting, didn't know this before -- 
>> although a really fully asynchronous modem comm handling would not 
>> need that, I think. But it would need more buffer RAM...
>>
>> Hm... the SIM908 manual says: "In Multiplex mode, it is recommended 
>> to use the hardware flow control."
>>
>> As far as I understand the circuit schemes the current hardware does 
>> not connect RTS/CTS (pins 66+67) -- right?
>>
>> Regards,
>> Michael
>>
>>
>> Am 30.04.2013 03:28, schrieb Mark Webb-Johnson:
>>> Michael,
>>>
>>> Looking at the design for 3G modules, I noticed something 
>>> recommended for those modules that seems to be supported for the 
>>> SIM900 base we use (both SIM900 OVMS v1 and SIM908 OVMS v2). CMUX mode.
>>>
>>>     http://www.m2m-platforms.com/seminars/2007seminar_material/Telit_CMUX_2_M2M_Platforms_seminar_2007.pdf
>>>
>>>
>>> This is GSM 7.10 TE-MS multiplexor protocol.
>>>
>>> The main problem we have is co-ordinating all the different comms 
>>> channels of OVMS onto a single serial link to the modem. Incoming 
>>> SMSes, status indicators, outgoing GPRS TCP data, etc, all run over 
>>> each other.
>>>
>>> The CMUX protocol appears to allow us to have 3 (or 4?) virtual 
>>> async links, over a single physical async port. We could use one for 
>>> SMS, another for TCP, and a third for GPS polling. The 3G modules 
>>> even allow the GPS NMEA data to be streamed back over one of the 
>>> virtual ports (but I don't think that is available in the SIM908 
>>> module we use - hard to tell as the CMUX documents focus on SIM900 
>>> not SIM908 [the SIM908 embeds SIM900 and adds GPS]).
>>>
>>> It would mean some major restructuring of the comms layer (NET), and 
>>> we would have to build a small library to encode/decode GSM 7.10, 
>>> but may be a much more stable approach in the long-term.
>>>
>>> SIMCOM seem to implement a subset of the full GSM 7.10 
>>> functionality. Here's the manual on it for SIM900:
>>>
>>>     http://www.mt-system.ru/sites/default/files/sim900_multiplexer_user_manual_application_note_v1.3.pdf
>>>
>>>     <Mail Attachment.png>
>>>
>>>
>>> Regards, Mark.
>>>
>>> On 29 Apr, 2013, at 11:48 PM, Michael Balzer wrote:
>>>
>>>> Am 28.04.2013 17:06, schrieb Michael Balzer:
>>>>> I just checked in my fix for the USSD reply handler. My tests with 
>>>>> streaming enabled did not produce any more crashes, so I think 
>>>>> that was the problem.
>>>>
>>>> I was wrong, it's still crashing. I have to dig deeper it seems.
>>>>
>>>> Has anyone observed crashes with streaming mode on other vehicles, 
>>>> or is this a Twizy specific bug now?
>>>>
>>>> Thanks,
>>>> Michael
>>>>
>>>> -- 
>>>> Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
>>>> Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
>>>> <dexter.vcf>_______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>
>>>
>>>
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.teslaclub.hk
>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>
>> -- 
>> Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
>> Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
>> <dexter.vcf>_______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>
>
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.teslaclub.hk
> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev

-- 
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20130507/a5f63182/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dexter.vcf
Type: text/x-vcard
Size: 206 bytes
Desc: not available
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20130507/a5f63182/attachment-0002.vcf>


More information about the OvmsDev mailing list