[Ovmsdev] Crashes with streaming mode
Mark Webb-Johnson
mark at webb-johnson.net
Wed May 8 22:08:43 HKT 2013
Michael,
I reviewed the patch. It seems fine.
When I originally wrote the net.{c,h} framework, I thought about doing some sort of send-expect (chat) control. It was just hard to implement with limited ram, while keeping the state based system. A lot of the lower state transitions are event based (most of the initialisation works that way), but once we get into the main online state it stays there. Is it possible to introduce some more READY states to handle this? Maybe even a transition system based on a modem result (i.e.; wait until you get string "OK" then go to this state, or time-out after N seconds)?
Last point is that some of the delays are because there is no response for some modem commands, and if you send too fast that causes a loss of data.
Regards, Mark.
On 8 May, 2013, at 2:52 AM, Michael Balzer <dexter at expeedo.de> wrote:
> Mark, List,
>
> I just checked in some changes that have improved my GPS streaming stability on all my latest test drives to 100%.
>
> https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92
>
> I still don't know why or how the modem rx buffer full condition could lead to real crashes, but eliminating those also eliminated the crashes.
>
> A major part in the buffer filling up was played by the massive delays needed on net_msg_start() to make sure the modem is ready without polling for it's status prompts. That indirectly also played a role in the other problem source, being my network functions assuming the modem is basically "ready" if net_msg_sendpending is 0.
>
> But it is not ready if it still processes some command -- like the GPS NMEA request. I now insert another hard coded delay to make sure the modem is ready.
>
> These hard delays don't really feel good and they do add up. Any net msg now already needs at least 1.5 seconds just for the net_msg_start() part, so this also disturbs the ticker timing.
>
> I think we could eliminate most hard delays by introducing some sort of modem status following. A simple inline modem reply waiting loop might already solve most problems. This could basically be a loop of net_poll() calls with exit condition on checking the buffer for "OK" / "ERROR" / other prompts -- maybe with an additional timeout.
>
> Problem could be this net_poll() loop would run outside the main loop, so it would omit all "idle" and "ticker" callbacks until the response comes in (or timeout occurs).
>
> The benefit would be a much tighter and faster coupling of the modem, and being able to react according to the modem status responses for certain commands.
>
> Have there been any thoughts on this before? Shall I follow this path a bit more or isn't it worth the effort?
>
> Regards,
> Michael
>
>
>
> Am 01.05.2013 15:05, schrieb Mark Webb-Johnson:
>> Michael,
>>
>> No hardware flow control in OVMS circuit. But, we are only 9600baud and interrupt-driven - I think it should be ok.
>>
>> My concern is RAM - say we have two channels open, we're going to need 2 more line buffers.
>>
>> Here is the latest 'map' file:
>>
>> Section Type Address Location Size(Bytes)
>> --------- --------- --------- --------- ---------
>> TX_CRYPTO udata 0x000100 data 0x000100
>> RX_CRYPTO udata 0x000200 data 0x000100
>> PM_CRYPTO udata 0x000300 data 0x000100
>> .stack udata 0x000c00 data 0x000100
>> .udata_crypt_hmac.o udata 0x000400 data 0x0000d8
>> NETMSG_SP udata 0x000500 data 0x0000c8
>> NETBUF_SP udata 0x000600 data 0x0000c8
>> NETBUF udata 0x000700 data 0x0000c8
>> .udata_UARTIntC.o udata 0x000800 data 0x0000c7
>> LOGGING udata 0x000060 data 0x000088
>> SFR_UNBANKED1 udata 0x000f80 data 0x000080
>> .idata_ovms.o idata 0x000900 data 0x000070
>> vehicle_overlay_data udata 0x000970 data 0x00006e
>> .idata_net_sms.o idata 0x000a00 data 0x000063
>> SFR_BANKED12 udata 0x000f00 data 0x000060
>> SFR_BANKED11 udata 0x000e20 data 0x000060
>> .idata_net_msg.o idata 0x000a63 data 0x000045
>> .udata_crypt_md5.o udata 0x000aa8 data 0x000040
>> .idata_crypt_md5.o idata 0x000b00 data 0x000040
>> .idata_net.o idata 0x0005c8 data 0x000035
>> .udata_ovms.o udata 0x0006c8 data 0x000030
>> .idata_vehicle.o idata 0x0007c8 data 0x00002b
>> .udata_net_msg.o udata 0x0004d8 data 0x000026
>> .udata_params.o udata 0x0009de data 0x000020
>> .idata_diag.o idata 0x0008c7 data 0x00001b
>> SFR_UNBANKED0 udata 0x000f60 data 0x000018
>> .idata_vehicle_twizy.o idata 0x0000e8 data 0x000014
>> MATH_DATA udata 0x000020 data 0x000014
>> .udata_vehicle.o udata 0x000ae8 data 0x000012
>> SFR_BANKED2 udata 0x000d80 data 0x00000c
>> SFR_BANKED1 udata 0x000d70 data 0x00000c
>> SFR_BANKED0 udata 0x000d60 data 0x00000c
>> .udata_c018i.o udata 0x0007f3 data 0x00000a
>> .udata_crypt_base64.o udata 0x0006f8 data 0x000008
>> SFR_BANKED6 udata 0x000de0 data 0x000008
>> .udata_net_sms.o udata 0x0008e2 data 0x000007
>> .idata_led.o idata 0x000afa data 0x000005
>> SFR_BANKED7 udata 0x000df0 data 0x000004
>> SFR_BANKED3 udata 0x000d90 data 0x000004
>> .udata_net.o udata 0x0005fd data 0x000003
>> .idata_stokpr.o idata 0x0009fe data 0x000002
>> SFR_BANKED4 udata 0x000dd4 data 0x000002
>> SEED_DATA idata 0x0004fe data 0x000002
>> .idata_logging.o idata 0x0007fd data 0x000001
>> SFR_BANKED10 udata 0x000dfc data 0x000001
>> .udata_led.o udata 0x000aff data 0x000001
>> SFR_BANKED9 udata 0x000dfa data 0x000001
>> SFR_BANKED8 udata 0x000df8 data 0x000001
>> SFR_BANKED5 udata 0x000dd8 data 0x000001
>> DELAYDAT1 udata 0x000034 data 0x000001
>>
>> Stack definitely seems separately accounted for. The TX_CRYPTO and RX_CRYPTO are the rc4 contexts. The PM_CRYPTO is for Paranoid Mode (a lot of RAM for that, considering how few use it).
>>
>> The crypt_hmac needs 64 bytes for iPad, 64 bytes for oPad, then 70 bytes for a MD5_CTX. Looking at the code, a lot could be saved. It looks like the iPad and oPad could be merged (slightly slower code, but saves a lot). Then, the pad and context could be on the stack (134 bytes of 256 bytes) - scary but a big win.
>>
>> The crypt_md5 PADDING (64 bytes) seems to be able to be moved to rom with little impact.
>>
>> From there, the returns seem to diminish.
>>
>> Regards, Mark.
>>
>> On 1 May, 2013, at 6:18 PM, Michael Balzer <dexter at expeedo.de> wrote:
>>
>>> Mark,
>>>
>>> I also see the current modem communication layer as a problem source. The read buffer only takes 128 bytes, which will not be sufficient for many situations -- e.g. a GPS response now needs already two NMEA sentences adding up to about 108 bytes -- come in another modem answer before the buffer gets handled, and bang we go...
>>>
>>> The multiplex mode is quite interesting, didn't know this before -- although a really fully asynchronous modem comm handling would not need that, I think. But it would need more buffer RAM...
>>>
>>> Hm... the SIM908 manual says: "In Multiplex mode, it is recommended to use the hardware flow control."
>>>
>>> As far as I understand the circuit schemes the current hardware does not connect RTS/CTS (pins 66+67) -- right?
>>>
>>> Regards,
>>> Michael
>>>
>>>
>>> Am 30.04.2013 03:28, schrieb Mark Webb-Johnson:
>>>> Michael,
>>>>
>>>> Looking at the design for 3G modules, I noticed something recommended for those modules that seems to be supported for the SIM900 base we use (both SIM900 OVMS v1 and SIM908 OVMS v2). CMUX mode.
>>>>
>>>> http://www.m2m-platforms.com/seminars/2007seminar_material/Telit_CMUX_2_M2M_Platforms_seminar_2007.pdf
>>>>
>>>> This is GSM 7.10 TE-MS multiplexor protocol.
>>>>
>>>> The main problem we have is co-ordinating all the different comms channels of OVMS onto a single serial link to the modem. Incoming SMSes, status indicators, outgoing GPRS TCP data, etc, all run over each other.
>>>>
>>>> The CMUX protocol appears to allow us to have 3 (or 4?) virtual async links, over a single physical async port. We could use one for SMS, another for TCP, and a third for GPS polling. The 3G modules even allow the GPS NMEA data to be streamed back over one of the virtual ports (but I don't think that is available in the SIM908 module we use - hard to tell as the CMUX documents focus on SIM900 not SIM908 [the SIM908 embeds SIM900 and adds GPS]).
>>>>
>>>> It would mean some major restructuring of the comms layer (NET), and we would have to build a small library to encode/decode GSM 7.10, but may be a much more stable approach in the long-term.
>>>>
>>>> SIMCOM seem to implement a subset of the full GSM 7.10 functionality. Here's the manual on it for SIM900:
>>>>
>>>> http://www.mt-system.ru/sites/default/files/sim900_multiplexer_user_manual_application_note_v1.3.pdf
>>>>
>>>> <Mail Attachment.png>
>>>>
>>>> Regards, Mark.
>>>>
>>>> On 29 Apr, 2013, at 11:48 PM, Michael Balzer wrote:
>>>>
>>>>> Am 28.04.2013 17:06, schrieb Michael Balzer:
>>>>>> I just checked in my fix for the USSD reply handler. My tests with streaming enabled did not produce any more crashes, so I think that was the problem.
>>>>>
>>>>> I was wrong, it's still crashing. I have to dig deeper it seems.
>>>>>
>>>>> Has anyone observed crashes with streaming mode on other vehicles, or is this a Twizy specific bug now?
>>>>>
>>>>> Thanks,
>>>>> Michael
>>>>>
>>>>> --
>>>>> Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
>>>>> Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
>>>>> <dexter.vcf>_______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.teslaclub.hk
>>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.teslaclub.hk
>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>>
>>> --
>>> Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
>>> Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
>>> <dexter.vcf>_______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.teslaclub.hk
>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>>
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.teslaclub.hk
>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
>
> --
> Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
> Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
> <dexter.vcf>_______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.teslaclub.hk
> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.teslaclub.hk/pipermail/ovmsdev/attachments/20130508/6ff1ab39/attachment-0001.html>
More information about the OvmsDev
mailing list