Mark, List,

I just checked in some changes that have improved my GPS streaming stability on all my latest test drives to 100%.

https://github.com/markwj/Open-Vehicle-Monitoring-System/commit/df459b6b5b07c3b3e1c85b1bd6667a60b5d44a92

I still don't know why or how the modem rx buffer full condition could lead to real crashes, but eliminating those also eliminated the crashes.

A major part in the buffer filling up was played by the massive delays needed on net_msg_start() to make sure the modem is ready without polling for it's status prompts. That indirectly also played a role in the other problem source, being my network functions assuming the modem is basically "ready" if net_msg_sendpending is 0.

But it is not ready if it still processes some command -- like the GPS NMEA request. I now insert another hard coded delay to make sure the modem is ready.

These hard delays don't really feel good and they do add up. Any net msg now already needs at least 1.5 seconds just for the net_msg_start() part, so this also disturbs the ticker timing.

I think we could eliminate most hard delays by introducing some sort of modem status following. A simple inline modem reply waiting loop might already solve most problems. This could basically be a loop of net_poll() calls with exit condition on checking the buffer for "OK" / "ERROR" / other prompts -- maybe with an additional timeout.

Problem could be this net_poll() loop would run outside the main loop, so it would omit all "idle" and "ticker" callbacks until the response comes in (or timeout occurs).

The benefit would be a much tighter and faster coupling of the modem, and being able to react according to the modem status responses for certain commands.

Have there been any thoughts on this before? Shall I follow this path a bit more or isn't it worth the effort?

Regards,
Michael



Am 01.05.2013 15:05, schrieb Mark Webb-Johnson:
Michael,

No hardware flow control in OVMS circuit. But, we are only 9600baud and interrupt-driven - I think it should be ok.

My concern is RAM - say we have two channels open, we're going to need 2 more line buffers.

Here is the latest 'map' file:

                  Section       Type    Address   Location Size(Bytes)
                ---------  ---------  ---------  ---------  ---------
                TX_CRYPTO      udata   0x000100       data   0x000100
                RX_CRYPTO      udata   0x000200       data   0x000100
                PM_CRYPTO      udata   0x000300       data   0x000100
                   .stack      udata   0x000c00       data   0x000100
      .udata_crypt_hmac.o      udata   0x000400       data   0x0000d8
                NETMSG_SP      udata   0x000500       data   0x0000c8
                NETBUF_SP      udata   0x000600       data   0x0000c8
                   NETBUF      udata   0x000700       data   0x0000c8
        .udata_UARTIntC.o      udata   0x000800       data   0x0000c7
                  LOGGING      udata   0x000060       data   0x000088
            SFR_UNBANKED1      udata   0x000f80       data   0x000080
            .idata_ovms.o      idata   0x000900       data   0x000070
     vehicle_overlay_data      udata   0x000970       data   0x00006e
         .idata_net_sms.o      idata   0x000a00       data   0x000063
             SFR_BANKED12      udata   0x000f00       data   0x000060
             SFR_BANKED11      udata   0x000e20       data   0x000060
         .idata_net_msg.o      idata   0x000a63       data   0x000045
       .udata_crypt_md5.o      udata   0x000aa8       data   0x000040
       .idata_crypt_md5.o      idata   0x000b00       data   0x000040
             .idata_net.o      idata   0x0005c8       data   0x000035
            .udata_ovms.o      udata   0x0006c8       data   0x000030
         .idata_vehicle.o      idata   0x0007c8       data   0x00002b
         .udata_net_msg.o      udata   0x0004d8       data   0x000026
          .udata_params.o      udata   0x0009de       data   0x000020
            .idata_diag.o      idata   0x0008c7       data   0x00001b
            SFR_UNBANKED0      udata   0x000f60       data   0x000018
   .idata_vehicle_twizy.o      idata   0x0000e8       data   0x000014
                MATH_DATA      udata   0x000020       data   0x000014
         .udata_vehicle.o      udata   0x000ae8       data   0x000012
              SFR_BANKED2      udata   0x000d80       data   0x00000c
              SFR_BANKED1      udata   0x000d70       data   0x00000c
              SFR_BANKED0      udata   0x000d60       data   0x00000c
           .udata_c018i.o      udata   0x0007f3       data   0x00000a
    .udata_crypt_base64.o      udata   0x0006f8       data   0x000008
              SFR_BANKED6      udata   0x000de0       data   0x000008
         .udata_net_sms.o      udata   0x0008e2       data   0x000007
             .idata_led.o      idata   0x000afa       data   0x000005
              SFR_BANKED7      udata   0x000df0       data   0x000004
              SFR_BANKED3      udata   0x000d90       data   0x000004
             .udata_net.o      udata   0x0005fd       data   0x000003
          .idata_stokpr.o      idata   0x0009fe       data   0x000002
              SFR_BANKED4      udata   0x000dd4       data   0x000002
                SEED_DATA      idata   0x0004fe       data   0x000002
         .idata_logging.o      idata   0x0007fd       data   0x000001
             SFR_BANKED10      udata   0x000dfc       data   0x000001
             .udata_led.o      udata   0x000aff       data   0x000001
              SFR_BANKED9      udata   0x000dfa       data   0x000001
              SFR_BANKED8      udata   0x000df8       data   0x000001
              SFR_BANKED5      udata   0x000dd8       data   0x000001
                DELAYDAT1      udata   0x000034       data   0x000001

Stack definitely seems separately accounted for. The TX_CRYPTO and RX_CRYPTO are the rc4 contexts. The PM_CRYPTO is for Paranoid Mode (a lot of RAM for that, considering how few use it).

The crypt_hmac needs 64 bytes for iPad, 64 bytes for oPad, then 70 bytes for a MD5_CTX. Looking at the code, a lot could be saved. It looks like the iPad and oPad could be merged (slightly slower code, but saves a lot). Then, the pad and context could be on the stack (134 bytes of 256 bytes) - scary but a big win.

The crypt_md5 PADDING (64 bytes) seems to be able to be moved to rom with little impact.

From there, the returns seem to diminish.

Regards, Mark.

On 1 May, 2013, at 6:18 PM, Michael Balzer <dexter@expeedo.de> wrote:

Mark,

I also see the current modem communication layer as a problem source. The read buffer only takes 128 bytes, which will not be sufficient for many situations -- e.g. a GPS response now needs already two NMEA sentences adding up to about 108 bytes -- come in another modem answer before the buffer gets handled, and bang we go...

The multiplex mode is quite interesting, didn't know this before -- although a really fully asynchronous modem comm handling would not need that, I think. But it would need more buffer RAM...

Hm... the SIM908 manual says: "In Multiplex mode, it is recommended to use the hardware flow control."

As far as I understand the circuit schemes the current hardware does not connect RTS/CTS (pins 66+67) -- right?

Regards,
Michael


Am 30.04.2013 03:28, schrieb Mark Webb-Johnson:
Michael,

Looking at the design for 3G modules, I noticed something recommended for those modules that seems to be supported for the SIM900 base we use (both SIM900 OVMS v1 and SIM908 OVMS v2). CMUX mode.


This is GSM 7.10 TE-MS multiplexor protocol.

The main problem we have is co-ordinating all the different comms channels of OVMS onto a single serial link to the modem. Incoming SMSes, status indicators, outgoing GPRS TCP data, etc, all run over each other.

The CMUX protocol appears to allow us to have 3 (or 4?) virtual async links, over a single physical async port. We could use one for SMS, another for TCP, and a third for GPS polling. The 3G modules even allow the GPS NMEA data to be streamed back over one of the virtual ports (but I don't think that is available in the SIM908 module we use - hard to tell as the CMUX documents focus on SIM900 not SIM908 [the SIM908 embeds SIM900 and adds GPS]).

It would mean some major restructuring of the comms layer (NET), and we would have to build a small library to encode/decode GSM 7.10, but may be a much more stable approach in the long-term.

SIMCOM seem to implement a subset of the full GSM 7.10 functionality. Here's the manual on it for SIM900:


<Mail Attachment.png>

Regards, Mark.

On 29 Apr, 2013, at 11:48 PM, Michael Balzer wrote:

Am 28.04.2013 17:06, schrieb Michael Balzer:
I just checked in my fix for the USSD reply handler. My tests with streaming enabled did not produce any more crashes, so I think that was the problem.

I was wrong, it's still crashing. I have to dig deeper it seems.

Has anyone observed crashes with streaming mode on other vehicles, or is this a Twizy specific bug now?

Thanks,
Michael

-- 
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
<dexter.vcf>_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev



_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev


-- 
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26
<dexter.vcf>_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev



_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev


-- 
Michael Balzer * Paradestr. 8 * D-42107 Wuppertal
Fon 0202 / 272 2201 * Handy 0176 / 206 989 26