This is bizarre:

Modifying the standard hello_world template App, to add:

#include <sys/socket.h>
int sock = socket(AF_INET, SOCK_STREAM, 0);
printf("Allocated socket #%d\n",sock);
if (sock >= 0) close(sock);

I get this output when compiling against ESP IDF v3.0 / v3.1:

Hello world!
This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 1, 16MB external flash
Allocated socket #4096

When compiling against ESP-IDF v2.1, I get:

Hello world!
This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 1, 16MB external flash
Allocated socket #0

This messes up the select() call, using:

FD_ZERO(&fds);
FD_SET(sock,&fds);

This commit in lwip component:

commit 90bf40587ed6335f6ea3d343c3b3a43cac3a7c53
Merge: 6e7dd596 539262b5
Author: Angus Gratton <angus@espressif.com>
Date:   Tue Oct 17 14:17:09 2017 +0800

    Merge branch 'feature/sockets_files_shared_fd_space' into 'master'

    lwip & vfs: POSIX I/O functions operate on sockets and files (first stage, no select() yet)

    See merge request !1352

It looks like Espressif have decided to change the way file descriptors work in IDF 3.x. Without telling anyone they’ve added an offset to indicate which component ‘owns’ the file descriptor. In their lwip code they seem to have redefined FD_SET, etc, to cope with that offset. But, the standard sys/select.h hasn’t changed so using that generates invalid file descriptor sets. They seem to have done this for their VFS code (so tcp/ip FDs can be mounted on VFS). But, they could simply have extended the socket structure to indicate which VFS component owned it. Why screw with such a fundamental thing as a file descriptor?

The offsets seem to depend on vfs component instantiation order; which means that they are non-deterministic. I get 4096 in my test app, and 8192 in ovms v3 code.

I’ll try to find a workaround solution. It seems to work for mongoose, so perhaps try to do it the way they do...

Regards, Mark.

On 16 Jan 2018, at 9:19 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:

I’m looking into the ‘server response is incomplete’ message now. Seems something broken in the socket layer / memory corruption.

Regards, Mark.

On 16 Jan 2018, at 5:41 AM, Michael Balzer <dexter@expeedo.de> wrote:

More issues:

a) changing log levels only works without specifying a tag

b) getting frequent logs like these:

I (8324105) ovms-server-v2: Server response is incomplete (0 bytes)
E (8324105) ovms-server-v2: Status: Error: Server response is incomplete

…resulting in a server reconnect.

V (8415335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,,
V (8415335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53.....
V (8415335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31)
V (8415345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3)
V (8415355) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53
V (8415355) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,,
V (8415355) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9       | ,,,,,,N*53.... 
V (8415355) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31)
V (8415355) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3)
V (8415355) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53
V (8416335) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,,
V (8416335) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53.....
V (8416335) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31)
V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3)
V (8416345) gsm-nmea: IncomingLine: $GNGNS,,,,,,NN,,,,,,*53
V (8416345) simcom: rx: 05 ff 33 24 47 50 52 4d 43 2c 2c 56 2c 2c 2c 2c | ..3$GPRMC,,V,,,,
V (8416345) simcom: rx: 2c 2c 2c 2c 2c 2c 4e 2a 35 33 0d 0a 84 f9       | ,,,,,,N*53.... 
V (8416345) gsm-mux: ProcessFrame(CHAN=1, ADDR=05, CTRL=ff, FCS=84, LEN=31)
V (8416345) gsm-mux: ChanProcessFrame(CHAN=1, ADDR=05, CTRL=ff, LEN=28, IFP=3)
V (8416345) gsm-nmea: IncomingLine: $GPRMC,,V,,,,,,,,,,N*53
I (8416515) ovms-server-v2: Server response is incomplete (0 bytes)
V (8416515) gsm-ppp: tx: 7e 21 45 00 00 28 00 6c 00 00 ff 06 e5 3c 0a aa | ~!E..(.l.....<..
V (8416515) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d6 3f f5 | ....K..@....M.?.
V (8416515) gsm-ppp: tx: f4 17 50 11 16 2a 38 8b 00 00 b9 4c 7e          | ..P..*8....L~  
V (8416515) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6c 00 00 ff 06 | ...[~!E..(.l....
V (8416515) simcom: tx: e5 3c 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .<......K..@....
V (8416515) simcom: tx: 4d d6 3f f5 f4 17 50 11 16 2a 38 8b 00 00 b9 4c | M.?...P..*8....L
V (8416515) simcom: tx: 7e 45 f9                                        | ~E.            
E (8416525) ovms-server-v2: Status: Error: Server response is incomplete
V (8417315) simcom: rx: f9 09 ff 5b 7e 21 45 00 00 28 b7 32 40 00 2d 06 | ...[~!E..(.2@.-.
V (8417315) simcom: rx: c0 76 bc 8a 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 | .v..K........@?.
V (8417315) simcom: rx: f4 17 00 00 4d d7 50 11 39 08 15 ac 00 00 f2 89 | ....M.P.9.......
V (8417325) simcom: rx: 7e 45 f9                                        | ~E.            
V (8417325) gsm-mux: ProcessFrame(CHAN=2, ADDR=09, CTRL=ff, FCS=45, LEN=51)
V (8417325) gsm-mux: ChanProcessFrame(CHAN=2, ADDR=09, CTRL=ff, LEN=48, IFP=3)
V (8417325) gsm-ppp: rx: 7e 21 45 00 00 28 b7 32 40 00 2d 06 c0 76 bc 8a | ~!E..(.2@.-..v..
V (8417325) gsm-ppp: rx: 4b e5 0a aa c3 0d 1a d3 ee 40 3f f5 f4 17 00 00 | K........@?.....
V (8417335) gsm-ppp: rx: 4d d7 50 11 39 08 15 ac 00 00 f2 89 7e          | M.P.9.......~  
V (8417335) gsm-ppp: tx: 7e 21 45 00 00 28 00 6d 00 00 ff 06 e5 3b 0a aa | ~!E..(.m.....;..
V (8417335) gsm-ppp: tx: c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 4d d7 3f f5 | ....K..@....M.?.
V (8417335) gsm-ppp: tx: f4 18 50 10 16 29 38 8b 00 00 a9 af 7e          | ..P..)8.....~  
V (8417335) simcom: tx: f9 09 ff 5b 7e 21 45 00 00 28 00 6d 00 00 ff 06 | ...[~!E..(.m....
V (8417335) simcom: tx: e5 3b 0a aa c3 0d bc 8a 4b e5 ee 40 1a d3 00 00 | .;......K..@....
V (8417335) simcom: tx: 4d d7 3f f5 f4 18 50 10 16 29 38 8b 00 00 a9 af | M.?...P..)8.....
V (8417335) simcom: tx: 7e 45 f9                                        | ~E.            
V (8417345) simcom: rx: f9 05 ff 33 24 47 4e 47 4e 53 2c 2c 2c 2c 2c 2c | ...3$GNGNS,,,,,,
V (8417345) simcom: rx: 4e 4e 2c 2c 2c 2c 2c 2c 2a 35 33 0d 0a 84 f9 f9 | NN,,,,,,*53.....



Am 15.01.2018 um 19:21 schrieb Michael Balzer:
Transition went fine. "make flash" didn't even erase "/store", I had expected that to happen, good to know.

And…

OVMS > sd status
SD CARD is inserted
Name: SL32G
Type: SDHC/SDXC
Speed: default speed
Size: 30436MB
CSD: ver=1, sector_size=512, capacity=62333952 read_bl_len=9
SCR: sd_spec=2, bus_width=5

→ me too (the minion) :)


BUT… after reconfiguration to the same components as before, I'm now missing about 30-40 KB of free RAM.

After loading just the Twizy module & the v2 server, I'm now again down to…

OVMS > module memory
Free 8-bit 2632/286352, 32-bit 17972/45144, SPIRAM 0/0

…so it crashes as soon as it needs to actually do anything, like sending a notification.

This is without any of the new CAN logging stuff yet, that's stashed away for the transition.

I'll now check the config again for any more options to save RAM…

Regards,
Michael


Am 15.01.2018 um 06:48 schrieb Stephen Casner:
In order for the the "module memory" and "module tasks" commands to
work, the configuration needs to be changed from the sdk.default.hw30
that Mark just provided. CONFIG_FREERTOS_USE_TRACE_FACILITY needs to
be enabled, CONFIG_HEAP_POISONING must be enabled (LIGHT or
COMPREHENSIVE), and CONFIG_HEAP_TASK_TRACKING must be enabled.  I have
attached a diff for the sdkconfig.  That diff also includes a section
for enabling telnet, which some of you may be using.

Please also note that in this revised implementation of "module
memory" the block address is now the address that is returned from
malloc, not the address of the header that precedes it, and the length
of the block is now exclusive of the debugging overhead.

                                                        -- Steve

On Mon, 15 Jan 2018, Mark Webb-Johnson wrote:


All committed and pushed now.

Developers MUST now update their ESP IDF and XTENSA build tools to use this new build. They should also compare sdkconfig and sdkconfig.defaults.hw30 to verify differences. I also suggest you do a ‘make flash’ (rather than ‘make app-flash’) at least once, to update the bootloader to the latest (as bootloaders before v2.1 are no longer supported).

Check
https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain <https://esp-idf.readthedocs.io/en/latest/get-started/index.html#setup-toolchain>
to see the version of XTENSA toolchain required for latest Espressif IDF ‘master’ builds. As of this writing, it is 1.22.0-80-g6c4433a-5.2.0.

I hope this goes smoothly for you.

Regards, Mark.

commit dfcc658557fc480bcccb09b23bc24399bb6e7497 (HEAD -> master, origin/master, origin/HEAD)
Author: Mark Webb-Johnson <mark@webb-johnson.net>
Date:   Mon Jan 15 07:56:45 2018 +0800

    Provide a sdkconfig.default.hw31 for OVMS v3.1 hardware

commit 003592d553c881f735fb228b08836a13c306cec3
Merge: e58c5b6 b28db5e
Author: Mark Webb-Johnson <mark@webb-johnson.net>
Date:   Mon Jan 15 07:49:13 2018 +0800

    Merge branch 'for-master'. This requires ESP-IDF v3.0 support:

    OVMS developers should now:

    1] Pull the latest OpenVehicles IDF, and checkout MASTER branch.

    2] Update XTENSA tools to match version required by Espressif for MASTER branch.

    3] For ovms hardware v3.0, a sdkconfig default file sdkconfig.default.hw30 has
       been provided. That can be merged/copied to sdkconfig as appropriate.

    4] For ovms hardware v3.1, a sdkconfig default file sdkconfig.default.hw31 will
       been provided in the next update.

    5] It is recommended that developers perform at least one full 'make flash'
       with this version, to update the bootloader to latest. Note that the
       sdkconfig.default.* files are now set to require bootloader at least v2.1
       (or later), and bootloader v2.0 is no longer supported.

    Enjoy.



On 15 Jan 2018, at 7:25 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:

Merge is ok, but triple checking a couple of things. Should be committed and pushed within the next half hour.

Regards, Mark


On 15 Jan 2018, at 1:09 AM, Stephen Casner <casner@acm.org> wrote:

Mark,

It looks like you were not able to do the sdkconfig.default and merge
as you planned?

                                                      -- Steve


On Fri, 12 Jan 2018, Stephen Casner wrote:


On Sat, 13 Jan 2018, Mark Webb-Johnson wrote:

@Steve can you update our master clone to latest from Espressif and
make sure your stuff is still ok?

I have just now done this.  I had already done a rebase a few days ago
before committing the improved version of the OS changes so I could
issue a pull request.  There were a few commits since then, which I
have now merged.  My code still runs correctly.


I guess steps for developers will be:

1) pull the openvehicles IDF and switch to master branch. Sub module update.

2) download and install xtensa build chain to match.

3) update OVMS master, make clean, check menu config, then build and play.

Sounds right.


I can get my part done by Sunday 14th night (HKT), so suggest to do
the Merge of the OVMS firmware master branch then.

I'll be sleeping then, so you can go ahead and do the merge.

                                                      -- Steve

_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev

_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev

>


_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev


-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26



_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev


-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.teslaclub.hk
http://lists.teslaclub.hk/mailman/listinfo/ovmsdev