<div dir="auto">I may try this on my spare board as a start! (The Aux 12v switch didn't work... But otherwise it's mostly OK)<div dir="auto"><br></div><div dir="auto">Michael. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 26 Jan 2023, 4:41 pm Mark Webb-Johnson, <<a href="mailto:mark@webb-johnson.net">mark@webb-johnson.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-break:after-white-space">Progress looks good on this. Thanks for stepping up to try to tackle it.<div><br></div><div>Pain points for me with our current approach are:</div><div><br></div><div><ol><li>RAM usage (in particular IRAM)</li><li>IDF bugs that may be improved in newer versions</li><li>Support for newer hardware (including ESP32-S3)</li></ol><div><br></div><div>#3 would give us a welcome 12 more GPIOs with minimal hardware changes, but possibly worse RAM situation (520KB->512KB, see #1).</div><div><br></div><div>My preference would be v5 IDF (for the newer hardware support).</div><div><br></div><div>I think it would be fine to have this as a new branch. Not particularly important to make it work in the master one.</div><div><br></div><div>Regards, Mark.</div><div><br><blockquote type="cite"><div>On 26 Jan 2023, at 8:56 AM, Ludovic LANGE <<a href="mailto:ll-ovmsdev@lange.nom.fr" target="_blank" rel="noreferrer">ll-ovmsdev@lange.nom.fr</a>> wrote:</div><br><div>
<div>
<div>Hi,</div>
<div><br>
</div>
<div>A quick note to share with you the
progress on the ESP-IDFv4 + cmake endeavour:</div>
<div>
<ul>
<li>The branch
<a href="https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network" target="_blank" rel="noreferrer">https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network</a>
is now ready for greater review. Most of the warnings are
gone, and I'll need some help for the last ones.<br>
<br>
</li>
<li>I created a draft PR
<a href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/806" target="_blank" rel="noreferrer">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/pull/806</a>
. Not that I want to merge it as-is nor soon, but it may help
the review process to have some central discussion point.<br>
<br>
</li>
<li>I'll now switch to ESP-IDFv5 porting - on a new branch
(based on the v4) :
<a href="https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v5" target="_blank" rel="noreferrer">https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v5</a>
(empty at the moment).<br>
</li>
</ul>
</div>
<div><br>
</div>
<div>Regards,<br>
</div>
<div><br>
</div>
<div>Le 23/01/2023 à 22:19, Ludovic LANGE a
écrit :<br>
</div>
<blockquote type="cite">
<div>Michael,</div>
<div><br>
</div>
<div>Thanks for your help. FYI I've just
pushed the last touches to the vehicle components which are now
all converted to cmake and "should work". Still compiles under
3.3.x.</div>
<br>
<div>A new things that need to be checked
: in
<a href="https://github.com/llange/Open-Vehicle-Monitoring-System-3/commit/f31117a48d4e293b2be01e8b48c1d580d0af834e" target="_blank" rel="noreferrer">https://github.com/llange/Open-Vehicle-Monitoring-System-3/commit/f31117a48d4e293b2be01e8b48c1d580d0af834e</a>
i had to replace `INT` with another type. This time I choose
`int32_t` but I'm not really sure because I could not find a way
to see where and how INT was defined. I could only verify (by
compilation tricks) that sizeof(INT) == sizeof(int32_t).<br>
</div>
<div><br>
</div>
<div>Regarding cmake, I understand your
concern with dependencies, and I did have the same reaction.
There certainly is a way - I did not spend too much time on it.
We will however have some "bugs" in the current state of things
when disabling components in menuconfig where a dependency was
implied and is now exposed. I'll let the cmake experts guide us
:-)<br>
</div>
<div><br>
</div>
<div>
<div>I'll now work on the compilation
warnings.</div>
<div><br>
</div>
<div>Btw, another thing that I need to
mention is that from 3.3 to 5.x, many constants (used in
sdkconfig) have been renamed etc... So our sdkconfig is
upgraded by the build toolchain when we compile with 4.x or
5.x. I recommend doing a backup if it has been customized - in
case of.<br>
</div>
<div><br>
</div>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Ludovic</div>
<div><br>
</div>
<div><br>
</div>
<div>Le 23/01/2023 à 21:11, Michael Balzer
a écrit :<br>
</div>
<blockquote type="cite">
Ludovic,<br>
<br>
really great & appreciated you're going ahead on this. I
second skipping idf 4 and going directly for version 5. I'll try
to get my idf 5 build environment up ASAP.<br>
<br>
I need to get into cmake myself as well. Needing to hard code
dependencies doesn't sound good though, I thought cmake is
supposed to be smart about this.<br>
<br>
I'll have a look at porting the custom crash handler to idf 5.
There were some other additions that haven't been accepted by
Espressif, IIRC. I'll see if I can check them as well.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div>Am 23.01.23 um 11:04 schrieb
Ludovic LANGE:<br>
</div>
<blockquote type="cite">
<p>Hi list,</p><p>I am interested in upgrading our codebase to a latest
ESP-IDF version (preferentially v5 which has just happened
end of 2022 - Happy New Year ! :-)) - mostly to be able to
use some components for which a backport seems difficult
(for example : <a href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752" target="_blank" rel="noreferrer">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752</a>
), and as a matter of balance between stability vs latest
bugfixes, security fixes, new features etc...</p><p>In the past it seems that some endeavours already set the
basis to make this happen (<a href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263" target="_blank" rel="noreferrer">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263</a>,
<a href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360" target="_blank" rel="noreferrer">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360</a>
).</p><p>I'm now trying to revive this effort, and after some work
managed to have:</p>
<ul>
<li>OVMS codebase building under v4.x using Makefile</li>
<li>OVMS codebase building under v4.x using cmake / idf.py
(not 100% finished, multiple vehicle_* components are
missing)</li>
</ul><p>(<i><b>WARNING</b>: some important features of OVMS have
been deactivated for the moment, see end of post)</i><br>
</p><p>The produced binaries seem to "work on my <strike>machine</strike>
board", as they end with a "OVMS#" prompt and a few (very)
basic tests look OK (to be honest, unfortunately no real
tests have been done except to check that the binary does
not crash during boot and a few seconds of run, checking
that WiFi / web Interface is OK, sdcard is OK - but very
basic)<br>
</p><p><br>
</p><p>You'll find the changes here : <a href="https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network" target="_blank" rel="noreferrer">https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network</a><br>
(In the README.md, there is a short notice at the top with a
few useful informations)<br>
</p><p><br>
</p><p>I'm now reaching out here mostly for :</p>
<ul>
<li>gathering feedback both around the effort (is it worth
pursuing this goal ?) and the implementation (do my
CMakeLists.txt look weird ? yes they do.)</li>
<li>having volunteers using those builds on real-world
workload (which I cannot do at the moment) and finding
bugs</li>
<li>guidance and fixes regarding the CMakeLists.txt files<br>
</li>
</ul><p><br>
</p><p>My next steps could be (of course, subject to feedback) :</p>
<ol>
<li>Have 100% of the cmake implementation finished (all the
missing vehicle_* components) -> easy (only missing
time)</li>
<li>Work on the warnings (currently disabled in the builds)
and try to fix as much as possible<br>
</li>
<li>Iron out all the bugs, improve the dev documentation
-> need help on finding them with real-world use</li>
<li>1st goal is to propose this result as a merge request
(either as a whole, or split depending on feedback -
ideally I would like to have a first (big) MR accepted
quickly so that people can play with it, and deliver
incremental updates as expected)</li>
<li>then work on the v5 and do the same.<br>
</li>
</ol><p><br>
</p><p>(Please note that my cmake skills are inexistent - so be
careful when opening those files to not hurt yourself.)</p><p># The guidelines I tried to follow:</p>
<ul>
<li>Stay compatible with our 3.3.x branch which should still
compile without any issue nor loss of functionality</li>
<li>Keep Makefiles running for 3.3.x (and 4.x if possible)</li>
<li>When possible, isolate differences (between 3.x, 4.x,
5.x) by testing the versions of ESP-IDF.<br>
</li>
<li>Focus on 5.x (if there are impossibilities, let's make
it easier to build v5.x than v4.x)</li>
<li><a href="https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html" target="_blank" rel="noreferrer">https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html</a><br>
</li>
<li><a href="https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system" target="_blank" rel="noreferrer">https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system</a></li>
<li><a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html" target="_blank" rel="noreferrer">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html</a></li>
<li><a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html" target="_blank" rel="noreferrer">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html</a><br>
</li>
</ul><p><br>
</p><p># Missing features:<br>
</p><p>In the course of this work I had to make some decisions to
speed up the work - that will need to be acted upon:</p>
<ul>
<li>I disabled the crash handler
(`xt_set_error_handler_callback` and
`esp_task_wdt_get_trigger_tasknames`) for the moment, we
need to decide whether we "fork" ESP-IDF again to port it
; or if the new APIs are enough to (partially ?)
reimplement it</li>
<li>There is a crash in `<span><span>OvmsConsole::Poll` which I did not
analyse (yet) and which I worked around by declaring a
variable static.</span></span></li>
<li><span><span>I decided to transform our local copies
of `wolfssh` and `wolfssl` in submodules (and move
them one level below in terms of directories) - mainly
to be able to have a CMakeLists.txt different from the
upstream one.<br>
In the process, I "lost" one of our patches : <a href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20" target="_blank" rel="noreferrer">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20</a>
and we need to decide how to handle this.</span></span></li>
<li><span><span>A lot of dependencies are now explicitly
(hard-)coded in the CMakeLists.txt - which may, or may
not be a good thing. Let's discuss it.</span></span></li>
<li><span><span>I had to change a set of defines into a
header generation (in ovms_webserver) because I was
unable to implement those in a satisfying manner in
cmake. If you manage to do it, it's a win ! <br>
</span></span></li>
<li><span><span>Build warnings were disabled to keep the
output clean and focus on the build issues - but now
they need to be re-activated.<br>
</span></span></li>
<li><span><span>etc...</span></span></li>
<li><span><span>Even if there is experimental support
for cmake / idf.py in ESP-IDF 3.3.x, I did not make
any effort to be compatible with it:</span></span></li>
<ul>
<li><span><span>Our own fork does not seem to include
this build system</span></span></li>
<li><span><span>It was experimental, the component API
changed and I did not want to have edge case for
something not used at the moment.<br>
</span></span></li>
</ul>
</ul><p><br>
</p><p>Let me know your comments and questions.<br>
</p><p>Best regards,</p>
</blockquote>
<br>
<pre cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
OvmsDev mailing list
<a href="mailto:OvmsDev@lists.openvehicles.com" target="_blank" rel="noreferrer">OvmsDev@lists.openvehicles.com</a>
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" target="_blank" rel="noreferrer">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote><p><br>
</p>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
OvmsDev mailing list
<a href="mailto:OvmsDev@lists.openvehicles.com" target="_blank" rel="noreferrer">OvmsDev@lists.openvehicles.com</a>
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" target="_blank" rel="noreferrer">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote><p><br>
</p>
<div id="m_5160214521944131701grammalecte_menu_main_button_shadow_host" style="width:0px;height:0px"></div>
</div>
_______________________________________________<br>OvmsDev mailing list<br><a href="mailto:OvmsDev@lists.openvehicles.com" target="_blank" rel="noreferrer">OvmsDev@lists.openvehicles.com</a><br><a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" target="_blank" rel="noreferrer">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br></div></blockquote></div><br></div></div>_______________________________________________<br>
OvmsDev mailing list<br>
<a href="mailto:OvmsDev@lists.openvehicles.com" target="_blank" rel="noreferrer">OvmsDev@lists.openvehicles.com</a><br>
<a href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" rel="noreferrer noreferrer" target="_blank">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a><br>
</blockquote></div>