<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi list,</p>
<p>I am interested in upgrading our codebase to a latest ESP-IDF
version (preferentially v5 which has just happened end of 2022 -
Happy New Year ! :-)) - mostly to be able to use some components
for which a backport seems difficult (for example :
<a class="moz-txt-link-freetext" href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752</a>
), and as a matter of balance between stability vs latest
bugfixes, security fixes, new features etc...</p>
<p>In the past it seems that some endeavours already set the basis
to make this happen
(<a class="moz-txt-link-freetext" href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263</a>,
<a class="moz-txt-link-freetext" href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360</a>
).</p>
<p>I'm now trying to revive this effort, and after some work managed
to have:</p>
<ul>
<li>OVMS codebase building under v4.x using Makefile</li>
<li>OVMS codebase building under v4.x using cmake / idf.py (not
100% finished, multiple vehicle_* components are missing)</li>
</ul>
<p>(<i><b>WARNING</b>: some important features of OVMS have been
deactivated for the moment, see end of post)</i><br>
</p>
<p>The produced binaries seem to "work on my <strike>machine</strike>
board", as they end with a "OVMS#" prompt and a few (very) basic
tests look OK (to be honest, unfortunately no real tests have been
done except to check that the binary does not crash during boot
and a few seconds of run, checking that WiFi / web Interface is
OK, sdcard is OK - but very basic)<br>
</p>
<p><br>
</p>
<p>You'll find the changes here :
<a class="moz-txt-link-freetext" href="https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network">https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network</a><br>
(In the README.md, there is a short notice at the top with a few
useful informations)<br>
</p>
<p><br>
</p>
<p>I'm now reaching out here mostly for :</p>
<ul>
<li>gathering feedback both around the effort (is it worth
pursuing this goal ?) and the implementation (do my
CMakeLists.txt look weird ? yes they do.)</li>
<li>having volunteers using those builds on real-world workload
(which I cannot do at the moment) and finding bugs</li>
<li>guidance and fixes regarding the CMakeLists.txt files<br>
</li>
</ul>
<p><br>
</p>
<p>My next steps could be (of course, subject to feedback) :</p>
<ol>
<li>Have 100% of the cmake implementation finished (all the
missing vehicle_* components) -> easy (only missing time)</li>
<li>Work on the warnings (currently disabled in the builds) and
try to fix as much as possible<br>
</li>
<li>Iron out all the bugs, improve the dev documentation ->
need help on finding them with real-world use</li>
<li>1st goal is to propose this result as a merge request (either
as a whole, or split depending on feedback - ideally I would
like to have a first (big) MR accepted quickly so that people
can play with it, and deliver incremental updates as expected)</li>
<li>then work on the v5 and do the same.<br>
</li>
</ol>
<p><br>
</p>
<p>(Please note that my cmake skills are inexistent - so be careful
when opening those files to not hurt yourself.)</p>
<p># The guidelines I tried to follow:</p>
<ul>
<li>Stay compatible with our 3.3.x branch which should still
compile without any issue nor loss of functionality</li>
<li>Keep Makefiles running for 3.3.x (and 4.x if possible)</li>
<li>When possible, isolate differences (between 3.x, 4.x, 5.x) by
testing the versions of ESP-IDF.<br>
</li>
<li>Focus on 5.x (if there are impossibilities, let's make it
easier to build v5.x than v4.x)</li>
<li><a class="moz-txt-link-freetext" href="https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html">https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html</a><br>
</li>
<li><a class="moz-txt-link-freetext" href="https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system">https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system</a></li>
<li><a class="moz-txt-link-freetext" href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html</a></li>
<li><a class="moz-txt-link-freetext" href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html</a><br>
</li>
</ul>
<p><br>
</p>
<p># Missing features:<br>
</p>
<p>In the course of this work I had to make some decisions to speed
up the work - that will need to be acted upon:</p>
<ul>
<li>I disabled the crash handler (`xt_set_error_handler_callback`
and `esp_task_wdt_get_trigger_tasknames`) for the moment, we
need to decide whether we "fork" ESP-IDF again to port it ; or
if the new APIs are enough to (partially ?) reimplement it</li>
<li>There is a crash in `<span class="blob-code-inner
blob-code-marker js-code-nav-pass " data-code-marker=" "><span
class="pl-en">OvmsConsole::Poll` which I did not analyse
(yet) and which I worked around by declaring a variable
static.</span></span></li>
<li><span class="blob-code-inner blob-code-marker js-code-nav-pass
" data-code-marker=" "><span class="pl-en">I decided to
transform our local copies of `wolfssh` and `wolfssl` in
submodules (and move them one level below in terms of
directories) - mainly to be able to have a CMakeLists.txt
different from the upstream one.<br>
In the process, I "lost" one of our patches :
<a class="moz-txt-link-freetext" href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20</a>
and we need to decide how to handle this.</span></span></li>
<li><span class="blob-code-inner blob-code-marker js-code-nav-pass
" data-code-marker=" "><span class="pl-en">A lot of
dependencies are now explicitly (hard-)coded in the
CMakeLists.txt - which may, or may not be a good thing.
Let's discuss it.</span></span></li>
<li><span class="blob-code-inner blob-code-marker js-code-nav-pass
" data-code-marker=" "><span class="pl-en">I had to change a
set of defines into a header generation (in ovms_webserver)
because I was unable to implement those in a satisfying
manner in cmake. If you manage to do it, it's a win ! <br>
</span></span></li>
<li><span class="blob-code-inner blob-code-marker js-code-nav-pass
" data-code-marker=" "><span class="pl-en">Build warnings were
disabled to keep the output clean and focus on the build
issues - but now they need to be re-activated.<br>
</span></span></li>
<li><span class="blob-code-inner blob-code-marker js-code-nav-pass
" data-code-marker=" "><span class="pl-en">etc...</span></span></li>
<li><span class="blob-code-inner blob-code-marker js-code-nav-pass
" data-code-marker=" "><span class="pl-en">Even if there is
experimental support for cmake / idf.py in ESP-IDF 3.3.x, I
did not make any effort to be compatible with it:</span></span></li>
<ul>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">Our
own fork does not seem to include this build system</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">It
was experimental, the component API changed and I did not
want to have edge case for something not used at the
moment.<br>
</span></span></li>
</ul>
</ul>
<p><br>
</p>
<p>Let me know your comments and questions.<br>
</p>
<p>Best regards,</p>
<div id="grammalecte_menu_main_button_shadow_host" style="width:
0px; height: 0px;"></div>
</body>
</html>