<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Ludovic,<br>
<br>
really great & appreciated you're going ahead on this. I second
skipping idf 4 and going directly for version 5. I'll try to get my
idf 5 build environment up ASAP.<br>
<br>
I need to get into cmake myself as well. Needing to hard code
dependencies doesn't sound good though, I thought cmake is supposed
to be smart about this.<br>
<br>
I'll have a look at porting the custom crash handler to idf 5. There
were some other additions that haven't been accepted by Espressif,
IIRC. I'll see if I can check them as well.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 23.01.23 um 11:04 schrieb Ludovic
LANGE:<br>
</div>
<blockquote type="cite"
cite="mid:d57c130f-388c-8478-40ac-c292470d006f@lange.nom.fr">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<p>Hi list,</p>
<p>I am interested in upgrading our codebase to a latest ESP-IDF
version (preferentially v5 which has just happened end of 2022 -
Happy New Year ! :-)) - mostly to be able to use some components
for which a backport seems difficult (for example :
<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/752</a>
), and as a matter of balance between stability vs latest
bugfixes, security fixes, new features etc...</p>
<p>In the past it seems that some endeavours already set the basis
to make this happen
(<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/263</a>,
<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/360</a>
).</p>
<p>I'm now trying to revive this effort, and after some work
managed to have:</p>
<ul>
<li>OVMS codebase building under v4.x using Makefile</li>
<li>OVMS codebase building under v4.x using cmake / idf.py (not
100% finished, multiple vehicle_* components are missing)</li>
</ul>
<p>(<i><b>WARNING</b>: some important features of OVMS have been
deactivated for the moment, see end of post)</i><br>
</p>
<p>The produced binaries seem to "work on my <strike>machine</strike>
board", as they end with a "OVMS#" prompt and a few (very) basic
tests look OK (to be honest, unfortunately no real tests have
been done except to check that the binary does not crash during
boot and a few seconds of run, checking that WiFi / web
Interface is OK, sdcard is OK - but very basic)<br>
</p>
<p><br>
</p>
<p>You'll find the changes here :
<a class="moz-txt-link-freetext"
href="https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network"
moz-do-not-send="true">https://github.com/llange/Open-Vehicle-Monitoring-System-3/tree/experimental-esp-idf-v4-rework-network</a><br>
(In the README.md, there is a short notice at the top with a few
useful informations)<br>
</p>
<p><br>
</p>
<p>I'm now reaching out here mostly for :</p>
<ul>
<li>gathering feedback both around the effort (is it worth
pursuing this goal ?) and the implementation (do my
CMakeLists.txt look weird ? yes they do.)</li>
<li>having volunteers using those builds on real-world workload
(which I cannot do at the moment) and finding bugs</li>
<li>guidance and fixes regarding the CMakeLists.txt files<br>
</li>
</ul>
<p><br>
</p>
<p>My next steps could be (of course, subject to feedback) :</p>
<ol>
<li>Have 100% of the cmake implementation finished (all the
missing vehicle_* components) -> easy (only missing time)</li>
<li>Work on the warnings (currently disabled in the builds) and
try to fix as much as possible<br>
</li>
<li>Iron out all the bugs, improve the dev documentation ->
need help on finding them with real-world use</li>
<li>1st goal is to propose this result as a merge request
(either as a whole, or split depending on feedback - ideally I
would like to have a first (big) MR accepted quickly so that
people can play with it, and deliver incremental updates as
expected)</li>
<li>then work on the v5 and do the same.<br>
</li>
</ol>
<p><br>
</p>
<p>(Please note that my cmake skills are inexistent - so be
careful when opening those files to not hurt yourself.)</p>
<p># The guidelines I tried to follow:</p>
<ul>
<li>Stay compatible with our 3.3.x branch which should still
compile without any issue nor loss of functionality</li>
<li>Keep Makefiles running for 3.3.x (and 4.x if possible)</li>
<li>When possible, isolate differences (between 3.x, 4.x, 5.x)
by testing the versions of ESP-IDF.<br>
</li>
<li>Focus on 5.x (if there are impossibilities, let's make it
easier to build v5.x than v4.x)</li>
<li><a class="moz-txt-link-freetext"
href="https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html"
moz-do-not-send="true">https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-reference/network/tcpip_adapter_migration.html</a><br>
</li>
<li><a class="moz-txt-link-freetext"
href="https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system"
moz-do-not-send="true">https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32/api-guides/build-system.html#migrating-from-esp-idf-gnu-make-system</a></li>
<li><a class="moz-txt-link-freetext"
href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html"
moz-do-not-send="true">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.0/index.html</a></li>
<li><a class="moz-txt-link-freetext"
href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html"
moz-do-not-send="true">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/release-5.x/5.1/index.html</a><br>
</li>
</ul>
<p><br>
</p>
<p># Missing features:<br>
</p>
<p>In the course of this work I had to make some decisions to
speed up the work - that will need to be acted upon:</p>
<ul>
<li>I disabled the crash handler
(`xt_set_error_handler_callback` and
`esp_task_wdt_get_trigger_tasknames`) for the moment, we need
to decide whether we "fork" ESP-IDF again to port it ; or if
the new APIs are enough to (partially ?) reimplement it</li>
<li>There is a crash in `<span class="blob-code-inner
blob-code-marker js-code-nav-pass " data-code-marker=" "><span
class="pl-en">OvmsConsole::Poll` which I did not analyse
(yet) and which I worked around by declaring a variable
static.</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">I
decided to transform our local copies of `wolfssh` and
`wolfssl` in submodules (and move them one level below in
terms of directories) - mainly to be able to have a
CMakeLists.txt different from the upstream one.<br>
In the process, I "lost" one of our patches :
<a class="moz-txt-link-freetext"
href="https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20"
moz-do-not-send="true">https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/commit/51444539047daef7bd2accb23ef40d1bc14fdb20</a>
and we need to decide how to handle this.</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">A
lot of dependencies are now explicitly (hard-)coded in the
CMakeLists.txt - which may, or may not be a good thing.
Let's discuss it.</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">I
had to change a set of defines into a header generation
(in ovms_webserver) because I was unable to implement
those in a satisfying manner in cmake. If you manage to do
it, it's a win ! <br>
</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">Build
warnings were disabled to keep the output clean and focus
on the build issues - but now they need to be
re-activated.<br>
</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">etc...</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span class="pl-en">Even
if there is experimental support for cmake / idf.py in
ESP-IDF 3.3.x, I did not make any effort to be compatible
with it:</span></span></li>
<ul>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span
class="pl-en">Our own fork does not seem to include this
build system</span></span></li>
<li><span class="blob-code-inner blob-code-marker
js-code-nav-pass " data-code-marker=" "><span
class="pl-en">It was experimental, the component API
changed and I did not want to have edge case for
something not used at the moment.<br>
</span></span></li>
</ul>
</ul>
<p><br>
</p>
<p>Let me know your comments and questions.<br>
</p>
<p>Best regards,</p>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</body>
</html>