[Ovmsdev] Auto start/init
Mark Webb-Johnson
mark at webb-johnson.net
Sun Mar 11 21:32:26 HKT 2018
I guess it is possible, but probably easiest to just fallback to factory (rather than the other OTA partition).
I suggest something like if auto is inhibited, and we’ve had five further crashes within 10 seconds of boot, and we are running OTA, then switch back to factory.
Regards, Mark
> On 9 Mar 2018, at 11:29 PM, Michael Balzer <dexter at expeedo.de> wrote:
>
> Should we extend that logic to also cover firmware updates?
>
> I.e. if the system can't get to the stable state after disabling auto init, we could try to activate the other ota partition with a fallback to factory.
>
> Regards,
> Michael
>
>
> Am 09.03.2018 um 07:14 schrieb Mark Webb-Johnson:
>> Looks good to me. Working well in simulations (module fault).
>>
>> Regards, Mark.
>>
>>> On 9 Mar 2018, at 6:54 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>
>>> Done:
>>>
>>> E (1226) housekeeping: Auto init inhibited: too many early crashes (6)
>>> …
>>> OVMS > boot status
>>> Last boot was 71 second(s) ago
>>> This is reset #6 since last power cycle
>>> Detected boot reason: EarlyCrash
>>> Crash counters: 6 total, 6 early
>>> CPU#0 boot reason was 12
>>> CPU#1 boot reason was 12
>>>
>>>
>>> A manual reset by "module reset" is detected as a soft reboot and resets all counters like a hard reset. Hard resets (i.e. ^T^R in the monitor) look just like a normal power on.
>>>
>>> Regards,
>>> Michael
>>>
>>>
>>> Am 08.03.2018 um 04:55 schrieb Mark Webb-Johnson:
>>>>> I wasn't aware of the option to add custom segments in there.
>>>>
>>>> Me neither. It was fun finding out how to do this. I learned a lot about GCC linker scripts and how the Espressif IDF framework uses them. Also learned that the error messages coming out of the linker are so obscure as to be useless.
>>>>
>>>> I think we should be able to store the reason for a crash in that area, then after reboot submit it as a notification to the servers (in much the same way we did for OVMS v2, but more powerful as we have the reason before the crash, not just the crash itself). It seems that some of the system crash handlers can be overridden (such as void __attribute__((weak)) vApplicationStackOverflowHook).
>>>>
>>>>> I'll optimize the auto start logic after finishing the webserver fixes.
>>>>
>>>> Thanks.
>>>>
>>>> Regards, Mark.
>>>>
>>>>> On 8 Mar 2018, at 1:25 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>>>
>>>>> Mark,
>>>>>
>>>>> I also had the problem of losing auto start when rebooting within 10 seconds. I was about to add more config instances to track the state, but having the RTC RAM for this is much better. I wasn't aware of the option to add custom segments in there.
>>>>>
>>>>> I'll optimize the auto start logic after finishing the webserver fixes.
>>>>>
>>>>> Thanks,
>>>>> Michael
>>>>>
>>>>>
>>>>> Am 07.03.2018 um 15:39 schrieb Mark Webb-Johnson:
>>>>>> I’m having problems with this approach. Specifically, permanently disabling the auto start system if the system fails once during a boot.
>>>>>>
>>>>>> I tried the module in my car today. Plugged it in, but the access point didn’t come up. No laptop, just an iPad.
>>>>>>
>>>>>> It seems that while I was plugging it in, the power jiggled. So it booted for a second or so, reset, then booted again. It treated this as a problem, so turned off the auto init system permanently. No matter how many times I reset, I couldn’t get an access point to come up, and couldn’t get into the system from the iPad.
>>>>>>
>>>>>> I’m not sure that storing this in config (fat filesystem) is optimal. I think the core reason for that was because the ESP IDF framework wipes RTC memory on boot for anything apart from a deep sleep. So, I set about finding a solution to that core problem. The result is quite neat, and visible in main/component.mk, and main/ovms_boot.*. It turns out that the ESP framework (start_cpu0) wipes the RTC BSS segment only. It leaves other RTC data untouched. So...
>>>>>>
>>>>>> I create a linker section ‘.rtc.noload’. This goes in RTC slow ram, alongside the other RTC bss sections. But the NOLOAD flag is specified, to stop GCC from wiping this memory on boot.
>>>>>>
>>>>>> I then create a typedef struct boot_data_t and storage boot_data to store the actual boot data. It is tagged __attribute__((section(".rtc.noload"))) to force it into that NOLOAD section. I’ve now got some data that never gets initialised, and survives a reset.
>>>>>>
>>>>>> The last step is to check the actual boot reason (using rtc_get_reset_reason). If it is POWERON_RESET then I zero the boot_data storage (otherwise it is truly garbage - perhaps good for random number initialisation, but not much else).
>>>>>>
>>>>>> It seems to work well for me, and is a quite neat solution. I added a ‘boot status’ command to show the status of the last boot, including the count of reboots.
>>>>>>
>>>>>> Regarding differentiating between ‘module reset’ and another type of reset, I think we can now simply add a bool to boot_data_t and have ‘module reset’ set it before resetting. It can then be picked up during boot (reason SW_CPU_RESET, and check for this flag) to pickup on this specific case. If it is a SW_CPU_RESET and this flag is not set, then we can assume it was a crash.
>>>>>>
>>>>>> With this, I think we can do some more sophisticated logic. The ‘first ten seconds’ flag goes in boot_data_t. We can also keep a count of failure resets during first 10 seconds, and if it exceeds say 5 then turn off auto init (but with a setting just in ram, leaving the config 'auto init' to indicate whether the user wants init to be auto or not).
>>>>>>
>>>>>> Workable?
>>>>>>
>>>>>> Regards, Mark.
>>>>>>
>>>>>>> On 26 Feb 2018, at 3:51 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>>>>>
>>>>>>> First version of the auto init system is in the hub.
>>>>>>>
>>>>>>> As discussed, I added an "auto" config param. It currently has these instances:
>>>>>>>
>>>>>>> Instance Default Remark
>>>>>>> ----------------------------------------------------------------------
>>>>>>> init yes
>>>>>>>
>>>>>>> wifi.mode ap off|ap|client
>>>>>>> wifi.ssid.ap OVMS
>>>>>>> wifi.ssid.client - empty = scanning mode
>>>>>>>
>>>>>>> modem no
>>>>>>>
>>>>>>> vehicle.type - moved from vehicle/type
>>>>>>>
>>>>>>> server.v2 no
>>>>>>> server.v3 no
>>>>>>>
>>>>>>>
>>>>>>> The web UI has a new config page for this as well.
>>>>>>>
>>>>>>> On boot, the housekeeper temporarily sets "init" to false. 10 seconds after booting has finished it re-enables it, assuming the system is stable.
>>>>>>>
>>>>>>> The auto wifi ap mode does a fallback to "OVMS" with the module password if no AP network is configured. It does not start the AP mode if the password is empty.
>>>>>>> So in order to enable the AP mode for configuration, it's sufficient to set the module password and leave all auto configs to their defaults.
>>>>>>>
>>>>>>>
>>>>>>> Unfortunately a crash reboot seems to be indistinguishable from a manual reset. I've added a log output of the reset reason provided by the ESP. The reason
>>>>>>> codes are:
>>>>>>>
>>>>>>> typedef enum {
>>>>>>> NO_MEAN = 0,
>>>>>>> POWERON_RESET = 1, /**<1, Vbat power on reset*/
>>>>>>> SW_RESET = 3, /**<3, Software reset digital core*/
>>>>>>> OWDT_RESET = 4, /**<4, Legacy watch dog reset digital core*/
>>>>>>> DEEPSLEEP_RESET = 5, /**<3, Deep Sleep reset digital core*/
>>>>>>> SDIO_RESET = 6, /**<6, Reset by SLC module, reset digital core*/
>>>>>>> TG0WDT_SYS_RESET = 7, /**<7, Timer Group0 Watch dog reset digital core*/
>>>>>>> TG1WDT_SYS_RESET = 8, /**<8, Timer Group1 Watch dog reset digital core*/
>>>>>>> RTCWDT_SYS_RESET = 9, /**<9, RTC Watch dog Reset digital core*/
>>>>>>> INTRUSION_RESET = 10, /**<10, Instrusion tested to reset CPU*/
>>>>>>> TGWDT_CPU_RESET = 11, /**<11, Time Group reset CPU*/
>>>>>>> SW_CPU_RESET = 12, /**<12, Software reset CPU*/
>>>>>>> RTCWDT_CPU_RESET = 13, /**<13, RTC Watch dog Reset CPU*/
>>>>>>> EXT_CPU_RESET = 14, /**<14, for APP CPU, reseted by PRO CPU*/
>>>>>>> RTCWDT_BROWN_OUT_RESET = 15, /**<15, Reset when the vdd voltage is not stable*/
>>>>>>> RTCWDT_RTC_RESET = 16 /**<16, RTC Watch dog reset digital core and rtc module*/
>>>>>>> } RESET_REASON;
>>>>>>>
>>>>>>> I get these values:
>>>>>>>
>>>>>>> …boot after make app-flash:
>>>>>>> I (537) housekeeping: reset_reason: cpu0=1, cpu1=14
>>>>>>>
>>>>>>> …after triggering vfs-edit-bug:
>>>>>>> I (527) housekeeping: reset_reason: cpu0=12, cpu1=12
>>>>>>>
>>>>>>> …after "module reset":
>>>>>>> I (527) housekeeping: reset_reason: cpu0=12, cpu1=12
>>>>>>>
>>>>>>> …after "module fault":
>>>>>>> I (527) housekeeping: reset_reason: cpu0=12, cpu1=12
>>>>>>>
>>>>>>> …and no, the 12/12 were not sticky, I checked with a clean boot.
>>>>>>>
>>>>>>> There's also a bug in the ESP32 that may trigger false positives: https://github.com/espressif/esp-idf/issues/494#issuecomment-291916959 <https://github.com/espressif/esp-idf/issues/494#issuecomment-291916959>
>>>>>>>
>>>>>>> I haven't had that effect up to now, but it seems we cannot rely on the reset reason.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Michael
>>>>>>>
>>>>>>>
>>>>>>> Am 25.02.2018 um 15:06 schrieb Michael Balzer:
>>>>>>>> Am 22.02.2018 um 04:22 schrieb Mark Webb-Johnson:
>>>>>>>>> When looking at the deep sleep stuff, I found there is a way to tag a global variable as ‘rtc’. That should survive a deep sleep (or reboot).
>>>>>>>> Unfortunately, it does not survive a reboot:
>>>>>>>>
>>>>>>>>> * Data in RTC memory is initialised whenever the SoC restarts, except when waking from deep sleep. When waking from deep sleep, the values which were present
>>>>>>>>> before going to sleep are kept.
>>>>>>>> There is an NVS library: https://esp-idf.readthedocs.io/en/v1.0/api/nvs_flash.html <https://esp-idf.readthedocs.io/en/v1.0/api/nvs_flash.html>
>>>>>>>>
>>>>>>>> …but I've got the impression that could interfere with our own flash usage (?).
>>>>>>>>
>>>>>>>> Also, if flash is the way and as /store is already present at housekeeping init, I think we can just as well use our own config module…
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Michael
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OvmsDev mailing list
>>>>>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OvmsDev mailing list
>>>>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>>>
>>>>> --
>>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>> _______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>>
>>> --
>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>>
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.teslaclub.hk <mailto:OvmsDev at lists.teslaclub.hk>
>> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
>
> --
> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.teslaclub.hk
> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20180311/79da8ae3/attachment.htm>
More information about the OvmsDev
mailing list