[Ovmsdev] OVMS Poller module/singleton

[Michael Geddes]

OK.

I did some logging changes to confirm that it's only _just_ getting
overwhelmed..ie dropping 1 (or occasionally 2) RX frames in a row but then
picking the next ones up...

So increasing the queue size (I doubled it but maybe that was excessive)
seems to do the trick.  I think the queue size is in  bytes and the queue
packet size has increased at least a little bit.

Michael

On Sat, 4 May 2024, 09:24 Michael Geddes, <frog at bunyip.wheelycreek.net>
wrote:

> Hey Derek,
>
> I do have a friend with a Leaf in his EV collection,  so I can try grab
> some more info from his car and work out exactly what (tf) is going on.
>
> //.
>
> On Sat, 4 May 2024 at 08:13, <frog at bunyip.wheelycreek.net> wrote:
>
>> HI Derek,
>>
>> For a start, I’m very sorry that the ovms put your car in limp mode.
>> That’s quite bad.
>>
>>
>>
>> Just FYI those overflows that you have copied are about receiving and not
>> sending information, so maybe the poller thread is being overwhelmed or
>> stalled somehow.
>>
>>
>>
>> I’ve been looking at the code and wondered if you have the ‘can write’
>> flag set to true or not?  (Though it shouldn’t make that much difference)
>>
>>
>>
>> Even if it were somehow mistakenly polling (ie in the wrong pollstate),
>> it should not be doing it that frequently – (at most once a minute). And
>> the poll throttle would still be on 1 per second.. so that should put a
>> limit  on sends.. and so it is really worrying that somehow that is not
>> happening :/
>>
>>
>>
>> If you have more logs that you send me that don’t just include that
>> particular overflow message that would be good.  Did it crash/reboot?
>> Maybe it crashed repeatedly?
>>
>>
>>
>> //.ichael
>>
>>
>>
>> *From:* OvmsDev <ovmsdev-bounces at lists.openvehicles.com> *On Behalf Of *Derek
>> Caudwell via OvmsDev
>> *Sent:* Friday, May 3, 2024 6:54 PM
>> *To:* OVMS Developers <ovmsdev at lists.openvehicles.com>
>> *Cc:* Derek Caudwell <d.caudwell at gmail.com>
>> *Subject:* Re: [Ovmsdev] OVMS Poller module/singleton
>>
>>
>>
>> Hi Michael,
>>
>>
>>
>> When running firmware  3.3.004-74-gbd4e7196 on my Nissan Leaf I suspect
>> (but can't be 100% sure as it's only been 24h without fault) the new poller
>> caused the car to throw the attached faults from overloading the can bus
>> whilst driving. The fault was sufficient to send the car into limp mode and
>> could not be driven until cleared with LeafSpy. Ovms should not be polling
>> in the driving state, so I'm not sure why the poller is overflowing.
>>
>>
>>
>> The ovms log repeats this overflow in rapid succession right before the
>> failure occurred. I have since reverted to an earlier version and it might
>> be advisable for other Leaf owners to do so until the new poller is fully
>> operational.
>>
>>
>>
>> 2024-05-02 12:22:28.316 NZST I (36847486) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.316 NZST I (36847486) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.316 NZST I (36847486) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.326 NZST I (36847496) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.336 NZST I (36847506) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.336 NZST I (36847506) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.336 NZST I (36847506) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.336 NZST I (36847506) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.346 NZST I (36847516) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.346 NZST I (36847516) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.346 NZST I (36847516) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.346 NZST I (36847516) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.356 NZST I (36847526) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.356 NZST I (36847526) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.356 NZST I (36847526) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>> 2024-05-02 12:22:28.356 NZST I (36847526) vehicle-poll: Poller[Frame]:
>> Task Queue Overflow
>>
>>
>>
>>
>>
>> On Fri, 3 May 2024 at 05:40, Michael Balzer via OvmsDev <
>> ovmsdev at lists.openvehicles.com> wrote:
>>
>> A task may delete itself by calling vTaskDelete(NULL) as the final
>> statement, and that's normally a better way to shutdown a task, if you can
>> signal the task it should terminate. Deleting a task from outside may
>> always result in resources taken by the task not being freed, as the task
>> gets terminated wherever it just is.
>>
>> As the vehicle task also needs to access vehicle data memory, vehicle
>> destruction should not begin until the task has finished, otherwise a task
>> job already running while the vehicle destruction takes place may run into
>> an illegal memory access (access after free). Keep in mind, destructors are
>> executed bottom up, so waiting for the task shutdown within the destructor
>> won't help.
>>
>> Also, I think "MyCan.DeregisterCallback(TAG)" in ShuttingDown() is now
>> obsolete.
>>
>> Regards,
>> Michael
>>
>> Am 02.05.24 um 15:23 schrieb Michael Geddes:
>>
>> Thanks.
>>
>>
>>
>> On that race condition... do you think we are better to just delete the
>> task  or I saw one recommendation that
>>
>> We can delete it outside the while loop in the task itself!
>>
>>
>>
>> //.ichael
>>
>>
>>
>> On Tue, 30 Apr 2024, 17:46 Michael Balzer via OvmsDev, <
>> ovmsdev at lists.openvehicles.com> wrote:
>>
>> OK, understood & merged, thanks for the quick fix/workaround.
>>
>> I'll ask the Twizy driver to test this.
>>
>> Regards,
>> Michael
>>
>> PS: side note: there's a shutdown race condition arising from `while
>> (!m_is_shutdown)` in the new `OvmsVehicle::VehicleTask()`: if the task
>> receives a message while m_is_shutdown already has been set, it will exit
>> the loop and return from the task function, which isn't allowed. I think
>> FreeRTOS will abort in that case.
>>
>>
>> Am 30.04.24 um 08:55 schrieb Michael Geddes:
>>
>> No it will start the poll thread of the poller fine for those that don't
>> use it.  That's the last commit.... so that should be covered. When the
>> callback is registered the thread is force started.
>>
>>
>>
>>
>>
>> This is not to say that what you are suggesting isn't a better way
>> forward... just that it should work.
>>
>>
>>
>> Michael
>>
>>
>>
>>
>>
>> On Tue, 30 Apr 2024, 14:51 Michael Balzer via OvmsDev, <
>> ovmsdev at lists.openvehicles.com> wrote:
>>
>> Michael,
>>
>> (taking this back to the list, as other developers may have helpful ideas
>> or suggestions)
>>
>> from a first check, your new patch (PR #1008) won't help for vehicles
>> that don't use the poller, like the generic DBC vehicle or the Fiat 500
>> (plus possibly non-public third party vehicles). As the poller is normally
>> compiled in, these would need a run time switch to re-enable the vehicle
>> task, or more suitably default into that configuration and switch to the
>> poller task only when the poller gets initialized.
>>
>> How about moving the task back into the vehicle class, but keeping your
>> task message extensions while doing so, and adding a way for the poller to
>> hook into the task? The dedicated vehicle task is an essential part of the
>> vehicle framework, but I do remember some occasions where a general way to
>> pass custom messages to the vehicle task would have been helpful to use the
>> task for more than CAN processing.
>>
>> If the poller shall become avaible as a separate service that can be used
>> without any vehicle instance (currently not a defined use case), the
>> construct of a CAN processor task that can be extended for custom messages
>> (or a message processor task that can subscribe to CAN frames) could be
>> factored out into a dedicated class or template. The poller could then
>> create his own instance of that class, if it cannot hook into an existing
>> one.
>>
>> Btw, in case you're not aware of this, FreeRTOS also provides queue sets (
>> https://www.freertos.org/RTOS-queue-sets.html). We haven't used them in
>> the OVMS yet, but they could be useful, especially if message unions become
>> large or tasks shall be able to dynamically subscribe to different message
>> sources.
>>
>> Regards,
>> Michael
>>
>> Am 30.04.24 um 01:20 schrieb Michael Geddes:
>>
>> It might be worth reverting, but, I've got a patch suggestion that I'll
>> push up which will let me know if I understand everything and which might
>> provide a solution.
>>
>>
>>
>> If this isn't going to work then revert. (P/R coming)
>>
>>
>>
>> While the poller was still a part of the vehicle class it was probably
>> still all ok.   The poller had taken over what I assume you are talking
>> about as the vehicle task.  A call-back fromthe poller was calling
>> IncomingPollRxFrame which was then coming from the (now) poller task (is
>> that correct?)
>>
>>
>>
>> While we have OVMS_COMP_POLLER config defined, we could just use the
>> poller task to provide the IncomingPollRxFrame call-back from the poller
>> (was vehicle?) task.
>>
>>
>>
>> The problem is when OVMS_COMP_POLLER is undefined, we need an alternate
>> (maybe we could use the 'Event' loop then) which is when I hacked that bad
>> solution.
>>
>>
>>
>>
>>
>> //.ichael
>>
>>
>>
>>
>>
>> I was thinking though that because everything is being queued we could
>> divert some calls into the car
>>
>> On Mon, 29 Apr 2024, 18:58 Michael Balzer, <dexter at expeedo.de> wrote:
>>
>> Michael,
>>
>> I've found a severe design flaw with your poller change. Sorry, I should
>> have seen that before, and shouldn't have merged this.
>>
>> You have moved the standard CAN frame processing from the vehicle task
>> into the CAN task by changing the vehicle from a CAN listener to a CAN
>> callback processor. That will break all kind of things, vehicles like the
>> Twizy, Smart and many others rely on frame processing being done in the
>> dedicated vehicle task.
>>
>> This especially induces issues regarding CAN processing capacity, as the
>> vehicles rely on having a separate context and not needing to decouple
>> complex processing of incoming frames. So this will degrade the CAN
>> processing performance seriously in many cases, where additional steps
>> involving file or network I/O need to be done.
>>
>> So this needs to be changed back.
>>
>> I assumed you introduced a new dedicated poller task but kept the vehicle
>> task intact. From your naming (OvmsVehicle::IncomingPollRxFrame), it seems
>> you misinterpreted the vehicle task's purpose as only being used for
>> polling.
>>
>> I assume this isn't a small change…? If so we should revert the poller
>> merge, to avoid having a defunctional edge build state until the fix.
>>
>>
>> Secondly: logging is generally expensive regardless of the log level
>> control and log channels enabled. Any log message needs to be queued to the
>> log system, so involves a lock and a potential wait state (for the queue) &
>> resulting context switch (= minimum delay of 10 ms). Therefore, logging
>> must be avoided as far as possible in any time critical context, and is
>> forbidden under some circumstances, e.g. in a timer callback (see FreeRTOS
>> docs on this). The log system load multiplies with connected web clients or
>> enabled file logging and enabled debug / verbose level logging -- that
>> quickly becomes too much, usually triggering the task watchdog.
>>
>> Your logging of nearly all bus events passing to/from the poller
>> (especially the log entry in Queue_PollerFrame) becomes an issue on any
>> vehicle that has high frequency running process data frames on the bus,
>> like the Twizy or Smart. As a counter measure, I've just added a runtime
>> control for all the poller's verbose logging (by default now off), and
>> changed some debug level logs to verbose. Not sure if I've catched all that
>> may need to be silenced by default. Please check all your log calls and
>> place them under the new "trace" control flag wherever appropriate.
>>
>> This won't help avoiding issues with process data frame buses though.
>>
>>
>> Shall I revert the poller merge for now?
>>
>> Regards,
>> Michael
>>
>>
>> Am 29.04.24 um 06:18 schrieb Michael Geddes:
>>
>> Btw I included the log changes in the p/r which is a few small commits
>> for the poller.
>>
>>
>>
>> //.
>>
>>
>>
>> On Mon, 29 Apr 2024, 07:20 Michael Geddes, <frog at bunyip.wheelycreek.net>
>> wrote:
>>
>> Hey Michael,
>>
>> I've got to work now (it's Monday), but I suspect those 'giving up' are
>> from unsolicited messages on the bus.
>>
>> I can re-order things so that the message will likely be  'dropped (no
>> poll entry)' rather than the time-out message.
>>
>> And make it a verbose log.
>>
>>
>>
>> //.ichael
>>
>>
>>
>> On Mon, 29 Apr 2024 at 00:25, Michael Balzer <dexter at expeedo.de> wrote:
>>
>> Michael,
>>
>> forwarding the Twizy logs to you directly, as they contain the user
>> location.
>>
>> He has just verified it's the new version, he says the module stops
>> responding as soon as he turns on the Twizy.
>>
>> His description of his actions is a bit ambiguous, and it seems he didn't
>> enable logging to the file persistently.
>>
>> According to the server, these were the version boot times:
>>
>> 2024-04-28 15:30:56    0    3.3.004-32-g125e0841/ota_0/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 26 2024 19:16:50)
>> 2024-04-28 16:21:08    0    3.3.004-32-g125e0841/ota_0/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 26 2024 19:16:50)
>> 2024-04-28 16:38:33    0    3.3.004-63-gf3595561/ota_1/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 27 2024 07:44:50)
>> 2024-04-28 16:40:57    0    3.3.004-63-gf3595561/ota_1/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 27 2024 07:44:50)
>> 2024-04-28 16:43:14    0    3.3.004-63-gf3595561/ota_1/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 27 2024 07:44:50)
>> 2024-04-28 16:46:39    0    3.3.004-63-gf3595561/ota_1/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 27 2024 07:44:50)
>> 2024-04-28 16:54:44    0    3.3.004-32-g125e0841/ota_0/edge (build idf
>> v3.3.4-849-g6e214dc335 Apr 26 2024 19:16:50)
>>
>> Attached is also his crash debug log -- a2ll doesn't tell much about what
>> happened, but maybe you get an idea from this.
>>
>> After the boot at 16:46, there are immediately lots of these messages:
>>
>> 2024-04-28 16:46:33.792 CEST D (39042) vehicle-poll: Poller: Queue
>> PollerFrame()
>> 2024-04-28 16:46:33.792 CEST D (39042) vehicle-poll: [1]Poller: Incoming
>> - giving up
>>
>> Regards,
>> Michael
>>
>> Am 28.04.24 um 16:44 schrieb Michael Geddes:
>>
>> Ah. OK.
>>
>>
>>
>> I could try to fix the vin thing using the new way of doing it and get
>> rid of a semaphore?
>>
>> It would at least identify the problem possibly?
>>
>>
>>
>> Michael
>>
>>
>>
>>
>>
>> On Sun, 28 Apr 2024, 22:32 Michael Balzer via OvmsDev, <
>> ovmsdev at lists.openvehicles.com> wrote:
>>
>> Not sure if that's the problem, but I've found a different behaviour with
>> the new PollSetState() implementation.
>>
>> The old version only did anything if the new state actually was different
>> from the previous one. The Twizy relies on this behaviour, calling
>> PollSetState() from the per second ticker (see
>> OvmsVehicleRenaultTwizy::ObdTicker1).
>>
>> The new implementation apparently always sends the PollState command to
>> the task, and that in turn always at least locks the poller mutex. Not sure
>> if/how that could cause the observed issues, but it definitely adds quite
>> some (unnecessary?) lock/unlock operations.
>>
>>
>> Am 28.04.24 um 16:05 schrieb Michael Balzer via OvmsDev:
>>
>> The Twizy uses the poller to query the VIN (once) and DTCs (every 10
>> seconds while driving), see rt_obd2.cpp.
>>
>> It also has its own version of the OBD single request
>> (OvmsVehicleRenaultTwizy::ObdRequest), which was the precursor for the
>> generalized version. This is used by custom/user Twizy plugins and scripts
>> to access ECU internals.
>>
>> The Twizy doesn't use IncomingPollRxFrame, but the Twizy's
>> IncomingPollReply handler will log any poll responses it doesn't know
>> about, so that could lead to a lot of log output if something goes wrong
>> there.
>>
>> Am 28.04.24 um 15:49 schrieb Michael Geddes via OvmsDev:
>>
>> AFAICT the twizzy doesn't use the poller list at all.  So is it missing a
>> call-back or something??
>>
>>
>>
>> I can see a potential problem with IncomingPollRxFrame being called twice
>> as much as it should be but only when there is a poll list.  Maybe
>> commenting out this would do it.  (I can find another away to get this
>> called on the thread I want).  This might be the problem with the smarted
>>
>>
>>
>> void OvmsVehicle::OvmsVehicleSignal::IncomingPollRxFrame(canbus* bus,
>> CAN_frame_t *frame, bool success)
>>   {
>>   //if (Ready())
>>   //  m_parent->IncomingPollRxFrame(frame, success);
>>   }
>>
>>
>>
>> //.
>>
>>
>>
>>
>>
>> On Sun, 28 Apr 2024 at 21:10, Michael Balzer via OvmsDev <
>> ovmsdev at lists.openvehicles.com> wrote:
>>
>> There may also be an issue with the Renault Twizy, I've received a report
>> of a user who is using the edge builds, that the latest build wouldn't work.
>>
>> He reports all kinds of errors and warnings signaled by the car during
>> driving, and switching back to the previous build fixed the issues.
>>
>> I've asked him to provide a debug log excerpt if possible.
>>
>> Regards,
>> Michael
>>
>> Am 28.04.24 um 14:29 schrieb Michael Geddes via OvmsDev:
>>
>> OK. That's bad.
>>
>>
>>
>> Does the reading work in general?
>>
>>
>>
>> Is it just the writing commands?
>>
>>
>>
>> Raise a ticket on github and tag me in and we can address it that way.
>>
>>
>>
>> Michael
>>
>>
>>
>> On Sun, 28 Apr 2024, 19:49 Thomas Heuer via OvmsDev, <
>> ovmsdev at lists.openvehicles.com> wrote:
>>
>> Hi,
>>
>>
>>
>> The new poller code doesn't seem to work properly with the smarted.
>>
>> D (218831) vehicle-poll: [1]PollerNextTick(PRI): cycle complete for
>> ticker=215
>>
>> V (218831) vehicle-poll: Standard Poll Series: List reset
>>
>> D (218831) vehicle-poll: PollSeriesList::NextPollEntry[!v.standard]:
>> ReachedEnd
>>
>> V (218831) vehicle-poll: [1]PollerSend: Poller Reached End
>>
>> D (219691) vehicle-poll: Poller: Queue PollerFrame()
>>
>> D (219691) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219691) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219691) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219691) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> V (219691) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219691) vehicle-poll: Poller: Queue PollerFrame()
>>
>> *OVMS#* unlock 22
>> Vehicle unlocked
>>
>> V (219691) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219691) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219691) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>> V (219701) vehicle-poll: Pollers: FrameRx(bus=2)
>>
>> D (219701) vehicle-poll: Poller: Queue PollerFrame()
>>
>>
>>
>>
>>
>> *Von:* OvmsDev <ovmsdev-bounces at lists.openvehicles.com> *Im Auftrag von *Michael
>> Geddes via OvmsDev
>> *Gesendet:* Sonntag, 28. April 2024 12:27
>> *An:* OVMS Developers <ovmsdev at lists.openvehicles.com>
>> *Cc:* Michael Geddes <frog at bunyip.wheelycreek.net>
>> *Betreff:* [Ovmsdev] OVMS Poller module/singleton
>>
>>
>>
>> Hey all,
>>
>>
>>
>> The poller singleton code that I've been working on for over a year now
>> is merged in. (Thanks Michael for expediting the final step).
>>
>>
>>
>> This includes separate multi-frame states per bus and multiple poll lists
>> as well as non-blocking one off queries. As well as more 'states'.
>>
>>
>>
>> I have included some programming documentation in the change but am happy
>> to supply more if needed.
>>
>>
>>
>> The ioniq 5 code has some examples of how it can be used. Some examples
>> are:
>>
>>
>>
>> * grabbing the vin as a one shot without blocking
>>
>> * having a short list of queries that are polled quickly for obd2ecu
>> (this also demonstrates using a shorter frame break value and then a break
>> after successful a response)
>>
>>
>>
>> Have a play please!
>>
>>
>>
>> Also interested in hearing what user tools might be worth looking at next
>> for the poller object.
>>
>>
>>
>> //.ichael G.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> _______________________________________________
>>
>> OvmsDev mailing list
>>
>> OvmsDev at lists.openvehicles.com
>>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> _______________________________________________
>>
>> OvmsDev mailing list
>>
>> OvmsDev at lists.openvehicles.com
>>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>>
>>
>> _______________________________________________
>>
>> OvmsDev mailing list
>>
>> OvmsDev at lists.openvehicles.com
>>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
>>
>> --
>>
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20240505/ff7af306/attachment-0001.htm>